360Photography

Lytro is bringing its Light Field technology to the world of cinema and visual effects, shortly after its CEO announced in a blog post Lytro’s intention of abandoning the consumer stills camera space. Lytro Cinema turns every frame of live action into a 3D model, capturing intensity, color, and angular information of light rays. Coupling light field with a 755 MP sensor capable of capturing images at 300 fps, Lytro Cinema promises extensive post-production freedom, including adjustment of focus, depth-of-field, shutter speed and frame rate, as well as the elimination of green screens.

Although Lytro experienced some difficulty in adoption of light field technology in stills, the technology had, and continues to have, immense potential for imaging. Saving creative decisions for post-processing allows for more creative freedom, and allows a photographer or DP to focus on other elements during capture. Nowhere will this be more appreciated than in cinema, where the realities of production mean that any technology aimed at saving certain creative decisions, like focus, for post-capture are welcome.

Focus and aperture sliders in post-production. In video. No joke. I wish my Raw converter had this (Lytro’s Raw converter already does). Photo credit: Lytro

And that’s exactly what Lytro Cinema aims to do. By capturing directional information about light rays and essentially sampling multiple perspectives behind the aperture, Lytro Cinema allows for adjustment of focus placement, depth-of-field (via aperture adjustment), perspective, and more in post-processing. And since a depth map is rendered for every frame of video, Lytro claims Cinema will make it easier to combine CGI with live footage, no longer requiring green screens to extract elements or subjects from a scene. You’ll be able to just extract a subject based on its depth, which Lytro shows in a convincing example below:

Since light field cameras effectively sample multiple perspectives from behind the lens, you can even simulate camera movement as if it were moved on-set. The degree of motion is of course limited, but the technique can be very effective, as demonstrated in this haunting music video shot entirely on the stills-focused Lytro Illum. As Lead Engineer for Light Field Video Brendan Bevensee explains: “You have a virtual camera that can be controlled in post-production.” That means there’s also nothing stopping one from simulating short dolly motion or perspective shifts in post, with nothing but a static camera at the time of capture. “You can shift the camera to the left… [or] to the right, as if you had made that exact decision on set. It can even move your camera in and out” says Head of Light Field Video, Jon Karafin.

Furthermore, by precisely recording X, Y, Z, pitch, roll, and yaw, Lytro Cinema even offers automated camera tracking, which makes it easier to composite and mat CGI elements. And just as the Illum paired with Lytro Desktop software allowed one to select various objects and depths to throw them in and out of focus for selective depth-of-field and background blur, one can do the same in video with the Cinema, choosing, for example, to marry live footage from minimum focus to, say, 10m with different footage, or CGI, for everything beyond those distances. In other words, control over not just single planes, but ranges of planes.

Beyond just light field benefits, Lytro is also addressing another common headache: the selection of shutter angle (or shutter speed). Often, this is a decision made at the time of capture, dictating the level of blur or stuttering (a la action scenes in ‘Saving Private Ryan’ or ‘Gladiator’) in your footage. At high frame rates of capture, though, high shutter angles are required, removing some of the flexibility of how much motion blur you can or can’t have (e.g. 300 fps cannot be shot with shutter speeds longer than 1/300s, which inevitably freezes action). By decoupling the shutter angle of capture from the shutter angle required for artistic effect, a DP can creatively use motion blur, or lack thereof, to suit the story. The technology, which undoubtedly uses some form of interpolation and averaging in conjunction with the temporal oversampling, also means that you can extract stills with a desired level of motion blur. 

Lytro claims that by capturing at 300 fps, they can computationally allow for any of a number of shutter angles in post-production, allowing a cinematographer to decouple shutter angle required for capture from that required for artistic intent. Photo credit: Lytro

With every development over at Lytro, we’ve been excited by the implications for both stills and video. The implications for the latter, in particular, have always been compelling. Along with the announcement of the Lytro Immerge 360º virtual reality light field rig, we’re extremely excited to see light field video becoming a reality, and look forward to what creatives can produce with what is poised to be an unimaginably powerful filmmaking platform. Filmmakers can sign up for a demonstration and a personalized production package on Lytro’s site. For now, Lytro Cinema will be available on a subscription basis, understandable given the complexities involved (the immense data capture rates require servers on-set).

Head over to the Lytro Cinema page for more in-depth information. Lytro will be demo-ing “Life”, a short film shot using Lytro Cinema at NAB 2016.

Articles: Digital Photography Review (dpreview.com)