Hanlin Goh published a note.
Light Field Photography (Part 3: Third Dimension)
This is the last of a series of three notes on light field photography with the Lytro camera. In my first note, I gave an introduction of light field photography and in my previous note, I mentioned that the Lytro light field camera tries to decipher the depth of the scene to enable interesting refocusing application. In this note, I will talk about 3D.
One really cool way to explore the newly available depth information is to explore the image in 3D. Conventional cameras can only capture 2D and 3D is sometimes achieved by putting two cameras side by side to capture get what we call a "Stereogram". In this note, I will use my photo of Wooden Vehicles again as the main example [Click Here to View the Living Picture].
Parallax Effect
Parallax is one of the visual cues that humans use to understand the depth of the scene. Because a light field image records the direction of incoming rays, parallax is captured, albeit at a minute scale. When moving (in plane) across the scene, we want closer objects to move faster than distant ones, creating a faux-3D effect.
For Lytro, it is said that this feature may be in the pipeline and will perhaps launched by the end of the year.
Demo Time!
Click on the following link to explore the parallax demos that I programmed:
Notice how the objects shift ever so subtly (yet realistically) relative to each other.
Update (7 June 2012): A new video on CNET shows Lytro CEO, Ren Ng, demonstrating the same parallax effect controled by the cursor position.
Stereo Images
Since we can synthesize the scene from slightly varying viewpoints, a small extension will be to create stereograms. Stereograms are a pair of images that have a slightly different viewpoint. When placed side by side, we can exploit our binocular vision (left eye looks at left image, right eye looks at right image) to combine them into a 3D image scene.
Here are the same two examples as stereograms:
If you are having trouble seeing 3D, relax and cross your eyes by looking beyond the screen, until there are three images. The centre image should look 3D. If not, try moving towards or away from the screen, or tilting you head.
3D Scene Rendering
Finally, using a combination of depth information and the fully focused, we can render drastically new views of the scene. Taking the probability masked from the previous note, computing the depth expectation and overlaying the texture using the fully focused image (also from the previous note). Without boring you with more technical details, here is a rendering of the 3D scene from a new angle:
As we can see, the general 3D structure of the scene is rendered correctly - the table top generally flat, the wooden models in front occlude those behind. However, the rendering is definately not Hollywood quality. With better 3D rendering techniques and more accurate depth information, one should be able to improve on this significantly.
With this, I conclude this series of notes on the theory and applications of Light Field Photography using the Lytro camera. I hope you enjoy reading notes and playing with the demos as much as I have enjoyed writing and programming them. Meanwhile, continue to support my Living Picture Year project. The end.
