Post on 12-Jul-2015
Dual PhotographyPradeep Sen | Billy Chen | Gaurav Garg | Stephen Marschner | Mark
Horowitz | Marc Levoy | Hendrik Lensch
Prashanth PM
Introduction
Dual Photography uses the concept of Helmholtz Reciprocity to interchange the lights and cameras in a scene.
Helmholtz Reciprocity
Bidirectional Reflectance Distribution Function (BRDF)
fr(ωi, ωo) (outgoing radiance / Incoming radiance)
fr(ωi, ωo) = fr(ωo ,ωi)
Helmholtz Reciprocity
First Relighting Work to make use of Helmholtz Reciprocity.
Radiance transfer between incoming and outgoing directions is
symmetric
So, we can generate an image from the point of view of the projector.
Primal Configuration c’ : Captured Image (nm x 1)
p’ : Projected Pattern (pq x 1)
T : Transport Matrix (mn x pq)
Dual Configuration c”: Virtual Projected Pattern (nm x 1)
p”: Virtual Captured Image (pq x 1)
TT: Transposed Transport Matrix (pq x nm)
Transport Matrix
T’’ = TT due to Helmholtz Reciprocity
Dual Photography is the act of multiplying the transposed matrix by a
desired lighting image vector.
Photography without a camera.
Image at right acquire using a 3x3 pixel
square scan across the projector.
Two photo resistors are used instead of
a camera.
Comparison with Debevec et. al. [2000]
In Debevec’s “Acquiring the Reflectance Field of a Human Face“, distant point light sources are assumed, and there is no spatial variation within the light source.
Hence, sharp shadows cannot be cast onto the scene.
Besides, a fewer number of light sources are required.
Optimizations in Capturing the T matrix.
Brute Force Method
Scan a single pixel per captured camera frame
Projector and Camera each have O(n6) pixels
Full T matrix would have O(n12) elements
HDR imagery required for scenes containing both specular and diffuse inter-reflections.
Even at a rate of 25 HDR images per minute, the capture process could take weeks !
Fixed Pattern Scanning
Assume each projector pixel affects a small, localized region of the camera. Divide the region into blocks.
Repeat exposures and encode each block’s illuminated pixels with a unique binary encoding.
Fixed Pattern Scanning
1 17
….. …..
T
1 17
Projector Pixels
1
17
We can determine the spot being lit using a truth table (bit encoding) for 3 locations.
4 2 1 Spot
0 0 0 0
0 0 1 1
0 1 0 2
0 1 1 3
1 0 0 4
1 0 1 5
1 1 0 6
1 1 1 7
Hamming error correcting codes are added to account for bit errors due to noisy measurement.
Limitations of Fixed Pattern Scanning
Requires one-to-one correspondence between camera and projector pixels.
This only supports direct illumination properly.
Diffuse Illumination can map many projector pixels to the same camera pixel.
This violates the initial assumption.
Adaptive Multiplexed Illumination
LEVEL 1
An 8x8 pixel project is used.
All Pixels are Illuminated.
Adaptive Multiplexed Illumination
LEVEL 2
The single block from Level 1 is subdivided
into four blocks.
Conflict is detected between regions 2 and 4
Additional subdivision is required
Adaptive Multiplexed Illumination
Conflicting blocks from Level 2 are not
co-scheduled in Level 3.
Some blocks can be scheduled in parallel.
Two new conflicts detected (6 & 12, 8 & 10).
The lower-leftmost block causes no illumination
and is culled
Adaptive Multiplexed Illumination
Final subdivision is applied in Level 4.
Many blocks are acquired in parallel.
Drawbacks of Adaptive Multiplexed Illumination
Does not do well for a scene with several diffuse inter-reflections.
A single projector pixel can affect the entire scene.
Hierarchical Assembly of the Transport Matrix
Dual from Adaptive Multiplexing Dual from Hierarchical AssemblyInflections between red wall and boxis almost lost.
Results
Primal Dual
Scene Relighting
Primal and Dual Images can be relit by multiplying T or TT by the desired matte as p’ or c.”
Scene Relighting
Primal Dual Dual with relit pattern
Dual – hand Dual – moving light beam
Scene Relighting
Projectors are not parallelizable, only one projector can be used at a time. But cameras can be parallelized to capture a 6D reflectance field.
Scene Relighting
Results
Dual – left lit Dual – right lit
Results
Dual – soft shadows Dual – hi res matte
Results
Dual – Soft Shadows Dual – Shadows cast by synthetic model
Limitations
Scenes containing significant global illumination
effects reduce parallelism of the adaptive method.
Limitations
Cameras have greater depth-of-field, better focus control, and more imaging controls
Limited depth-of-field can result in out-of-focus dual images
Limitations
Suppose the camera and projector are at a large angle apart from each other.
There may be many regions in the scene with no direct light transport between the camera and projector.
How to read your opponent’s card ?
Thank You
Questions ?