Retinex Theory

Psych221 Final Project

Mike Jahr

March 16, 2000

Color Constancy

• Color depends on wavelength

• But, objects reflect different wavelengths

under different lighting conditions.

– Banana in daylight, fluorescent light, no light...

• To us, they seem to retain their color.

How is this possible?

• There is more to color than wavelength.

• The visual system must somehow “discount the illuminant”

A Juicy Burger

A Closer Look...

What’s going on?

• It’s not very saturated, but the red burger has browns, greens, tans…

• How can we see these colors in only red and white light?

Enter Edwin H. Land

• Land was the founder of Polaroid; interested in color

• While running Maxwell’s experiments (3 color projectors), he noticed this

• It spawned decades of experiments

The Mondrian Apparatus

• Land set up 3 filtered

light sources (LMS)

• Can calibrate each one;

precisely control light

• Telescopic photometerActually closer to a

Van Doesburg...

Mondrian Experiments

• Measure reflectance from a green patch

• Calibrate lights so that a blue patch

reflects an identical spectrum

• It still looks blue!

More Mondrian

• Calibrate lights for even reflectance

from the green patch

• Cover all other patches; looks gray

• Uncover all patches; looks green

Land’s Conclusions

• Perceived color depends on reflected spectrum, but also on surroundings

• Relative reflectance is more important than absolute reflectance

Discount the Illuminant: Retinex

• “A framework for computing perceived colors on the basis of the relative intensities of three wavelengths and their spectral interactions.”

• Processed in retina or cortex? Retinex!

Principles of Retinex

• Process each receptor class independently

• Objective is to calculate illuminant-independent “lightness” values

• Lightness values represent perceived color

The Algorithm

• Pick a starting pixel x1, then form a path by randomly selecting neighboring pixels

• Update an accumulator at each pixel:

• Threshold step: if difference is small, use previous sensor response

A xi( )← A xi( )+logρxi( ) −logρx1( )

The Algorithm II

• Keep a counter N(x) for each pixel

• After a number of paths, normalize A(x) by N(x) for each pixel

• Result is L(x), the lightness value

• Algorithm has two parameters:– number of paths, length of each path

What is Lightness?

• Should not depend on viewing conditions

• Should only depend on surface properties

• Results in a triplet that is tough to interpret– The retinex color space

• Issue: what to do with it?

My Implementation

1 Convert image from RGB to LMS via

phosphor spectra and cone sensitivities

2 Run algorithm to get lightness values

3 Do something with lightness values??

• B&W implementation

Retinex Variants

• McCann et al.– Retinex with reset

• Horn– Determining lightness from an image

• Marini– Retinex with Brownian motion

Illusions under Retinex

Original image Processed image

More Illusions

Original image Retinex image

Biological Basis

• Some monkey neurons respond to colors, not wavelengths– Cortical area V4 in prestriate cortex

• Even goldfish can discount the illuminant

Problems with Retinex

• Too dependent on composition of surfaces in image

• Higher-order processes influence color

Conclusion

• Retinex is a long-lived theory, has sparked much debate and many imitators

• Although not a generally accurate model of human vision, it does perform well in some situations

Appendix

• Source files, sample images, sample output, etc. can be found in src/ along with brief explanations of each.

References

• E. H. Land, “Recent advances in retinex theory and some implications for cortical applications: Color vision and the natural image,” Proc. Nat. Acad. Sci. USA 80, 5163–5169 (1983).

• E. H. Land, “Recent advances in retinex theory,” Vision Res. 26, 7–22 (1986).

• B. K. P. Horn, “Determining lightness from an image,” Comp. Graphics Image Process. 3, 277–299 (1974).

• D. H. Brainard and B. A. Wandell, “Analysis of the retinex theory of color vision,” J. Opt. Soc. Am. 3, 1651–1661 (1986).

• J. J. McCann, “Lessons learned from Mondrians applied to real images and color gamuts,” IS&T Rep. 14, 6 (1999). http://www.imaging.org/pubs/reporter/articles/14_6_mccann/index.html

References

• E. H. Adelsen, “Lightness perception and lightness illusions,” in M. Gazzaniga, M.S., Ed., The Cognitive Neurosciences, Cambridge, MA: MIT Press, pp. 339-351 (1999). http://www-bcs.mit.edu/people/adelson/publications/gazzan.dir/gazzan.htm

• F.W. Campbell, F.R.S., “Dr. Edwin H. Land,” Biographical Memoirs of Fellows of the Royal Society, 40, 195-219 (1994). http://www.rowland.org/land/land.html

• D. Marini and L. Marini, “Measuring the colours we receive,” Science Tribune, October (1997). http://www.tribunes.com/tribune/art97/mari.htm