|
Digital Processing of Quantum-Limited Images Conjecture on the Relationship between Spatial and Temporal Visual Processes Why do Stabilized Images Disappear? A Simple Model for Filling-In, Contrast, Contrast Constancy and Assimilation
|
What
is “True Color”? Abstract The idea that an image should be displayed in “true color” is frequently expressed, especially in reference to digital imagery. Possible definitions of the term are presented, and each is evaluated. By those definitions, there is virtually no way to display an image in “true color” and none of the definitions is meaningful. In order for the term to have coherent and logical meaning, its definition must be so relaxed that its usefulness is severely limited. What is “True Color”? Ophthalmologists often express concern about whether or not images of the eye are in “true color”. This concern has become particularly prevalent with the increasing popularity of digital imaging. The meaning of the term “true color” seems obvious, but on closer examination the issues become very complicated and it becomes clear that the use of the term can lead to serious ambiguities, empty debates, and wasted resources. When an ophthalmologist questions whether or not a fundus image is in “true color”, she might be asking either of two quite different questions. One is “Do the features in this image appear in the same colors as the colors I would see if I looked at the fundus with an ophthalmoscope or slit lamp?”. The other is “Do the colors that appear in this image correctly represent the physical properties of the imaged fundus?”. In the following, we will discuss the factors that must be understood in order to answer these two very different questions. Before discussing those issues, however, we should probably examine a simpler one. “True color” as the opposite of “False color” Since the advent of earth-imaging satellites, the term “false color” has entered the lexicon, and the existence of false color implies the existence of “true color”. This is not as superficial a source of the idea of true color as it might first appear. It is almost certainly the basis for the term “true color”, and probably serves as a psychological support for the idea itself. So let’s examine the meaning of “false color”. As you know, the typical process for generating an ordinary colored image is as follows. In effect, three separate images of the scene are collected, one sensing the blue light reflected from the scene, another the green light, and a third the red light. (These three are usually collected simultaneously by film or a digital color camera.) To display a single colored image, the image collected in blue light is displayed in blue, the green is displayed in green and the red in red, the three images being superimposed. Satellite photos of the earth usually come in sets that are collected through different filters too. For example, one image might be taken through a filter that transmits red light and two others through filters that transmit in two different regions of the infrared spectrum. These three images can then be displayed as a single colored image, for example by displaying the image taken through the red filter in red, one of the infrared images in blue and the other in green. This is called “false color” for obvious reasons. It follows, however, that virtually every colored image is really a false color image. Figures 1 and 2 are reminders. Figure 1 is a typical plot of the sensitivity of a device (here a CCD digital monochrome camera) as a function of the wavelength of the incident light. It is sometimes called a spectral sensitivity curve, or an action spectrum because the vertical axis is really a measure of the strength of activity caused by photons of wavelengths on the horizontal axis. Figure 2 contains typical plots of the intensities of light reflected from or emitted by surfaces as a function of wavelength. Figure 2a is the emission spectrum of the xenon flash tube that is used in most fundus cameras. Figure 2b is the emission spectrum of a tungsten filament at various intensity settings. Figure 2c is the reflectance spectrum of a small region of the human macula. 
Figure 1.
Figure 2a Emission spectrum of the xenon flash tube used in most fundus cameras.
Figure 2b Emission spectra of a tungsten lamp, such as in most ophthalmoscopes, at various intensity settings.
Figure 2c. Reflectance spectrum of a small region of a typical human ocular fundus. (from Reference 1.) Now consider an ordinary colored photo. In order for it not to be in false color (that is, to be in “true color”), the sensitivity of the blue sensing process at each wavelength across the spectrum (its spectral sensitivity curve) would have to perfectly match the reflectance (for a print) or the emittance (for a video monitor) spectrum of the blue channel of the display device, the spectral sensitivity curve of the green sensing process would have to match the spectral reflectance or emission of the green channel of the display, and the same for the red. Since different films or digital color cameras have differing spectral sensitivity curves, the spectral absorbances of films vary with manufacturer, development, etc, and the spectral emittances of monitors vary with manufacturer and color settings, it is extremely rare for such matches to occur. Therefore, virtually every colored image is really a false color image. The earth-satellite images are simply extreme examples. If the meaning of “true color” is taken as the opposite of “false color”, “true color” images almost never happen. (For completeness, we should define a third commonly used term, “pseudocolor”. A pseudocolored image is one in which arbitrary colors are used to represent properties other than color, for example, temperatures, or the heights of mountain ranges.) Now we can address the other two possible meanings of “true color”. To understand what they mean, both require developing an understanding of the processes involved in human color vision. The Basics of Human Color Vision There are two classes of photosensors in the human retina, called rods and cones. We will ignore rods in this discussion because they only contribute to vision when the light is very dim and play a negligible part in the perception of color under the conditions that obtain when viewing retinas or images of retinas. The retinas of people with normal color vision contain three classes of cones, differing in the parts of the spectrum to which they are most sensitive. The spectral sensitivity curves for the three cone types (and for rods) in a typical normal human retina are shown in Figure 3. To understand normal color vision, it is useful to begin by considering the vision of a person who has only one class of cones, for example, the class labeled “Green Cones” in Figure 3. Laser light consists of light of just one very narrow band of wavelengths. Suppose this person with only one kind of cones looked at the light from a helium-neon laser falling on a piece of white paper. (A person with normal color vision would call that light orange or red.) Some of the light reflected from the paper will enter the person’s pupils and fall on his retinas, where some of that light will be absorbed by the photopigment in his cones, triggering a complex series of physical-chemical reactions that ultimately produce signals in certain parts of his brain. If the laser light is increased in intensity, more photons will be absorbed by his cones, the signals generated in the rest of his visual system will be stronger, and he will say that the light looks brighter. All that seems pretty obvious.
Figure 3. Spectral sensitivities of the three cone types (and rods) in a typical human retina. Now imagine that a second laser, for instance an argon laser, emitting a different wavelength, illuminates a second spot on the paper. The photons reflected from that spot will strike a different part of the person’s retina and excite activity there. A person with normal color vision would say that one of the spots is red and the other green. What would the person with only one kind of cone say? He would surely say that there are two spots, and it is likely that the two spots would trigger different strengths of activity in the two corresponding regions of his retina, but could he see that they are different colors? He might very well say that one is red and the other is green, especially if he knows what lasers are being used, but can he really tell the colors apart? This can easily be tested in the following way. Gradually adjust the intensity of one of the lasers and ask him to report what he sees. There will be some intensity for which the two spots will produce identical activities on his retinas and therefore must look identical. Why do the two different wavelengths of light produce identical effects? After all, the photons from the two lasers are really different. They have different wavelengths and different energies. When a photon is absorbed by the pigment in a cone, it triggers a certain reaction. (A molecule isomerizes, that is, it changes its shape, and this isomerization triggers a complex series of further reactions.) The likelihood that a pigment molecule in a cone will absorb a photon depends upon the relationship between the nature of the molecule and the energy in the photon. Photons with far too much or far too little energy will pass right through but ones whose energies are closer to a good fit or resonance with the molecule have a better chance of being absorbed. The curves in Figure 3 are actually plots of the probability that a photon of the wavelength on the horizontal axis will be absorbed. However, the property of the visual system that is crucial to understanding almost everything related to color vision is this. If a cone pigment molecule does absorb a photon, the resulting effect of the photon is exactly the same regardless of the kind of photon that triggered it. The wavelength or energy of a photon determines how likely the photon is to be absorbed, but once it is absorbed, exactly the same kind of isomerization occurs no matter what the wavelength or energy of the photon. That is the reason why, if a person has cones with only one kind of pigment in them and is looking at two spots of light, it is possible to adjust the intensity of one of the spots until the two spots produce identical numbers of isomerizations and therefore identical visual responses; the person can’t tell that they are “really” different colors, regardless of what the “colors” or wavelengths actually are. Understanding this fact is the key to understanding all the following discussion, and the key to understanding almost all of the fundamental phenomena of color vision. (The vision of this person with only one kind of cones is exactly analogous to imaging with black and white film. If a photon is absorbed, it will produce the same kind of isomerization and, after development, the same kind of silver molecule, regardless of its wavelength.) Now consider one more experiment with this person who has only one kind of retinal pigment. Move one of the two laser spots until it falls on top of the other. To a person with normal color vision, the spot will now appear yellow (a mixture of red and green), but for the person with only one kind of pigment, the effects of the photons of the two wavelengths will just add together and spot will simply produce more of the same activity - it will just look brighter. In general, you could show this person two spots each of which contained any arbitrary mixture of wavelengths, and by adjusting the intensity of one or any number of wavelengths, you could make the two spots appear identical to him. We call a person who has cones that contain only one kind of pigment “totally color blind”. By that, we do not mean that he can’t see colors. There’s no way for us to know whether or not things look colored to him, or even what the word “color” means to him. But what we can determine, both by experiment and by logic, is that this person cannot distinguish among lights on the basis of their wavelengths. He is called a monochromat. (People with only one functional class of cones are extremely rare. Somewhat less rare is the condition call “rod monochromacy”, in which the person has no functional cones and all seeing is mediated by rods, which, even in normals, contain pigment with only a single spectral sensitivity curve.) A monochromat has only one kind of photopigment and his color vision is one-dimensional, that is, it varies in only one dimension, which we call brightness. Now we can begin to address the original question, what is “true color”? “True color” as a Correct Representation of the Physical Nature of the Scene Clearly, for a monochromat, by this definition no image can be in true color, since many physical properties of a scene are revealed by differences in the amounts of different wavelengths reflected to the eye, and the monochromat’s vision cannot capture those differences. So, by this definition of “true color”, no image can be in true color for a monochromat. You probably now want to say “so what”. The ophthalmologists who are worried about true color are not totally colorblind. Most of us have not just one but three different kinds of visual pigment and we can see all kinds of colors. True, but we are still very colorblind. If we had two kinds of pigments, we would be somewhat less colorblind than monochromats. (People with two functional kinds of pigments are called “colorblind”, e.g. protanopes, deuteranopes, or tritanopes, and their color vision is two-dimensional). The rest of us, with what we call normal color vision, are simply somewhat less colorblind than dichromats, but still seriously colorblind. That is, while our color vision is three-dimensional, the range of colors in the world varies over an infinite number of corresponding dimensions. Therefore, the perceived colors in an image are never “true” in the sense that they represent the true nature of a scene. Clear examples of this are abundant. For one, suppose you had a sample of a liquid and wanted to know some of its physical properties. You would put it into a device that illuminates it with wavelengths across some region of the spectrum and measures the percentage of light at each wavelength that is absorbed by the liquid. Figure 4 is the result of such a spectral analysis of two components of macular pigment. Alternatively, you could just look at the sample to see what color it was, but that would obviously be much less instructive. The reason it would reveal far less information is because the device that generated the curves in Figure 4 acted as though it contained photodetectors with a very large number of different spectral sensitivities while your eye contains only three. Figure 4. Absorption spectra of two components of human macular pigment. Here’s another example. If you look at a white
region on the screen of your computer monitor with a strong magnifier,
you will see that the display consists of an array of tiny dots.
Some of the dots are red, some green, and the rest blue. Without
the magnifier, the region looks white, that is, it has the same
color as a piece of white paper, even though white paper actually
reflects a completely different array of wavelengths to your eye
than the wavelengths emitted by the screen. Our normal color vision
systems can’t tell the difference between an object that emits
only red, green and blue, from all the objects of different colors
that can be reproduced on the screen. What is happening is that,
when you look at the screen from a normal distance, without a magnifier,
the images of the red, green and blue dots are spread out enough
by the optics of your eyes that they overlap, their wavelengths
mix, and they produce exactly the same effects on your retinas as
the very different mixture of wavelengths that is reflected from
a piece of paper or any other object. We are too colorblind to tell
the difference between the color of the screen and of a piece of
paper. Of course if such a procedure were implemented, it would be most useful to choose wavelengths among which the physical properties of interest vary. Figure 5 shows an example of this idea. The image on the right was collected with illumination in the wavelength band from 525 to 575 nanometers. (That light looks green.) The one on the left was illuminated by a band from 625 to 675. (That looks red.) The red light penetrates the retina more deeply and so reveals much more of the choroidal vasculature than does the green, but also note that in green light the oxygenated blood in the arteries and the reduced blood in the veins are about equally dark while in the red, the oxygenated blood is almost transparent. The differences between arteries and veins are much less noticeable in a standard color photograph.
Figure 5. Fundus images collected under different wavelengths of illumination. On the right, the wavelength band is from 525 to 575 nm., (green) and on the left from 625 to 675 (red). (Courtesy of Visual Pathways, Inc.) Figure 6 shows another example. The image on the left is a standard color image. The one in the middle is taken in green light and the one on the right in red. Note the striking differences in the visibility of the area of geographic atrophy. The image in the middle in Figure 7 was taken in red light while the one on the right in near-infrared (850 to 900 nm). Note that some features are dark in one image and light in the other, while for other features, the reverse is true. These two images indicate differences in the physical properties of the fundus that are not visible in the standard color image on the left.
Figure 6. Images of an area of geographic atrophy. The image in the middle is collected under green (525 to 575) illumination and the one on the right under red (725 to 775). The red wavelength band is optimal for displaying geographic atrophy. (Courtesy of Visual Pathways, Inc.)
Figure 7. The image in the middle is under red (625-675nm)
illumination and the one on the right under near infrared (850-900nm).
(Courtesy of Visual Pathways, Inc.) “True Color” as when the Image
Looks the Same as the Scene Itself Based upon the above discussion of color vision, we can consider what determines the perceived color of some particular region. First of all, light has to fall on the region in order that some light is reflected from it to the eye (unless the region is self-luminous, which the fundus is not under normal circumstances). Suppose that the light falling on the surface contains equal amounts of light at every wavelength. (We would call it “white”.) The surface will reflect a certain percentage of the incident light at each wavelength. For instance, the ocular fundus reflects a very high percentage of long wavelength (red) light and a low percentage of short wavelengths (blue), as plotted in Figure 2c. Some of the resulting photons that enter the viewer’s eye are absorbed by the pigment in the three types of cones, and at each small region of the retinal image, each of the three cone types will deliver a signal whose strength depends upon the amount of the incident light that it absorbs. That is, at each small region of the retina, three signals are generated. The color that the observer sees is related to the ratio of the strengths of these three signals. So first let’s ask what color an observer sees when she looks at the fundus through an ophthalmoscope. In addition to depending upon the nature of the ophthalmologist’s and patient’s eyes, it will depend upon the wavelength composition of the illuminating light, since, if, for example, the relative amount of short wavelength light increases, the fundus must appear bluer. The wavelength composition of the illuminating light, in turn, depends upon the kind of bulb in the ophthalmoscope and brightness setting of the bulb. The brighter it is, the more blue light relative to red (as plotted in Figure 2b) and so the bluer the fundus looks. Similarly, if the fundus is being viewed through a slit lamp microscope, not only does the brightness setting of the lamp affect color, but also the coatings on the elements in the microscope usually pass more red than blue light, affecting the apparent color of the fundus. (And because the emission spectra of the tungsten focusing lamp and the xenon flash lamp in a typical fundus camera are so different, as shown in Figures 2a and 2b, the colors of the reflected images will be very different.) Therefore there is already a problem with this definition of “true color” – we don’t really know the color of the fundus viewed directly, unless we happen to know the intensity settings, the kind of illumination, and the nature of the viewing optics. Other problems arise as a result of the nature of human color vision. Figure 8 shows a diagram that may be familiar. The area inside the outer, curved triangular border is a way of representing all possible colors that are visible to a normal human observer. The sharp triangle within the larger region includes all of the colors that can be reproduced by a typical video or computer monitor. Thus there are many colors in the real world that simply cannot be produced on a video monitor. (Color film images have similar limitations.)
Figure 8. The area included within the outer, rounded, triangle represents all the colors visible to a person with normal color vision. The area within the inner, sharp, triangle represents all the colors that can be displayed on a typical computer or television monitor. Another serious problem with color monitors (and an analogous problem with inks and color films) is the result of an interaction between brightness and hue. To describe a simple example, suppose a digital camera images a ripe lemon in good illumination and a monitor is adjusted so that the color of the image of the lemon is a perfect yellow. If the illumination on the lemon is reduced, the lemon itself will still appear yellow but the color of its image on the monitor will clearly become redder or greener, depending on the particular monitor. In general, if the scene being imaged includes regions of varying brightness, for example, a fundus image that includes the optic disk, there is no adjustment of the monitor that can consistently render all of its colors. Overview In view of all these arguments, why do ophthalmologists ask that fundus images be rendered in “true color”? Clearly it is because they want to be able to match features of the image they are examining to pathologies they have learned to recognize while examining other fundi or images of them. We can deal with that very valid need in both of two different ways. First, we can relax the definition of “true color”. We can take “true color” as meaning a color rendition that is similar enough to the range of what is typically seen through an ophthalmoscope that the colors of features are recognizable. . And when we recognize the limitations of the human color vision system , we can go further and offer the ophthalmologist an opportunity to overcome those limitations and learn to recognize features displayed in new ways. Summary “True color” in an image can be defined as the opposite of false color, a true representation of the physical properties of the scene, or a color identical to that perceived when looking at the original scene. When carefully analyzed, none of those definitions is meaningful. Figure Legends Figure 1. Sensitivity of a CCD camera as a function of the wavelength of the incident light. This kind of plot is also called a spectral sensitivity curve or an action spectrum. Figure 2. Reflection and emission spectra. a) Emission spectrum of the xenon flash tube used in most fundus cameras, b) emission spectra of a tungsten lamp, such as in most ophthalmoscopes, at various intensity settings, and c) reflectance spectrum of a small region of a typical human ocular fundus. (Figure 2c is from reference 1.) Figure 3. Spectral sensitivities of the three cone types (and rods) in a typical human retina. Figure 4. Absorption spectra of two components of human macular pigment. Figure 5. Fundus images collected under different wavelengths of illumination. On the right, the wavelength band is from 525 to 575 nm., (green) and on the left from 625 to 675 (red). (Courtesy of Visual Pathways, Inc.) Figure 6. Images of an area of geographic atrophy. The image in the middle is collected under green (525 to 575) illumination and the one on the right under red (725 to 775). The red wavelength band is optimal for displaying geographic atrophy. (Courtesy of Visual Pathways, Inc.) Figure 7. The image in the middle is under red (625-675nm)
illumination and the one on the right under near infrared (850-900nm).
(Courtesy of Visual Pathways, Inc.) References 1) Preece, S.J. and Claridge, E. MonteCarlo modeling of the spectral reflectance of the human eye, Phys.Med. Biol. 47 , pp2863-2877, (2002) Key words: True Color, False Color, Displays of Wavelength Information Proprietary Statement: The author is a shareholder and employee of Visual Pathways, Inc., which supplied the images for Figures 5, 6, and 7. |
