Color Constancy as Counterfactual

Jonathan Cohen¹

Abstract

Color constancy is a fascinating phenomenon in its own right, and also has implications regarding the ontology of color. In this paper I'll argue that the usual understanding of color constancy in the philosophical and empirical literature are unsatisfactory, and the propose a novel account of the phenomenon that (i) offers gains in descriptive adequacy, and (ii) undercuts ontological conclusions often drawn from color constancy.

In recent years, vision scientists and philosophers of perception have devoted considerable attention to questions about color constancy. Among the most important issues surrounding color constancy are those about how we should understand the phenomenon and those about what the phenomenon shows about the nature of color. I believe that the phenomenon of color constancy has been misunderstood, and that correcting this misunderstanding will have the effect of undercutting the apparent support the phenomenon provides to accounts of color as illumination-independent properties.

1 Constancy as Invariance

Since color constancy is easier to recognize than to define, I'll begin with an example - a coffee cup on a table, partially in direct sunlight and partially in shadow.²

Figure Figure 1: Partially sunlit coffee cup.

Consider the region of the coffee cup (and the region of the table) in direct sunlight, and compare it against a similar-sized, adjacent region of the coffee cup (and a region of the table) that are in shadow. If you are like most (normally sighted) subjects, you will find that these two regions are, in some sense to be explained, alike in their apparent color. On the other hand, again assuming you are like most (normally sighted) subjects, you will also find that the regions are easily, obviously, and quickly visually discriminable. Moreover, you are likely to explain this easy, obvious, and quick visual discriminability in terms of the readily apparent difference in illumination between the two regions. These two reactions to a pair of visual stimuli (in this case, the pair consists of the two simultaneously presented regions of the coffee cup) are the hallmarks of color constancy.

These two reactions lead naturally to what has become a standard understanding of color constancy as a kind of invariance. In particular, on this view (henceforth, invariantism), color constancy is an invariance of apparent color across changes in illumination.³ ⁴

The main problem that I wish to press against invariantism comes from something already noted. Namely, in paradigm cases of color constancy, subjects canonically have two different reactions: they judge not only that the members of the pair of surfaces are alike in apparent color, but also that they are easily and obviously visually discriminable. Of course, it does not seem (in general) that this discriminability can be accounted for in terms of a difference in the size, form, or texture (etc.) between the surfaces. Rather, we seem forced to admit, subjects are able to discriminate the members of the pair because they take the members to differ in respect of their apparent color.⁵ But to say this is to say that, while there is a sense in which subjects judge that the members of the pair are alike in apparent color, there is another sense in which they judge that the members of the pair are unalike in apparent color.

This fact is, I want to suggest, a serious problem for invariantist accounts of color constancy. For, if the foregoing is correct, then it is at best seriously misleading to say, with the invariantist, that color constancy involves an invariance of apparent color across variations in illumination. This claim respects only only one of the two reactions (normally sighted) subjects have to these cases. While it respects one of their reactions (their reaction that the two members of the pair of stimuli are alike in apparent color), it ignores their reaction to the effect that the members of the pair differ in apparent color - that there is not an invariance of apparent color across changes in the illumination.⁶

Now, the argument above turns crucially on the claim that subjects find there to be some sort of difference in apparent color between the surfaces in question (for example, between the two adjacent regions of the coffee cup depicted in figure 1)? What support is there for this claim? Let us begin with a fairly crude, qualitative test: let us ask whether the adjacent surfaces are different in apparent color by asking whether subjects can visually discriminate between them. The answer that I've received from 100% of the subjects I've asked to consider figure 1 is that they can visually discriminate between the two adjacent surfaces. This presumably means that there is some perceptually salient difference between the two surfaces; and, for reasons rehearsed, this difference most plausibly understood as a difference in apparent color.

On the other hand, it may be reasonably objected that the test of qualitative discriminability relied upon in the foregoing is a blunt (and outdated) instrument for investigating color constancy. Let us consider, therefore, how the line of argument I am running fares in the face of the sharper, quantitative measurements used in contemporary psychophysical investigations of color constancy. The main quantitative measure by which contemporary psychophysicists assess color constancy, known as asymmetric color matching [Wyszecki and Stiles, 1967], involves asking subjects to change the chromaticity (or lightness, in lightness constancy experiments) of a test patch under one illuminant until it perceptually matches a standard patch under a different illuminant. Since the subject eventually arrives at a perceptual match, investigators regard the chromaticity (/lightness) difference between the test and the standard patches as a quantitative measure of the perceptual effect of the illumination difference.

Unsurprisingly, asymmetric color matching brings out the points considered above even more clearly than the qualitative measure discussed. Roughly, it seems that (most) subjects can respond in two different modes - one mode that assimilates surface pairs like that in figure 1, and one that distinguishes such pairs. Moreover, it seems that (most) subjects can be made to switch between these two modes of response as a result of experimental instructions: instructions to "adjust the test patch to match its hue and saturation to those of the standard patch" lead subjects to distinguish members of such pairs, while instructions to "adjust the test patch to look as if it were `cut from the same piece of paper' as the standard, i.e., to match its surface color" ([Arend and Reeves, 1986], 1744) lead subjects to assimilate members of such pairs.⁷ (I'll follow [Bäuml, 1999] in talking about the first sort of matches - those that distinguish the two surfaces - as appearance matches, and the second sort of matches - those that assimilate the two surfaces - as surface matches. Obviously, no substantive conclusions should be inferred from the choice of these labels.)

To repeat, then: invariantism characterizes cases of color constancy as cases where there is an invariance of apparent color despite changes in the illuminant. But in fact, in what would pre-theoretically count as cases of color constancy, subjects can be made to judge either that there is a single color (hence that there is an invariance of apparent color) or that there are two colors (hence that there is not an invariance of apparent color). This means that, in order to sustain her account in the face of the experimental results, the invariantist owes us some reason for ignoring fully half of the data, as well as an account of the relation between appearance and surface matches.

2 Color Constancy and Color Ontology

At this point I want to consider the role color constancy has figured in philosophical arguments about the nature of color. I want to suggest that these arguments have problems quite independent of the points made above - that they fail on their own terms.

As noted, philosophical appeals to color constancy have generally been in the course of arguments to the conclusion that colors are objective, illumination-independent properties of some sort (e.g., surface spectral reflectance distributions). For example, here is Michael Tye: "The fact that objects appear to retain the same color through a wide variety of changes in illumination conditions (though certainly not all) strongly suggests that colors are illumination-independent properties of those objects" ([Tye, 2000], 147-148). Similarly, Hilbert begins with the claim that "The existence of color constancy should lead us to suspect the existence of some illumination-independent property of objects that is correlated with color," goes on to note that "The obvious candidate for such a property is ... surface spectral reflectance" ([Hilbert, 1987], 65), and ends up concluding that colors are indeed identical to (classes of) surface spectral reflectance distributions (see also [Byrne and Hilbert, 2002], §3.1 for a compact and explicit formulation of the same thought).

It seems that these philosophers are urging upon us something like the following chain of (non-deductive) reasoning (henceforth, the objectivist's inference):

(1): There is color constancy.
(2): Reflectance distributions are illumination-independent.
(3): Therefore, colors are identical to reflectance distributions.

On an invariantist account, premise (1) of the objectivist's inference amounts to the following, understood as an empirical claim:

(1¢): The apparent color of a surface is invariant across changes in illumination.

Of course, (1¢) will not, by itself, give us any reason to believe the intended conclusion (3) unless we take (1¢) to tell us something about color, as opposed to apparent color. The thought, then, must be that (1¢) is a (defeasible) reason for believing this:

(1^*): The color of a surface is invariant across changes in illumination.

The objectivist's inference, on this reading, comes out as a pair of inferences - namely, a pair of inferences to the best explanation. It moves first from the empirical premise (1¢) to the effect that apparent colors are illumination-independent to the allegedly best explanatory conclusion (1^*) to the effect that colors are illumination-independent; and from (1^*) and the definitionally true premise (2) to the effect that reflectance distributions are illumination-independent, to the allegedly best explanatory conclusion (3), that colors are identical to reflectance distributions.

Understood in this way, the objectivist's inference invites a number of worries. First, there is a concern that, in the context of a debate about what the empirical results about apparent color tell us about color, the inference from (1¢) to (1^*) will strike many as too hasty (cf. note 3). Although I am sympathetic to this criticism, I'll put it aside, insofar as I believe there is a more fundamental problem with the inference.

This problem arises from the observation that there are many uncontroversial cases - cases that even Hilbert and Tye would accept - where the alleged invariance cited in (1) fails. I take it that this much would conceded by all sides.⁸ Indeed, I take it that the parenthetical qualification in the quotation above from ([Tye, 2000], 147-148) is intended to concede that the claimed invariance fails in at least some cases. But if the invariance fails in many cases, then we are justified in wondering why it is properly thought of as an invariance at all (rather than a variance). And given that Tye, Hilbert, and others are prepared to take the invariance of apparent color across changes in illumination as evidence that (apparent colors, and therefore) colors are illumination-independent, why should we not take the failures of invariance as showing that (apparent colors, and therefore) colors are illumination-dependent properties of objects?

Tye and Hilbert are, of course, aware of this worry, and their attempts to answer it are instructive. They propose to regarding the violations of invariance as exceptional cases to the general rule (viz., the rule of invariance) on the grounds that these cases are cases of color misperception (cf. [Hilbert, 1987], 71-72, [Tye, 2000], 153ff).

Unfortunately, it is not clear to me that there are theory-independent reasons for endorsing this view. Of course, if colors are assumed to be illumination-independent properties, then cases in which apparent color is shown not to be illumination-independent are plausibly described as cases where the apparent color fails to reveal the true color of objects - as cases of color misperception. But this result, of course, turns on assuming that colors are illumination-independent, and presumably this is not a supposition that we should make in our characterization of color constancy if we hope to appeal to color constancy to motivate the view that colors are illumination-independent. Thus, short of begging the ontological question that color constancy is being enlisted to answer (via the objectivist's inference), it is not clear why the acknowledged failures of invariance should not be regarded (pace Hilbert and Tye) as evidence for the illumination-dependence of (apparent color, and therefore) color.

Without answers to these questions, it seems that the inavariantist account of color constancy cannot be used to show that colors are illumination-independent.

3 Color Constancy as Counterfactual

I want to propose an alternative understanding of color constancy, and then argue that it is superior to invariantism. Finally, in §4, I'll consider the implications this revised account of color constancy holds for color ontology.

Our problem is to characterize the phenomenon involving our perception of the adjacent regions of the coffee cup in figure 1. The discussion so far suggests two desiderata that an acceptable account of color constancy should meet. First, we want to say that there is some sense in which the colors of these two regions appear relevantly alike to subjects; this desideratum surely captures one intuition that we have about cases of color constancy, and it comes out clearly in subject judgments about surface matches. Second, we need a way of articulating this last idea while (unlike the invariantist) avoiding the straight-out insistence that the two regions are identical in apparent color, since, as shown by the appearance matching data, they are not.

As a way of meeting these desiderata, I propose to understand color constancy as an ability to make counterfactual verdicts - verdicts about the color appearance that objects would have if they were viewed in non-actual perceptual circumstances. In particular, I suggest, surface match responses answer this question: would region R₁ (presented under illumination I₁) share a color appearance with region R₂ (presented under illumination I₂) if, contrary to fact, both regions were presented under the same illumination?

This account (henceforth, counterfactualism) respects the desiderata cataloged above. First, it offers a direct account of the sense in which the colors of the two regions appear relevantly alike. Namely, the two regions appear alike in that they would share a color appearance if both regions were presented under the same illumination. Second, and unlike invariantism, counterfactualism explains this likeness in appearance while simultaneously respecting the facts about appearance matches. Namely, the counterfactualist allows that the two regions of interest in a case of color constancy can (i) appear relevantly similar in respect of color, even though (ii) they differ in color appearance.

In addition, counterfactualism has the advantage that, unlike invariantism, it provides a plausible understanding of the relationship between surface matches and appearance matches. For, if counterfactualism is correct, then surfaces match judgments are naturally understood as results computed by the visual system from the evidence of non-matching color appearances. Of course, appearance matches are, on this story, made by comparing the same color appearances that serve as evidence for surface match judgments. But if this is right, then surface match judgments are the results of computations that begin where appearance match judgments end. Consequently, counterfactualism predicts that appearance matches should be less difficult and faster than surface matches. I take the confirmation of these predictions ([Arend and Reeves, 1986], 1747-1748) as a further piece of support for the view.

4 Ontological Implications

The counterfactualist account of color constancy is explicated in terms of apparent colors, rather than colors. However, the question that has made color constancy worth caring about for most philosophers is whether colors are illumination-independent. But what we have learned about apparent color surely bears on the question of interest about color. For, if the apparent color of surface region R varies with the illumination, then to say that the color of R is stable across changes in illumination is to say that at most one of the distinct apparent colors that R has across a range of illuminations veridically represents the color that R has. Therefore, given that apparent colors are illumination-dependent, we can ask whether colors are illumination-independent by asking which among the various color appearances that R has (when presented under various illuminations) veridically represents the color of R.

In particular, in view of the illumination-dependence of color appearance, the defender of the thesis that colors are illumination-independent owes us a principled reason for singling out (at most) one of the various color appearances that R has as its veridical color appearance - as the unique appearance that veridically represents R's color. I shall not reiterate the arguments I have made elsewhere that such a principled reason is not forthcoming.⁹ Instead, I want to draw two morals of more immediate importance.

First, I want to note that, if counterfactualism is the right understanding of color constancy, then appeals to that phenomenon do not suffice to answer the question just bruited (assertions to the contrary in the literature notwithstanding). For, if counterfactualism is correct, then color constancy tells us nothing that would single out any one of the (phenomenally distinct) color appearances of R as veridical. On the contrary, it tells us only that subjects make counterfactual judgments about a range of color appearances that R would have under a range of illuminations. This is a significant lesson to draw, since, it seems to me, philosophers who appeal to color constancy often do so precisely in order to motivate a choice of a single color appearance among many (e.g., [Tye, 2000], 147-148, [Hilbert, 1987], chapter 5). My first moral, then, is that if counterfactualism is the correct understanding of color constancy, then the phenomenon cannot serve to show that colors are illumination-independent.

My second moral is that, on the contrary, a descriptively adequate understanding of color constancy is fully compatible with the view that colors are illumination-dependent. For if (as I believe) there is no principled reason for singling out any one of the range of color appearances of R as veridical, and if, therefore we conclude that the color of R is an illumination-dependent property of R, we can and should acknowledge the phenomenon of color constancy.

References

[Arend and Reeves, 1986]: Arend, L. and Reeves, A. (1986). Simultaneous color constancy. Journal of the Optical Society of America A, 3(10):1743-1751.
[Arend et al., 1991]: Arend, L., Reeves, A., Schirillo, J., and Goldstein, R. (1991). Simultaneous color constancy: patterns with diverse Munsell values. Journal of the Optical Society of America A, 8:661-672.
[Bäuml, 1999]: Bäuml, K.-H. (1999). Simultaneous colour constancy: how surface color perception varies with the illuminant. Vision Research, 39(8):1531-1550.
[Beck, 1972]: Beck, J., editor (1972). Surface Color Perception. Cornell University Press, Ithaca.
[Brainard et al., 2002]: Brainard, D. H., Kraft, J. M., and Longere, P. (2002). Color constancy: Developing empirical tests of computational models. In Mausfeld, R. and Heyer, D., editors, Colour Perception: From Light to Object. Oxford University Press, New York. Forthcoming.
[Byrne and Hilbert, 1997]: Byrne, A. and Hilbert, D. R. (1997). Readings on Color, Volume 2: The Science of Color. MIT Press, Cambridge, Massachusetts.
[Byrne and Hilbert, 2002]: Byrne, A. and Hilbert, D. R. (2002). Color realism and color science. Behavioral and Brain Sciences. In press.
[Cohen, 2000]: Cohen, J. (2000). Color Properties and Color Perception: A Functionalist Account. PhD thesis, Rutgers University, New Brunswick, New Jersey.
[Cohen, 2002]: Cohen, J. (2002). Color properties and color ascriptions: A relationalist manifesto. Under review.
[Cohen and Meskin, 2003]: Cohen, J. and Meskin, A. (2003). Photographs are not transparent. In preparation.
[Cornelissen and Brenner, 1995]: Cornelissen, F. W. and Brenner, E. (1995). Simultaneous colour constancy revisited: an analysis of viewing strategies. Vision Research, 35:2431-2448.
[Evans, 1948]: Evans, R. M. (1948). An Introduction to Color. Wiley, New York.
[Goldstein, 1999]: Goldstein, E. B. (1999). Sensation & Perception (5th Edition). Brooks/Cole Publishing, Pacific Grove, California.
[Hilbert, 1987]: Hilbert, D. R. (1987). Color and Color Perception: A Study in Anthropocentric Realism. CSLI, Stanford.
[Lennie, 1999]: Lennie, P. (1999). Color coding in the cortex. In Gegenfurtner, K. R. and Sharpe, L. T., editors, Color Vision: From Genes to Perception, pages 235-247. Cambridge University Press, Cambridge.
[Stoerig, 1998]: Stoerig, P. (1998). Wavelength information processing versus color perception:evidence from blindsight and color-blind sight. In Backhaus, W. G. K., Gliegl, R., and Werner, J. S., editors, Color Vision: Perspectives From Different Disciplines, pages 131-147. Walter de Gruyter, Berlin.
[Troost and deWeert, 1991]: Troost, J. M. and deWeert, C. M. M. (1991). Naming versus matching in color constancy. Perception & Psychophysics, 50:591-602.
[Tye, 2000]: Tye, M. (2000). Consciousness, Color, and Content. MIT Press, Cambridge, Massachusetts.
[Walton, 1984]: Walton, K. (1984). Transparent pictures: On the nature of photographic realism. Critical Inquiry, 11:246-276.
[Wyszecki and Stiles, 1967]: Wyszecki, G. and Stiles, W. S. (1967). Color Science. Wiley, New York.
[Zaidi, 1999]: Zaidi, Q. (1999). Color and brightness induction: from Mach bands to three-dimensional configurations. In Gegenfurtner, K. R. and Sharpe, L. T., editors, Color Vision: From Genes to Perception, pages 317-343. Cambridge University Press, Cambridge.

Footnotes:

¹Department of Philosophy, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093-0119, joncohenREMOVETHIS@aardvark.ucsd.edu (omit text in caps, which reduces automated spam)

² The case I have in mind is one in which the subject is visually perceiving a coffee cup, not a photograph of the coffee cup (notwithstanding the arguments of [Walton, 1984], I take it that when a subject sees a photograph of x, she does not ordinarily see x as well; see [Cohen and Meskin, 2003]). Hence, I am using figure 1 to depict the stimulus - I am not using it as the stimulus. This will matter, since it is plausible that there is a variation in the illumination under which the coffee cup is perceived, whereas it is doubtful that there is a significant variation in the illumination under which figure 1 is perceived. For ease of expression, I'll be pretending that the reader can perceive the depicted coffee cup in what follows.

³As noted, invariantism has become the de facto standard understanding of color constancy in both philosophical and scientific work on color. For example, versions of this characterization can be found in many recent textbooks and anthologies on color and vision including ([Byrne and Hilbert, 1997], 445), ([Zaidi, 1999], 339), ([Goldstein, 1999], 567), ([Stoerig, 1998], 141), and ([Brainard et al., 2002], 2). In several cases these paginations point to glossary entries for `color constancy'.

⁴ Why an invariance of apparent color rather than an invariance of color? As I shall discuss below, many invariantists want to appeal to color constancy in the course of arguing for a particular way of individuating colors; as such, it will not suit their purposes to describe the phenomenon in a way that depends on a particular (and controversial) way of individuating colors. In contrast, characterizing color constancy as an invariance in apparent color has the dialectical advantage that it does not beg the ontological questions that the phenomenon is enlisted to answer.

⁵Of course, there is one salient dimension along which the adjacent regions are unlike: they are unlike in respect of the illumination that falls on them. Does this mean that we should set aside subjects' visual discrimination of the regions as a way of determining whether the regions share an apparent color? Surely that policy would be unwise in the present setting; for if we are debarred from considering discriminations between regions that differ in illumination, then we can never decide whether regions are invariant in apparent color across variations in illumination. But variations in illumination are precisely the variations that we must consider in assessing whether or not there is color constancy, as that phenomenon is understood by an invariantist; therefore, the proposed strategy of setting aside subjects' visual discriminations in such cases would mean that these cases - paradigmatic cases, and therefore precisely the cases that an adequate understanding of color constancy ought to capture - could never be regarded as instances of color constancy (on an invariantist account). It seems, therefore, the present response on behalf of the invariantist is unsatisfactory.

⁶Of course, the invariantist could avoid the difficulty we are now discussing by insisting that, when subjects have such conflicting reactions to a pair of surfaces, the case is not an instance of color constancy. But this alternative strategy seems inadvisable. After all, insofar as the pair of conflicting responses is taken as a hallmark of paradigm cases of color constancy, the strategy in question amounts to emptying the phenomenon of instances merely to save a favored theory. Presumably this is unacceptable.

⁷These results have been confirmed by a number of studies. See [Evans, 1948], 163-164 and [Beck, 1972], 66-67 for an overview of some of the earlier work; more recent findings to the same effect are reported in [Arend and Reeves, 1986], [Arend et al., 1991], [Troost and deWeert, 1991], [Cornelissen and Brenner, 1995], and [Bäuml, 1999].

⁸Certainly it is uncontroversial for color scientists, who frequently take such cases to show the limitations on human color constancy (thus, in an entirely typical passage, Peter Lennie writes that "Formal accounts of color constancy characterize mechanisms that perform better than human observers: Human color constancy is imperfect" ([Lennie, 1999], 245-246)).

⁹See [Cohen, 2000] and [Cohen, 2002].

File translated from T_EX by T_TH, version 3.22.
On 18 Feb 2003, 13:34.