Tuesday 23 September 2014

Visual illusions and direct perception

A while back I reviewed a bunch of papers by Rob Withagen who is currently arguing that while perception is not typically based in specifying variables, it can and should still be ecological in nature. While we are also developing an account of information that goes beyond specification I still have some reservations about the details of Rob's work. That said, I do think there is a lot of overlap and I'm still interested in figuring out what that is

Rob's latest paper (de Wit, van der Kamp & Withagen, 2015) is about visual illusions, and how to talk about them ecologically. There is a lot of very useful stuff in here, not least the review of the many things Gibson said about illusions. After this review, the paper tries to put all of this into Rob's and Tony Chemero's evolutionary framework and uses this to formalise and tidy up what Gibson was up to. I'm adding this paper to the set I've covered from Withagen as I continue to think through these issues.

Review: What Did Gibson Have to Say About Illusions?
On the face of it, visual illusions seem to be real trouble for the ecological approach. Remember that the standard cognitivist line is that the light hitting the retina does not contain enough information to specify the world, and that visual perception as we know it is only possible if the organism can fill in the blanks via some kind of inference. Visual illusions are ambiguious experiences that seem to make it clear that a) the world is underspecified and b) our inferences can be wrong, leading to demonstrable errors. 

Because they are so troublesome (on the surface, anyway) Gibson was often accused of simply ignoring illusions. This was, of course, not true. Withagen points out that Gibson spent a lot of time in the 1966 and 1979 books talking about illusions, and he also had a very prominent debate with Ernst Gombrich about the perception of pictures (archived here). 

Withagen notes that Gibson's discussion of illusions is nested within his analysis of pictures (which I have discussed briefly here) because visual illusions are typically (although not always these days) pictures. Gibson's definition of a picture is
A picture is a surface so treated that a delimited optic array to a point of observation is made available that contains the same kind of information that is found in the ambient optic arrays of an ordinary environment. (Gibson, 1971, pg 31)
Gibson's analysis of illusions in pictures is, of course, grounded in information. Take the Muller-Lyer illusion:

Figure 1. the Muller-Lyer illusion
The inference story (e.g. Gregory, 1968) is the people infer that the bottom line is farther away and therefore larger (in order to project the same retinal size). The information story (Gibson, 1966) is that the information for 'length of line' has been altered by the presence of the arrowheads. This account also offers up an interesting fact: there is still information specific to the length of the line present in the display, and so people might be able to access that (given time, or training). 

In support of Gibson, Withagen points out that there is, in fact, wide individual variation in the magnitude of the illusion (from Heinrich, Heine & Norenzayan, 2010, the famous 'WEIRD' paper).

There is also the well-known effect that visual illusions affect perceptual judgments about size, etc but do not always affect reaching behaviour (Goodale & Milner, 1995). There therefore seems to be information specific to the actual length of the line in the display, and people sometimes use it to correctly perceive line length, but crucially, they don't always use it, and when they don't they fall prey to the illusion.

Withagen is very excited about individual variation in information use (see here and also here and here). His account of information use (with Chemero) is supposed to be consistent with evolutionary biology, and for evolution to be at work there must be variation for it to work with. This variation means that sometimes an organism will settle on a non-specifying variable, and this seems to be what's going on with illusions.

Being Ecological About Illusions
Currently, for Gibson and the ecological approach in general, in order for an optical variable to be information, it has to specify. Withagen thinks this is a problem and that illusions really do point this problem out. However, his ecological solution is to simply relax the specification requirement and allow organisms to vary in how tight their perceptual, epistemic contact with the world is. Any optical variable can now be information, but it will connect you to varying degrees to the world:
In other words, when a specifying optical variable is detected, the epistemic contact is perfect, but this contact weakens when the exploited variable correlates less with the to-be-perceived property, that is, is less useful. This idea of a continuum of direct epistemic contact is a slight alteration of Gibson's conception, but one that makes it more compatible with both evolutionary considerations and with the empirical data. It states that in the processes of evolution, development and learning, observers are in continuous epistemic contact with the environment, but that the strength of their grip on the environment improves when informational patterns are discovered that correlate more highly with the property of interest.
de Wit et al, pg 8
This is still non-inferential perception, and the quality of your behaviour is a function of the degree of fit the variable you are using gives you. In the Muller-Lyer example, when you attend to the information that is not specific to length but that only co-varies with it, you fall for the illusion (lower quality behaviour). When you successfully attend to the information specific to length (e.g. during an action task, or if you are from a non-WEIRD population) you don't fall for the illusion and you produce higher quality behaviour. 

There are some nice things in here:
  • I really like the information based analysis of the dorsal/ventral stream experiments where 'vision for perception' falls for illusions but 'vision for action' does not. This could easily still involve a dorsal/ventral stream story, but the point would be that the difference in behaviour does not come from where in the brain the information is processed but from the nature of the information being processed. This is precisely an embodied kind of way to think about this existing data set. If the judgment task and the action task can look at the same display but effectively see different things because they direct attention to different information, this is cool.
  • The review of Gibson on illusions is a handy resource. Next time someone says 'illusions disprove ecological psychology' I'll just point that person here. I do think they missed a trick by not getting into Runeson's amazing ecological analysis of the Ames Room, but then I'm the only person I know who knows about it so they get a pass there :)
I am more sympathetic to this work than I was when I last worked on these papers, thanks to Sabrina's hard work on language taking better and better shape. I do still think Withagen's account is incomplete and problematic in places; I think the argument about evolution with Tony is still too philosophy oriented and I think they need to spend more time thinking about the actual process by which structures in perceptual arrays gets created and how that structure comes to be informative by virtue of what we use it for. I will be reviewing this work again as I work on a paper with Sabrina and I hope to figure out how to build on this work rather than reinvent the wheel. 

de Wit, M. M., van der Kamp, J., & Withagen, R. (2015). Visual illusions and direct perception: Elaborating on Gibson's insights. New Ideas in Psychology, 36, 1-9. (Download $$)


  1. Another way of accounting for the perplexities of the Müller–Lyer diagram is to conceive of its affordances in terms of representational potential. As competent users of depictive representations we can use the Mülller–Lyer as a spatial depiction or else we can use it as a flat rendering of black marks. People unfamiliar with depictive representations (what Hudson (1960) described this as "pictorially illiteracy") are not susceptible to such illusions (as Segal et al also recorded) and therefore they are only capable of using the Müller-Lyer as what Donald Brook calls a "Matching representation". The vital thing to bear in mind here is that perception is by no means a conceptual skill. That means to say that perception is a non-inferential capacity to put things to communicative use. I have written about this in much more depth here:

    1. It's just occurred to me that this is effectively a restating of Withagen's idea; he also proposes that the individual variation is caused by our differing ability to interact with the picture, but he frames it in terms of access to information variables that support the various uses.

  2. "Currently, for Gibson and the ecological approach in general, in order for an optical variable to be information, it has to specify. Withagen thinks this is a problem and that illusions really do point this problem out. However, his ecological solution is to simply relax the specification requirement and allow organisms to vary in how tight their perceptual, epistemic contact with the world is."

    I remain suspicious of this. IF we are to go this direction, then the most obvious thing to do is to simply define perception as a connection with the world through specifying invariants. This seems to me to be a crucial part of Gibson, of TSM, and of Tony's 2.0. By this route, to say that some individual's connection with some aspect of the the world is not through specifying invariants, is to say that it is not perceptual.

    "What are they then?" you might ask. I think Gibson would say that they are cognition, and that in such cases the standard psychology account is relatively accurate. He would say that the interesting question is how far you can push perception (as defined above), before you need to resort to those other things. While I could support that with many quotes, frankly, I don't think this a deeply held first principle. Rather, I suspect it was a simple acknowledgement that he had a more vested interest in some questions in psychology, and that this necessitated the neglect of other questions.

    The problem is, I think, that Gibson and TSM rightly resist the inclusion of non-specifying energy patterns in perception, because once you do that it is much less clear how the ecological approach is unique. If there is some pattern the organism is responding to, and that pattern does specify something, but the something isn't the thing we think it should be... that is a very different situation than one in which we allow non-specifying patterns into the perception story.

    I will have to read the article to see where exactly it stands on this. I note that the excerpt above (which I really like!) does not use the word perception.

    P.S. I can't resist mentioning that Gibson-on-illusions grows naturally out of Holt-on-illusions. If you are working on this seriously without at least skimming my chapter on that topic, I am going to be offended. :- (

  3. This comment has been removed by a blog administrator.

  4. Thank you for your analysis, because it's really useful for me.