Visual illusions and direct perception

A while back I reviewed a bunch of papers by Rob Withagen who is currently arguing that while perception is not typically based in specifying variables, it can and should still be ecological in nature. While we are also developing an account of information that goes beyond specification I still have some reservations about the details of Rob's work. That said, I do think there is a lot of overlap and I'm still interested in figuring out what that is. 

Rob's latest paper (de Wit, van der Kamp & Withagen, 2015) is about visual illusions, and how to talk about them ecologically. There is a lot of very useful stuff in here, not least the review of the many things Gibson said about illusions. After this review, the paper tries to put all of this into Rob's and Tony Chemero's evolutionary framework and uses this to formalise and tidy up what Gibson was up to. I'm adding this paper to the set I've covered from Withagen as I continue to think through these issues.

A Way Forward on Specification

It's been a while since we've blogged; it's been a crazy summer and we've been insanely busy, but things are settling back down. We have a lot planned for the blog, there's much work to be done on a variety of topics in perception, action, embodied cognition and language. First I want to get back on track with my specification project, so that maybe this can start to move forward.

*****************************
I've been reviewing work that has been undermining the concept of specification in perception. Specification is the idea that the information we detect and use to control our behaviour maps 1:1 with some action-relevant property in the world, and the idea that this is even possible is one of Gibson's key contributions to psychology. Rob Withagen, along with Tony Chemero (hence W&C) have reason to believe that specification is actually too high a bar, and that individual variation in perceptual ability makes it likely that different people will use different, often non-specifying information to solve the same task. This variation is highly likely to be present, because that's kind of how evolution works, and they cite various studies (in collision judgements and dynamic touch) that seem to empirically confirm that this variation exists and the fact that it doesn't go away with fairly extensive practice.

Ecological psychologists insist on specification for a reason: the idea of direct perception seems to require it. Direct perception is Gibson's hypothesis that we perceive the world directly in terms of our ability to act on it, without needing internal mental gymnastics to figure out what is going on. Turvey, Shaw, Reed and Mace (1981) proposed that, in order for this to work, perceptual information had to be generated by lawful processes that produced one and only one information variable per property of the world, and that perception required the organism to detect that one variable. Anything less, and it's not clear how the organism can count on having detected the right property in the world without some internal states tracking probabilities and correlations. 

Withagen and Chemero think that a) evolution demands a less strict policy for perception to work, b) that there is individual variation, that c) this all shows that specification, while possible, is not required for perception (no 1:1 between world and optics or between the optics and perceiver) but that d) perception can still be direct, just on a continuum - the organism has varying degrees of successful contact with properties in the world, rather than the all-or-none contact implied by the Turvey et al analysis. In one respect, they are throwing out the baby and the bathwater but claiming this still lets them have their cake and eat it too. It's exciting if true, but while I think the fact of individual variation needs to be addressed, I'm not yet convinced by the data or the theory and I think there's more to do. Here are some initial thoughts in that direction.

These thoughts  are designed to lead to real science, and if you want in, let us know. Sabrina and I are very interested in the basic question of how information gets it's meaning - Sabrina's language analyses, for example, depend critically on being able to find a way to have ecological style information in a domain where specification can't possibly work. I've been chatting to Tony and Rob about this work and I'm keen to collaborate: but Sabrina and I are interested in developing broad collaborations to go after this stuff from as many angles as possible, including neuroscientifically. The goal here is to develop and propose grants and experiments - we'll worry about access to equipment later, let's first figure out what we need.

We also want this to bring ecological researchers together, not be yet another schism. Rob suggested in an email that I simply put all this in a paper and we could argue back and forth in the literature. I may yet write that paper, just for the record, but I think we can cut through the "talking past each other" bit much faster here on this blog. I would much rather just crack on with a research programme that involves multiple labs all pulling in the same direction for a change.

Evolution vs.Specification (Specification V)

One of Gibson's key contributions was to reveal that it was possible for the optic array to specify a meaningful property of the world. Gibson insisted that specification existed between the world and optics (each property produced one unambiguous pattern, and thus the mapping is 1:1). Specification, said Gibson, meant direct perception was possible, because picking up that one variable meant perceiving the one property that caused it. 

Turvey, Shaw, Reed & Mace (1981) formalised this idea by describing how ecological laws governed which properties of the world could be specified and identifying that these laws allowed affordances into this set. Turvey et al (hence TSM, because Reed changed his mind later on) then insisted that, in order for perception to be direct, specification also had to exist between the optics and the perceiver; an organism should only use one variable per property, and thus the mapping from world to perceiver is 1:1:1. This is a very high bar, and was put in place to defend ecological psychology from the Establishment attack (Fodor & Pylyshyn, 1981).

Withagen & Chemero (2009) think that the 1:1:1 account is incompatible with evolutionary thinking, and they aren't hot on the 1:1 account either. Specifically, they think that any given species will show individual variation in it's members ability to use information, and that in many cases species will end up using sub-optimal solutions (two important elements of evolutionary thinking). The1:1:1 bar, they say, is implausibly high and a naturalised theory of perception (one that is compatible with evolution) will instead predict the common use of non-specifying information. They also claim that this does not stop perception from being direct, so long as you allow 'directness' to live along a continuum.

I think there are some important issues here, but I think this paper's presentation is problematic. It contains no analysis of any particular information or task, and instead is full of sentences such as 'it seems more plausible to us that' and 'it is possible that'. This comes off as the kind of woolly evolutionary thinking psychology is rightly scolded for. Gibson and TSM spent a lot of time trying to make us pay less attention to what might be and more to what is.My concerns are mostly along these lines, and once I get them off my chest I want to turn in future posts to some ideas for a research programme to pursue this all in more detail.

Individual Variation in the Use of Perceptual Information (Specification IV)

If it is the case that perception requires the use of specifying variables, then there should be no individual variation in what information variables people use. However, as we've already seen, such variation exists: the dynamics of a collision event produces multiple kinematic patterns in the optic array, and people judging the mass ratio of colliding balls use all of these, only one of which actually specifies which ball is heavier. Even with training, people do not always find the specifying variable. 

This is an example of how the mapping between a property in the world (mass ratio) and the optic array can be one-to-many, with consequences for perception. Figure 1 on this post shows us that there is another mapping to investigate, namely the one from a perceptual array to the organism. Can this mapping also be one-to-many?  Withagen & van Wermeskerken (2009) suggest that it can, and that again training does not necessarily help.

Non-Specifying Variables in the Perception of Collisions (Specification III)

Part of Withagen's critique of specification and whether it's necessary to underpin direct perception is a brief review of some empirical literature that shows people using non-specifying variables. I want to spend a few posts reviewing these, because all good potentially sensible ideas need data to confirm whether they're right or not.

First up, the perception of relative mass after a collision. Events in the world are dynamic, that is, they involve motion caused by a pattern of underlying forces. Perceptual systems want access to the underlying dynamics of events, because this is the level at which the event is defined (Wilson & Bingham, 2001). However, perceptual systems can only detect kinematics, that is, motion - this is the perceptual bottleneck  (Bingham, 1988 and this note on dynamics and kinematics). We can only perceive the underlying dynamics of an event, according to the ecological approach, if we can detect motion that is specific to that dynamic. Runeson coined the phrase kinematic specification of dynamics (Runeson & Frykholm, 1983) and investigated whether there were such kinematic patterns and whether we can detect them. Working with Claire Michaels and David Jacobs, he has also investigated the use of non-specifying variables.

How Information Gets Its Meaning (Specification II)

Gibson proposed that specification was required in order for perceptual information to have meaning that was tied to the world in a manner an organism could use. The concept of specification has been placed back under the microscope by recent theoretical and empirical work. Here I want to briefly summarise the theoretical argument put forth by Withagen & van der Kamp (2010), who worry that specification places too strong a constraint on what a perceiving-acting organism might find informative. They suggest that (visual) perception can still be direct with non-specifying patterns, if you stop thinking information is in the relation between the environment and the optic array but rather, in the relation between the optic array and the organism. They propose this because recent empirical work suggests that organisms can happily get around using non-specifying variables; they want to keep directness, however and they don't think Chemero's solution to the problem does the trick. I'll review the studies they cite over the next few posts; first, let's lay out the solutions they propose.

Again, I want to emphasise that this is very much a work in progress for me. I'm using these posts to come to grips with the arguments, and I don't yet endorse any of these various critiques. My goal is simply to have a clear understanding of what everyone says, so that we can evaluate those claims later on when I review some data.

Specification: What It Is, and Why We Need It (Specification I)

The first thing I need to do in a discussion of specification is explain what it is and why it's important to ecological psychology. I've tried to maintain a clear logical progression in this post, building towards the need for specification. In my next post, I'll take a first swing at explaining what specification gives us, namely a reason why information means one thing and not another.

The issue of specification comes from Gibson's (1966, 1979) analysis of visual perception, so that's where I'll start too. Most descriptions of visual perception begin with the anatomy of the eye; people note that the eye resembles a camera, and that the lens seems to focus a messy, upside down image onto the retina. The retina then pixelates that image into neural activity, and this pixelated structure then shows up in primary visual cortex (this is topographic mapping). If vision does indeed begin this way, then a huge amount of work seems to be required to take this impoverished stimulus and use it as the basis for the rich, 3D visual world we experience.

Gibson's ecological theory begins with a re-evaluation of the stimulus for vision. The first three chapters of the 1979 book are about the world and what it contains, while chapter 4 is about how this world can interact with light to produce information. Only once he lays out the information available to the organism does he begin to talk about the act of perception itself; this re-evaluation of the 'job description' for a visual system is one of his most important contributions to psychology. Gibson's reanalysis leads him to conclude that action relevant properties of the world (specifically, affordances) can be specified in the optic array, and this concept underpins the directness of his theory of perception.

The issue of specification is assumed to be critical for the success of a direct theory of perception. The traditional views propose a 'many-to-one' mapping; a given pattern of stimulation on the retina is ambiguous because it could be caused by many possible states in the world. Specification is the hypothesis that there is a 'one-to-one' mapping - a given pattern in the optic array comes from one and only one state of the world. This can happen, according to Turvey, Shaw, Reed & Mace (1981) if (and only if) the creation of information about the world is a lawful process. If the projection of world into optics is underwritten by a law and thus one-to-one, then detecting the optical pattern is equivalent to detecting the property of the world: detecting the information is perceiving the world, with no additional processing work required. Perception can be direct.

A theory of direct perception will require several elements: there must be invariant structure within the endless flow across the retina that relates 1:1 to some property of the world. To be invariant, this structure must be relational, and therefore higher order. If perception is to be direct, these higher-order invariants must be detectable as a piece, and not built out of their elements in some post-perceptual process. Only if you have all this do you have the possibility of a one-to-one mapping between the world and vision, i.e. the possibility of specification.This post lays out what this all means, and  how these pieces come together in ecological psychology.

Specification & Its Discontents

A topic that has been flying under the radar a little in Sabrina's language posts is the issue of specification. Sabrina's ecological analysis of language discusses information and what it means, but is not committed to the kind of law based account that is typically invoked in the perception-action literature. It can't - language can be used to talk about things in their absence, and it's not clear what kind of ecological laws might govern the connection between the speech event and it's meaning. Breaking specification has consequences, however, and has been a topic of some debate lately; the next few posts from me will be an in-depth look at the theoretical and empirical contributions to the debate as I try to come to terms with the idea and whether it can still support a direct theory of perception.