A Way Forward on Specification

It's been a while since we've blogged; it's been a crazy summer and we've been insanely busy, but things are settling back down. We have a lot planned for the blog, there's much work to be done on a variety of topics in perception, action, embodied cognition and language. First I want to get back on track with my specification project, so that maybe this can start to move forward.

*****************************
I've been reviewing work that has been undermining the concept of specification in perception. Specification is the idea that the information we detect and use to control our behaviour maps 1:1 with some action-relevant property in the world, and the idea that this is even possible is one of Gibson's key contributions to psychology. Rob Withagen, along with Tony Chemero (hence W&C) have reason to believe that specification is actually too high a bar, and that individual variation in perceptual ability makes it likely that different people will use different, often non-specifying information to solve the same task. This variation is highly likely to be present, because that's kind of how evolution works, and they cite various studies (in collision judgements and dynamic touch) that seem to empirically confirm that this variation exists and the fact that it doesn't go away with fairly extensive practice.

Ecological psychologists insist on specification for a reason: the idea of direct perception seems to require it. Direct perception is Gibson's hypothesis that we perceive the world directly in terms of our ability to act on it, without needing internal mental gymnastics to figure out what is going on. Turvey, Shaw, Reed and Mace (1981) proposed that, in order for this to work, perceptual information had to be generated by lawful processes that produced one and only one information variable per property of the world, and that perception required the organism to detect that one variable. Anything less, and it's not clear how the organism can count on having detected the right property in the world without some internal states tracking probabilities and correlations. 

Withagen and Chemero think that a) evolution demands a less strict policy for perception to work, b) that there is individual variation, that c) this all shows that specification, while possible, is not required for perception (no 1:1 between world and optics or between the optics and perceiver) but that d) perception can still be direct, just on a continuum - the organism has varying degrees of successful contact with properties in the world, rather than the all-or-none contact implied by the Turvey et al analysis. In one respect, they are throwing out the baby and the bathwater but claiming this still lets them have their cake and eat it too. It's exciting if true, but while I think the fact of individual variation needs to be addressed, I'm not yet convinced by the data or the theory and I think there's more to do. Here are some initial thoughts in that direction.

These thoughts  are designed to lead to real science, and if you want in, let us know. Sabrina and I are very interested in the basic question of how information gets it's meaning - Sabrina's language analyses, for example, depend critically on being able to find a way to have ecological style information in a domain where specification can't possibly work. I've been chatting to Tony and Rob about this work and I'm keen to collaborate: but Sabrina and I are interested in developing broad collaborations to go after this stuff from as many angles as possible, including neuroscientifically. The goal here is to develop and propose grants and experiments - we'll worry about access to equipment later, let's first figure out what we need.

We also want this to bring ecological researchers together, not be yet another schism. Rob suggested in an email that I simply put all this in a paper and we could argue back and forth in the literature. I may yet write that paper, just for the record, but I think we can cut through the "talking past each other" bit much faster here on this blog. I would much rather just crack on with a research programme that involves multiple labs all pulling in the same direction for a change.

Some reflections on the evidence so far

My first thought is that the evolutionary story is intriguing, but not that convincing. I found the paper (Withagen & Chemero, 2009) to be full of speculation and vague generalised claims about when picking up information would be hard. I think the biological emphasis is a good idea, but exactly none of this can be resolved one way or the other without data. This data has to come from specific, well characterised tasks; there are no 'in general' solutions here. A task could easily involve numerous 'non-specifying variables' but none of them might be stable enough to challenge the specifying variable for attention, for example. You can only do this in particular, one task at a time (Bingham, 1988).

In addition, while the idea of individual variation in perceptual capabilities is probably correct, it's important to remember that the back end of the perceptual learning system is the brain. We ecological types tend to ignore the brain, but Sabrina and I are working on getting it back in the game, on our terms. The central nervous system is, I think, our fast response system - the part of us that operates on a millisecond time scale and can support the kind of flexibility our slow-to-change bodies require to survive. This kind of high dimensional system offers all kinds of potential to bypass the two problems W&C think we face:

1. individual variation is only a problem if the different systems can't perform the same function. But there is plenty of evidence for extreme degeneracy in nervous systems - many different 'wirings' can produce the same function (the humble lobster gut can produce one particular critical rhythm with any of 450,000 different nervous systems; Prinz, Bucher & Marder, 2001)
2. sub-optimal solutions (local minima in a task solution space) are most easily avoided by high-dimensional systems, ones with room to manoeuvre. This is a standard trick in neural network programming - adding functionality (dimensions) to the network's capabilities creates ways out of minima, because those minima might only exist to trap behaviour in the lower dimensional network.

Whatever the science is that tackles this question, it will have to include neuroscience. Hell, maybe one of the reasons to have a nervous system this flexible is so that perception via specification can work!

There is, of course, data to support the evolutionary analysis - work on individual variation in collision judgements and dynamic touch, which Withagen and Chemero cite heavily. My second thought is the same as the first: the empirical case is intriguing, but not that convincing, and, in the case of the dynamic touch work, entirely off topic.

1. The collisions judgement task is not bad - collisions are a completely describable physical event that creates a describable set of kinematic variables in the optic array. Only one of these variables specifies mass-ratio, while the rest correlate to varying degrees. This is a good task, therefore, except that a) judging mass-ratios is an odd task and b) judgement tasks in general are less stable and way harder to train than action task (compare performance in my action learning vs perceptual learning studies). So it's a good start, but judgement studies need to be placed in context where the relevant information is used to control action (Bingham & Pagano, 1998). Bingham started his phase perception work with judgement studies but only ever as a first step; we got back into action measures as soon as it became feasible and worthwhile.
2. Dynamic touch is no use here. The question at hand is what information variables people use. Dynamic touch tasks involve the non-visual perception of object properties, and there is plenty of evidence that the dynamic property being perceived is the inertia tensor. This is not an information variable, however, and there are no hypotheses about what kinematic information the inertia tensor creates.  Withagen makes a lot of hay from the fact that the inertia tensor seems to be involved in the perception of multiple object properties - this, he says, means that there is a one-to-many relationship between the world and perception and thus, no specification. Of course, the inertia tensor may in fact be doing this by creating multiple sources of kinematic information, each of which is specific to a different property. In fact, this is the ecological hypothesis that retains specification. But we don't know because we don't know what the variables created are. So you cannot use experiments involving dynamic touch to answer questions about which information variables people use.

So, how to progress? We need to first settle on some terminology, the scope of the enquiry and the methods that we can apply. 

Terminology

The first thing that has to go is the term 'non-specifying variable'. As Sabrina is fond of telling me, there is no such thing, not in general. There are, however, 'variables which do not specify the property of interest in the current task'. The former is not simply short hand for the latter - this distinction matters. Remember, there is no general case to be made here: we have to take this one task at a time.

Second, we need to get clear about the difference between the world and information about the world. We are (at least initially) interested in properties of the world called affordances and the information people use to perceive these. This means first running studies to identify the affordances that are actually being perceived (see Mon-Williams and Bingham, 2011 for a master class in this; I will review this paper in detail soon now it's become so relevant). Only then, once you know what affordance property has to be specified, can you start asking questions about information, let alone interpreting the results. The key here is perturbation methods (e.g. Wilson & Bingham, 2008); you identify all the kinematic variables which might be informative about your affordance, and you systematically and selectively perturb them one at a time. The perturbation that breaks skilled performance identifies your information variable. The correlational methods from the collision work are also quite handy, I think.

Third, I think we need to step back a little. We throw around the idea that all perception is based in specification, but of course Gibson was primarily interested in the perception that allowed skilled action, and action measures are the gold standard for identifying variable use (not the only way, of course, just the best). I'm inclined to think that we need to focus initially on the perceptual control of action by the perception of affordances, and not conflate this topic with the perception of non-action related information. If we do this, and (one way or the other) shore up our understanding of the former, then we can get back to expanding the ecological approach. Right now however the field simply isn't working together hard enough and we need to regain some focus in order to be able to move on.

The Task Specific Approach

W&C make a big deal about how Turvey et al were wrong to base their defence of ecological psychology in physics, and want to move instead to developmental biology. Getting the organism side of things back to the fore is great; but the reason Turvey et al did what they did is because they were talking about the lawful process by which events in the world can create information about those events. This is (ecological) physics, and it's the correct starting point for any ecological analysis.

We therefore need specific tasks which we can characterise in sufficient detail to understand the affordance property to be perceived and the potential sources of information for the perception of that affordance. Coordinated rhythmic movement has been such a workhorse in the literature because it allows precisely this kind of analysis (strictly speaking it's event perception, but the principle is the same). Bingham's model handles all the key features of the task except learning. I have a whole raft of experiments no one wants to fund right now designed to go after learning, amplitude control and more, which would provide the perfect backdrop for investigating individual variation and information use.

We need other tasks, however, to extend the argument. Interception is a classic and ripe for learning studies too. There are many potential variables and solutions (e.g. Tresilian, 1999) and you could do both judgement and action studies to complement each other and allow the necessary perturbations to be applied. I'm also working on long distance throwing to a target which has all kinds of potential (because, as an example of projectile motion, the physics are tractable) although this is complicated by the need to perceive object properties via dynamic touch. Perception of distance to the target remains an option, however, and there are many useful action measures to take, plus rich simulation work that is letting us get a good handle on the control laws. More on this soon, I will be getting my head into these papers shortly.

The essence is this: I think that which variables are harder to pick up than others depends on the relative stability of the variables present in a task (in terms of how reliably they are present and for how long) as well as their relation to the property in question. You can only answer these questions within the confines of a clearly defined ecological task space.

Thoughts on other tasks welcome - you have to be able to characterise both the world properties and the potential information for those properties, and there has to be a way to take action measures. But I think it's an error to simply abandon the TSM 'laws' style of analysis, and I think we should instead couple it to the developmental biology W&C favour to get the organism back into the picture.

Summary

This topic has made me realise that we are all off slaving away in our own little silos and we aren't really talking to one another. Geoff Bingham's been addressing so many of these empirical issues it's ridiculous (seriously - every time I realise something, I also tend to realise Geoff beat me there by 20 years) but exactly no one is reading or citing his work, not even ecological psychologists who should know better. The same goes for most of the rest of us. Sabrina and I want to actually change the way psychology is done, and this means working together with as many people as possible in common cause.

So anybody who thinks they have something to contribute to taking this forward, please either leave a comment or email us. I'd prefer the former - this needs a conversation and I'd like us to have it where we can all see and contribute, but I know some people are less into being active on the web. Be specific about your interest, and what you can and can't do (so we can identify gaps that need filling). If you have comments or thoughts about what I have here, let's hear them, positive or negative. Let's find an actual project we can build up and fund, and let's actually try and answer this goddamn question empirically.

References
Bingham, G.P. (1988). Task specific devices and the perceptual bottleneck. Human Movement Science, 7, 5-264. (Download)

Bingham, G.P. & Pagano, C.C. (1998). The necessity of a perception/action approach to definite distance perception: Monocular distance perception to guide reaching. Journal of Experimental Psychology: Human Perception and Performance, 24 , 145-168. (Download) 

Mon-Williams, M. & Bingham, G.P. (2011). Discovering affordances that determine the spatial structure of reach-to-grasp movements. Experimental Brain Research, 211(1), 145-160. (Download)

Prinz, A., Bucher, D., & Marder, E. (2004). Similar network activity from disparate circuit parameters. Nature Neuroscience, 7 (12), 1345-1352. (Download)

Tresilian, J.R. (1999). Visually timed action: Time-out for “tau.” Trends in Cognitive Sciences, 3, 301–310.

Wilson, A. & Bingham, G.P. (2008). Identifying the information for the visual perception of relative phase. Perception & Psychophysics, 70(3), 465-476. (Download)