Friday, 15 June 2012

Non-Specifying Variables in the Perception of Collisions (Specification III)

Part of Withagen's critique of specification and whether it's necessary to underpin direct perception is a brief review of some empirical literature that shows people using non-specifying variables. I want to spend a few posts reviewing these, because all good potentially sensible ideas need data to confirm whether they're right or not.

First up, the perception of relative mass after a collision. Events in the world are dynamic, that is, they involve motion caused by a pattern of underlying forces. Perceptual systems want access to the underlying dynamics of events, because this is the level at which the event is defined (Wilson & Bingham, 2001). However, perceptual systems can only detect kinematics, that is, motion - this is the perceptual bottleneck  (Bingham, 1988 and this note on dynamics and kinematics). We can only perceive the underlying dynamics of an event, according to the ecological approach, if we can detect motion that is specific to that dynamic. Runeson coined the phrase kinematic specification of dynamics (Runeson & Frykholm, 1983) and investigated whether there were such kinematic patterns and whether we can detect them. Working with Claire Michaels and David Jacobs, he has also investigated the use of non-specifying variables.

Runeson uses judgments of relative mass as an exemplar task. He simulates a collision between two moving balls of varying mass; their behaviour after the collision reflects which one is heavier. There are several kinematic variables available after the collision, including exit speed and scatter angle (the angle between the ball's original and new trajectory). These do not specify relative mass, and how they correlate with it depends on details of the collision. They often feature in cognitive models of judgements in this task, acting as cues refined by heuristics (eg Gilden & Proffitt, 1989).

Runeson & Vedeler (1993; Runeson, 1995) identified a kinematic variable that specifies the mass ratio: the relative amount of velocity change. They then demonstrated that participants used this invariant rather than any of the cue-heuristic approaches. One issue with this study was the fact that all the observers had at least one experiment's worth of experience with the task, and Jacobs, Michaels & Runeson (2000) therefore investigated whether there was any perceptual learning in the task that might be causing different labs to find different information being used.

8 naive observers performed baseline and post-training judgements of mass ratio, with extensive training in between (288 trials split over 4 blocks and two days). Participants improved with training (their judgements became more accurate and less variable); the question was then, what had changed to support this learning?

Jacobs et al correlated the judgements people generated for each collision with the answers each of 5 possible variables would predict; the invariant 'relative amount of velocity change', (INV), exit speed difference (ESD), scatter angle difference (SAD), exit speed and angle differences combined (ESA) and a heuristic model from Gilden & Proffitt (1989; HM). They looked at these correlations in each block of trials (1=baseline, 2-5 = training, 6 = post training), and the results are in Figure 1.

Figure 1. Correlations between judgments and various sources of information about the mass ratio, for each observer across all sessions (Figure 5 from Jacobs et al, 2000)
The first thing to note are the individual differences; each observer starts out most likely relying of different information (e.g. Observers 2 & 4 both begin using exit speed). Most of the other observers, are using either the invariant or the linear combination of exit speed and angle (ESA); part of the problem here is that these variables are themselves highly correlated (r=.93) and thus it's not possible to disentangle them from these data (but see below). The second thing to note is that with practice, most of the observers were producing judgements with high correlations to the specifying invariant. Finally, the heuristic model rarely outperformed the other variables. 

We therefore have evidence of different people using different variables to judge the same event, and most of these variables did not specify the dynamic property being judged. With training, however, people tended to switch and begin to rely on the invariant (or a non-specifying variable that happened, for these collisions, to correlate highly with the invariant).

De-correlating the candidate variables
Jacobs, Runeson & Michaels (2001) ran a follow up study to cope with the problem that ESA and the invariant were highly (.93) correlated. There simply may not have been enough instability in performance controlled by ESA to drive further exploration of the space for better information, so Jacobs et al created some.

Experiment 1 tested baseline performance using the collisions from Jacobs et al (2000), then trained people with collisions from one of two sets. The global constraint set were collisions where the correlations corresponded to the set of all possible collisions, and the local constraint set were collisions from a limited set where all the variables specified the mass ratio. Participants in the global constraint group replicated Jacobs et al, and all changed variable use to either the invariant or ESA (which correlated at .89 here). Participants in the local constraint group never changed variable - whatever they began using, they stuck with. They therefore ended up using different (but in their experience, equally effective) information variables and didn't find the actual invariant.

Experiment 2 then trained people on sets of collisions in which the non-specifying information variables either did not vary over collisions between balls of different masses (no variation) or had no correlation to the actual mass ratios (zero correlation). A random condition randomised the precollision velocities and the various non-specifying kinematic variables had intermediate correlations with the actual mass ratio. In the no variation group, most observers came to find the invariant during training, but interestingly often switched back to, say, exit speed in the post-test when there was useful variation and correlation for this variable again. They therefore haven't come to rely on the invariant, although they had come to detect it. The zero correlation training group stopped using  non-specifying variables during training where they were not informative about mass ratio but didn't always succeed in finding the invariant; if not, their performance simply remained poor throughout. As in Experiment 1 and Jacobs et al, the random group often didn't find the invariant because the non-specifying variables correlated fairly well with mass ratio.

Finally, Experiment 3a trained people in a zero correlation condition where only one alternative (either exit speed or scatter angle) had zero correlation with mass ratio. This meant each collision set had the invariant and one other fairly good (.75) information source. The speed zero correlation group all stopped using exit speed and mostly found the invariant; the angle zero correlation group weren't using scatter-angle to begin with and never tried to; in addition they mostly didn't switch from what they started with because it was either the invariant or exit speed (.75 correlation) and both worked well enough. Experiment 3b took people from the angle zero correlation group who were still using exit speed and trained them again with a speed zero correlation training set; here, 3 of the 4 observers switched variables.

The lessons here: different observers began using different variables which correlated to varying degrees with the dynamic property being judged, relative mass. People were very sensitive to the collision ecology in which they were operating - after training, people typically used an informative variable, but this was not always the invariant. When other information was available and sufficiently correlated to relative mass, people often settled on the non-specifying variable. When prompted to continue their search by a change in collision ecology, however (Jacobs et al, 2001, Expt 3b) people were able to continue their search and typically found the invariant.

The take home message: people learn according to the local constraints of the event ecology and will therefore not always find the specifying invariant. How we perceive our environment depends on our personal developmental histories, and we use whatever works well enough to succeed under the constraints at the time. Change the constraints and the system will search for new information, but without that drive, the perceptual system will settle into locally optimal but globally non-specifying solutions. The scope of our experience is critical.

While the perception of relative mass is a slightly odd task, there is a clearly defined and available specifying variable so the task has the right kind of ecological validity (the definable kind!). The fact that people probably don't tend to do this task if left to their own devices makes the necessity of some training unsurprising, however. Of course, this is true of everything, even the things we do get up to on a regular basis; we had to learn how to do all of it. This task is therefore an okay model task for the question at hand.

That said, this is just a judgement task, and while it's a legitimate place to start it's not an action measure of perception and is thus an indirect way to evaluate perception. I've used judgments of relative phase before to measure visual perception of phase information; the results basically work out, but are very noisy and the training without action takes a very, very long time to work (up to 1500 trials, compared to about 40 or 50 for the action learning task). So while these data are relevant to the case Withagen is making, it's far from the kind of data required to topple something like specification and are, at best, a hint of something worth looking at further.

Gilden, D. L., & Proffitt, D. R. (1989). Understanding collision dynamics. Journal of Experimental Psychology: Human Perception & Performance, 15, 372-383.

Jacobs, D., Michaels, C., & Runeson, S. (2000). Learning to perceive the relative mass of colliding balls: The effects of ratio scaling and feedback Perception & Psychophysics, 62 (7), 1332-1340 DOI: 10.3758/BF03212135

Jacobs, D., Runeson, S., & Michaels, C. (2001). Learning to visually perceive the relative mass of colliding balls in globally and locally constrained task ecologies. Journal of Experimental Psychology: Human Perception and Performance, 27 (5), 1019-1038 DOI: 10.1037//0096-1523.27.5.1019

Runeson, S., & Frykholm, G. (1983). Kinematic specification of dynamics as an informational basis for person and action perception: Expectation, gender recognition, and deceptive intention. Journal of Experimental Psychology: General, 112, 617-632.

Runeson, S. (1995). Support for the cue-heuristic model is based on suboptimal observer performance: Response to Gilden & Proffitt (1994). Perception & Psychophysics, 57, 1262-1273.

Runeson, S., & Vedeler, D. (1993). The indispensability of precollision kinematics in the visual perception of relative mass. Perception & Psychophysics, 53, 617-632. 

Wilson, A. D., & Bingham, G. P. (2001). Dynamics, not kinematics, is an adequate basis for perception – Commentary on Shepard (2001). Behavioral and Brain Sciences, 24(4), 709-710. Download


  1. I really like these experiments, and you summarize them well. However, I'm still stuck thinking that the word "information" is just getting in the way. Some of your ways of using the term would in the above post would be held invalid by some of the systems being discussed... by narrow definition.

    I was meditating a lot yesterday about the discussion on the last post (there was much driving to be done). I think they key term to these disputes is "specify" (specification, specific to, etc.) not "information".

    1. This is all about specification as the way information gets its meaning. So yes, specification is the crux right now.

      I don't really see the term 'information' as all that problematic just yet; whatever ends up containing meaning to the organism, that's what information is. The issue is actually about what gets the meaning, ie via specification or something else.

  2. whatever ends up containing meaning to the organism, that's what information is.

    I'm not sure you have said anything that succinct until now. It is helpful, sort of.
    This next sentence gives me concern though.

    The issue is actually about what gets the meaning, ie via specification or something else.

    I am even more convinced that the terms are getting in the way. I assume we are in agreement that the "meaning" in question is in the world, right? at least in so much as certain states of the world allow certain actions on the part of the organism?

    If so, then there are a few red-flags in your, admittedly off the cuff, reply: The meaning is not "contained" inside of the ambient array or any sub-part of the array involved in any process, and the "what gets the meaning" is the organism. The "meaning is contained in an intermediary" game is part of the traditional approach that Gibson rejects. (Fodor and Phylsyn miss this completely.)

    One of the main Pragmatist principles (as Peirce explained it) is that any two ideas that have all the same consequences are the same idea. I'm still struggling to see how the TSM and the Withagen story have different consequences. (This, by the way, is not independent of my struggle to articulate the difference between the "Two Ecological Approaches". I desperately want you to convince me there is something bigger here, so I can steal your style of convincing.)

    1. There's a small issue of me not quite being precise here. By 'what gets the meaning' I was just referring to the world>optics>organism set up from TSRM (1981). For them, meaning gets into the equation during the projection from world to optics. For Withagen (here, anyway) meaning gets into the equation during the perception of optical structure by an organism. So yes, the meaning in question is something about the world (although affordances are properties of the world defined with respect to the organism, so it gets complicated; Gibson's 'points both ways' that freaks people out). And yes, it's the organism that gets the meaning. The issue is simply what is providing the organism with access to that meaning; specification or exploration?

      The key difference between TSM & Withagen is that only the latter allows a non-specifying variable to potentially serve as information about some property of the world. Whether or not this is a difference that makes a difference is something I'm still working on with these posts.

    2. The key difference between TSM & Withagen is that only the latter allows a non-specifying variable to potentially serve as information about some property of the world.

      But this cannot mean Withagen allows a non-specifying variables to be specifying, right? So what does it mean to say that he allows non-specifying variable to serve as information? Does it mean a non-specifying variable can be the thing that guides an organism's behavior? If so, how is this a controversy?

      I mean, I know it is a controversy, because I have seen Turvey argue about it at meetings, and I have argued with him about it... but it is silly.

      We know that non-specifying variables often guide behavior. (And in many other cases organisms entrain their behavior to variables that do not specify the world-property relevant to the task at hand.) The notion that perception evolved and develops, and that behavior is not perfect, all require that organisms sometimes guide their behavior using non-specifying variables. So, except that Turvey is a dogmatist-in-denial, I'm not sure what there is to discuss.*

      *I'm never quite sure if Turvey is a dogmatist-in-denial or if he just plays one on TV, under the assumption that it is the correct way to guide the field. Given his track record, it is more difficult to argue that the latter view is misguided.

    3. We know that non-specifying variables often guide behavior. (And in many other cases organisms entrain their behavior to variables that do not specify the world-property relevant to the task at hand.) The notion that perception evolved and develops, and that behavior is not perfect, all require that organisms sometimes guide their behavior using non-specifying variables.
      This is the crux of the argument. Name an example from outside a constructed lab setting where we know that organisms entrain their behaviour to non-specifying variables. I don't mind lab versions of real tasks, but lab tasks where specification isn't an option is no use.

      And here's the issue: if this is the case, then, according to TSM, perception cannot be direct. Withagen et al still want directness and try to keep it, but as far as the TSM account goes they've lost directness already and it is as yet an open question as to whether their alternative can support it.

    4. This is silly. Even the ecological solution to the size-weight illusion entails people attuning to the wrong information, i.e., information specifying throw-ability, not weight. And yes, I can turn "weight" into an affordance if required. However, that isn't really the issue:

      Take any task in which you think people eventually entrain to a specifying variable, and watch what they do before that! Most people suck at catching fly balls, sometimes people try to step up steps that are too high for them, try to cross gaps that are to long, etc.

      There was this woman named Eleanor, she studied some of that stuff, won some awards even. I think she called it "perceptual learning", though her first paper on the subject had two authors, "J.J." I think. She even had some students who went on to do good things (along with one who went on to do painfully bad things).

      Look, I don't mean to be too sarcastic, but if "perceptual learning" means, roughly, "becomes attuned to the correct variables", then organisms must start at a state where they are behaving based on something else. I just don't understand how this is a controversy (and I especially don't understand how TSM has managed to make it a controversy for 30 years).

      Further, a huge chunk of the ethological tradition, which studied behavior by the way, is dedicated to identifying cues that animals use. A gull-chick will peck at a stick with yellow and red stripes. A goose will roll all sorts of non-eggs back into its nest. A cross moving overhead, long-side first, does not frighten chickens, but if it moves short-side first they are scared. Young ground squirrels will give snake alarm calls at twisted sticks, but adults only at actual snakes.

    5. OK, first off, try not to get too patronising. Both Sabrina and I think EJ's learning stuff is a) critical and b) massively under-appreciated by everyone involved in this. We just haven't gotten there yet. You made a claim that non-specifying variable use was rampant, I just wanted you to be specific.

      Even the ecological solution to the size-weight illusion entails people attuning to the wrong information, i.e., information specifying throw-ability, not weight.
      Actually no. The ecological solution suggests that 'weight' simply isn't something we perceive; instead it's throwability (or perhaps the more general moveability, although the specific form of the illusion matches Geoff & Arthur's throwing data waaay too well). The size-weight illusion is a side effect of the experimenter asking the wrong question.

      As for animals responding to the 'wrong' cues - they are only the 'wrong' cues from the privileged perspective of the 3rd person analysis. Animals respond to information, not what the property 'actually' is.

      So all I'm saying at this point is that if you are happy people can use a non-specifying variable, then, by TSM, you don't have direct perception. This may or may not be the case, but it is the claim under consideration. I agree about the issue of learning; TSM don't do this well and I think there's mileage there. But this is all still just laying out terms to figure out what the hell the problem actually is.

    6. Sorry, I was venting at the field in general, not you two in particular. This has been a particular frustration since I became involved in larger field a decade ago.

      The way TSM define "direct perception", it is something that should happen a lot of the time, but certainly not all of it. Given the number of animals on this planet, and the number of perceptual-motor tasks they engage in every day, it would not be an exaggeration to say that exceptions are rampant (though I do not remember using that word, I would be happy to now). This is not, in and of itself, a problem... until TSM combine their narrow definition of direct perception with the declaration that all perception is direct.

      At that point either A) they are blatantly wrong, or B) the word "direct" has been rendered superfluous, and they need to come up with an alternative word to describe what these animals are doing when not "perceiving". But instead, they compound their problem by, essentially, declaring that their are no psychological processes other than perception. Thus, they cannot take option B.

      It is all a big mess.

    7. One thing about learning that occurred to me; TSM can talk about learning as the process of differentiating the one specifying variable. So the intermediate time reflects poor thresholds for that variable, rather than use of non-specifying variables. I wonder how you could distinguish these; I guess if performance is stable but correlated to the non-specifying variable, rather than just unstable.

      Anyway, a random thought.