Thursday 18 March 2010

Internal representation or behavioural dynamics?

More on Gibson later, but I wanted to get to this today. Yesterday I saw a talk about eye tracking, and how people control smooth pursuit movements (the tracking movements your eyes can make when you’re following something continuously). Tracking performance is a quandary for cognitive folks, because we are often very good at it. For instance, if you ask people to track a moving stimulus and record their eye movements, they will successfully foveate the target with almost no lag or erratic need to play catch up (foveating means using the fovea, the densely packed high resolution region on the retina we rely on for precise visual perception). The lack of ‘catch-up’ is the interesting bit, and cognitive psychology thinks that it is evidence of prediction by the system. Prediction requires a predictor, which for cognitive psychology is always a representation.

The main thing I learned from this talk is this: I am a rampant ego-maniac who is convinced I am right and other people are wrong, but at least I am capable of entertaining the idea that there is another way to conceive of the task. This speaker (and at least one other person in the room) was completely unaware that their perspective entailed assumptions about the underlying mechanism and simply couldn’t conceive of another way to describe the task: for them, prediction was clearly required and therefore internal representation was clearly required.

The argument is this: your visual system has detected the position of the dot at time t. You must then process this information (i.e. enrich the impoverished stimulus to resolve the various ambiguities of vision). You must then route this processed visual information to motor cortex so it can be used to control a response. This processing takes time – the standard guess (based on various reaction time studies, physiological limits in the speed of nerve conduction etc) is 200ms. Your response has therefore been generated by information that is 200ms out of date; if you used this old information, your eyes would be looking at where the stimulus was 200ms ago (i.e. you would be lagging behind). People don’t do this (quite crucially); therefore, the argument goes, you must use the information you have and transform it by running it through a representation that implements a predictive mechanism.

So what is another way to formulate this problem? The non-representational response is known as behavioural dynamics, which has been developing through the work of many people over the years but was recently summed up in a review article by Bill Warren. The closest example to the eye tracking-prediction question tackled so far is the outfielder problem: how does a baseball outfielder know where to go in order to catch a fly ball? Mathematically the problem entails calculus, and the cognitive theory (Saxburg) was that an internal representation implemented the required calculus transformations on the outfielder’s vision to predict where the ball will fall. This is entirely incorrect, and two viable control strategies (the Linear Optical Trajectory; e.g. MacBeath et al, 1995, and the Optical Acceleration Cancellation; e.g. Fink et al, 2009) have been identified; if a fielder moves so as to a) make the ball move (optically) in a straight line or b) make the ball move (optically) at constant velocity, the net result is that the outfielder will find themselves in the right place at the right time. Regardless of how this active research question is resolved, the lesson is simple: the assumption of poverty of stimulus and mental representation leads to one very specific way of thinking about a problem. The assumptions of behavioural dynamics leads to an entirely different characterisation of the task, and, in many of the problems tackled so far, dynamics trumps representation.

However, even Warren acknowledges that ‘representation-hungry’ problems (Andy Clark’s phrase) like prediction don’t yet have any clear solutions from this perspective. Behavioural dynamics might not (yet) have an answer to the problem, but it is a clear, alternative formulation of the problem-to-be-solved by the organism. Cognitive psychologists like the eye tracking researcher have never even encountered this approach as an option: they are literally blinded by the rules of the representational paradigm to the existence of non-representational explanations. We have the advantage of being the outsiders, the out-group; we have been explicitly defined at all times by our opposition to those assumptions, and so we remember their assumptions as clearly as we remember our own.

Fink, P.W., Foo, P.S., & Warren, W.H. (2009) Catching fly balls in virtual reality: A critical test of the outfielder problem. Journal of Vision, 9(13), 14:1-8. Download

McBeath, M. K., Shaffer, D. M., & Kaiser, M. K. (1995). How baseball outfielders determine where to run to catch fly balls. Science, 268, 569–573. Download

Warren, W. H. (2006). The dynamics of perception and action. Psychological Review, 113(2), 358-389. DOI


  1. You guys should buy matching leather jackets: "The Legendary Gibsonians" or "The Representationalists Rippers"

    I agree that having a close point of view is a very retrograde mind set, for any scientist.
    My supervisor loves Predictive Coding, and he is convinced that's how everything works.

  2. Its not just that the mind set was closed; its the complete lack of awareness that a choice had been made. We all have our theoretical blinders, but this person didn't even know she had these.

  3. We find an interesting effect in cognitive psychology, like prediction of an objects location, and we invoke mental representations to explain it.

    From your article it is not clear how behavioural dynamics adds anything, let alone trumps this assumption. Perhaps you could explain it more clearly.

    If the mind is not computation, what is it? This is clearly a working assumption, but it's one that has seemed satisfactory for me. If some data comes through that challenges this then i'd have a re-think (what kind of data could undermine this view?).

    1. If the mind is not computation, what is it?
      This is the most common question, and the answer is a non-computing nonlinear dynamical system, such as the Watts steam governor.

      Dynamics trumps representation if it can explain behaviour as well as or better than representations. The first step is a rigourous task analysis, where you identify the various resources you have available to solve a task; if the perceptual and behavioural resources are enough, then you don't need to invoke any other hypothetical construct such as a representation. This works to the extent that you can identify resources that are up to the task, eg the outfielder problem where you replace prediction with online control via current perceptual information that specifies a future state.

  4. It seems obvious to me that coordination such as that which is typical during sport, is not representational. But at the same time, can't there be representations in the brain? For example, would it make sense to postulate a top down representational system and a bottom up non-representational system? A sportsperson might perform best relying mostly on the latter, which would be more implicit/automatic and with minimal conscious interference, perhaps with only conscious observation rather than control. I think anyone with experience in sport can resonate with the idea that an overly conscious approach can introduce error, whereas motivation and effective engagement results in a flow which seems to disengage conscious control.