Tuesday, 9 November 2010

Establishing the Role of Perception in Coordination: Proprioception and Action Measures

Other coordination posts are here.

The dynamic pattern hypothesis had led to some predictions which, when tested, turned out not to be true. Instead, it seemed that the way in which we perceive relative phase (i.e. the coordination between two things) was the limiting factor. Visual judgments, in the absence of the need to produce an action, produced results that mirrored the movement stability data.

There were two immediate objections to these results, which we addressed empirically in the following way.

Objection 1: Human movement entails more than vision
The first objection to this analysis is that the judgment results came from vision. When a person is generating and coordinating actual movements themselves, there are motor commands which could be interfering with each other. Models involving neural crosstalk (Cattaert et al, 1999) or neural implementation of the oscillatory motion (Beek et al, 2002) were successfully reproducing the key movement phenomena, without any reference to perception. Movement also entails proprioception; do the (visual) perceptual results even occur in proprioception?

My first paper addressed these questions (Wilson et al, 2003). We had people track the coordinated rhythmic movement of two manipulanda, which enabled us to have people move (eyes closed) at 0º, 180º and, critically, 90º, with 4 levels of phase variability (as per Zaal et al, 2000). After testing to ensure that they were, indeed, moving at those relative phases, we analysed their judgments of the phase variability, and replicated the results from the visual perception task. Phase variability was only clearly differentiated at 0º and (at low frequencies) 180º, and not at all at 90º, which was judged to be maximally variable, even in the absence of added variability (see Figure 1).
Figure 1. Proprioceptive judgments of phase variability at 3 different relative phases and 3 frequencies (top panel), to be compared to the visual judgment data from previous work (from Wilson et al, 2003)
The results were not identical; proprioception preserves some discrimination longer than vision, for instance, and 180º is much less harder hit. This is the same qualitative pattern, however, and we proposed that the quantitative differences reflected different sensitivity of the two systems to the information.

An aside
Gav commented on a previous post about this article, on how skilled typists can detect errors both by checking the output on the screen and also when their fingers do the wrong thing., even though they were not aware that their fingers were making errors or not. He asked whether this counted as evidence against a claim I made, that you cannot control what you cannot perceive. The answer is no: These typists are clearly perceiving proprioceptively; this is clearly part of what it means to be a 'touch typist'. Just because they weren't consciously aware of the experimental manipulation regarding the errors, doesn't mean perception wasn't involved. There's more to us than vision and consciousness.

Objection 2. Well OK, but this still isn't people producing movements
The case is building, but these are still judgment studies; we need action measures (Bingham & Pagano, 1998). Perceptual judgment studies look at between-trial variability - high variability means poor discrimination of the information. Movement studies rely on within-trial variability, i.e. direct measurement of the movement's stability. We next developed an action procedure that allowed us to collect and compare within and between trial stability measures using the same dependent measure (Wilson et al, 2005).

Participants used a joystick to control a dot on a screen; their task was to track a computer controlled dot so as to produce either 0º, 90º or 180º on the screen. The unimanual task enabled us to make a critical manipulation: on some trials, we altered the mapping between the joystick and the dot it controlled, so that in order to produce 0º visually, between the two dots, the participants had to move at either 90º or 180º to the top dot. We also had people moving to produce 90º or 180º while moving at 0º.The question was, would the stable visual information stabilise the movements? Would people be able to move at 90º if they were trying to maintain 0º on the screen?
Figure 2. Movement stability (open diamonds) vs. perceptual stability (filled diamonds). '90:90' was moving at 90º while seeing 90º on the screen; '0:90' was moving at 90º while seeing 0º on the screen
The answer was yes (Figure 2). Movement stability largely followed the visual relative phase; when the goal was to move so as to produce 0º on the screen (0:90, 0:180) movement stability was elevated relative to the control, HKB conditions (90:90, 180:180). In addition, the two measures (within and between trial stability, the action and perceptual measures) were strongly positively correlated (with two outliers from the highest frequency condition, where performance was poor; this unimanual task turns out to be less stable than the bimanual version). This result matched another study by Bogaerts et al., (2003), who also found non-0º movements were stabilised by in-phase feedback. There were still consequences on stability when the movement was not at 0º, because this is a perception-action system; but movement stability is (largely) a function of perceptual stability.

The perceptual judgment results do not simply hold for vision; we replicated the key pattern with proprioceptive judgments of phase variability. This study ruled out 'neural crosstalk' explanations for lower movement stability at non-0º relative phases (Cattaert et al, 1999) and models which placed the HKB structure in the nervous system without reference to perception (Beek et al, 2002) because the participants were successfully moving at the correct relative phase but judged it in the HKB pattern. This was why we went to such extensive lengths to establish that the participants were moving at the correct mean relative phase. (To be honest, both of these models had been ruled out by the Schmidt et al (1990) data which showed the coordination phenomena persisted between people; but demonstrating that the perceptual basis underwrote results within a person helps justify the claim that these are all examples of the same task).

Then, when we manipulated the visual feedback, so that participants had a perceptually clearly discriminable target, the HKB pattern was gone from the movement stability data and this stability was elevated at 90º and 180º. We had now established a causal relation between visual perception of relative phase and the production of coordinated rhythmic movement, which tied the perceptual judgment studies to the movement literature.

Beek, P. J., Peper, C. E., & Daffertshofer, A. (2002). Modeling rhythmic interlimb coordination: Beyond the Haken–Kelso–Bunz model. Brain & Cognition, 48, 149–165. DOI

Bingham, G.P. & Pagano, C.C. (1998). The necessity of a perception/action approach to definite distance perception: Monocular distance perception to guide reaching. Journal of Experimental Psychology: Human Perception and Performance, 24 , 145-168. Download

Bogaerts, H., Buekers, M. J., Zaal, F. T. J. M., & Swinnen, S. P. (2003). When visuo-motor incongruence aids motor performance: The effect of perceiving motion structures during transformed visual feedback on bimanual coordination. Behavioral Brain Research, 138, 45–57. DOI

Cattaert, D., Semjen, A., & Summers, J. J. (1999). Simulating a neural cross-talk model for between-hand interference during bimanual circle drawing. Biological Cybernetics, 81, 343–358. DOI

Schmidt, R. C., Carello, C., & Turvey, M. T. (1990). Phase transitions and critical fluctuations in the visual coordination of rhythmic movements between people. Journal of Experimental Psychology: Human Perception and Performance, 16(2),  227-247. Download

Wilson, A., Bingham, G., & Craig, J. (2003). Proprioceptive Perception of Phase Variability. Journal of Experimental Psychology: Human Perception and Performance, 29 (6), 1179-1190 DOI: 10.1037/0096-1523.29.6.1179  (Download)

Wilson, AD., Collins, D., & Bingham, GP. (2005). Perceptual coupling in rhythmic movement coordination: stable perception leads to stable action Experimental Brain Research, 164 (4), 517-528 DOI: 10.1007/s00221-005-2272-3 (Download)


  1. These are nice studies. But, to once again play devil's advocate, could it not be argued that your manipulations placed an unfairly high weighting on perceptual information in order to complete the task? With the visual task where you introduced an offset, I'd go further and say that you enhanced the visual feedback to relative phase to such a degree that it'd be astonishing if subjects didn't use it (I definitely agree that it is easy to track in-phase movements by matching the 'turnaround' points, or whatever). This may not really be a criticism - I like the idea of enhancing perception to improve performance as evidence of perceptual information's initial explanatory value for HKB phenomena. Can perceptual information throw coordinative patterns out of whack? Can wonky visual feedback destroy your ability to coordinate your hands at 0 degrees?

  2. could it not be argued that your manipulations placed an unfairly high weighting on perceptual information in order to complete the task?
    Yes it can, and it is, regularly. The main problem is that this is a uni-directional coupling, ie it's a person coordinating with a computer signal that doesn't change in response to the person, so yes, this does push things in the (visual) perception direction. We're getting back to bimanual movements, but we had to do all this first because the unimanual task enabled all the futzing with the mapping we did. Note that the proprioception paper used bimanual movements, though.

    With the visual task where you introduced an offset, I'd go further and say that you enhanced the visual feedback to relative phase to such a degree that it'd be astonishing if subjects didn't use it
    The paper goes into this in more detail. There are two phase relations: the visual one, between the two dots, and the 'cross modal' one, between the computer controlled dot and the joystick. But the visual relation certainly dominates in this task; people can use the cross modal one, but not well.

    But this leads to this, which I should have added to the post:
    Can perceptual information throw coordinative patterns out of whack? Can wonky visual feedback destroy your ability to coordinate your hands at 0 degrees?
    Yes! In Experiment 2, we added two new conditions: 90º or 180º visual, 0º cross-modal (so 90:0 and 180:0 in our notation). The data are in Figure 9a of Wilson et al (2005), and show that movement stability follows the visual relative phase. Couple these together (the cross modal phase relation is available but not very clear, and a non-0º visual phase) and the result is still 'movement stability follows perception'.

    Wilson et al (2005b) followed all this up in detail (we established the likely reason for the 'visual dominance', among other things):

    Wilson, A. D., Collins, D. R., & Bingham, G. P. (2005b) Human movement coordination implicates relative direction as the information for relative phase. Experimental Brain Research, 165, 351-361.

    Maybe I need to come back to these papers after introducing Geoff's model.

  3. I wrote a very long post about conscious perception and the motor system, that your blog decided to delete with a system error.

    So you only get a very brief recap of my post.

    Error correction while failing to detect errors in an uncommon, consciously-mediated task (like typing or rhythmic bimanual coordination) seems rather surprising to me. Error corrections in a task like lifting objects, which occur sub-100ms after the initial errors are more like reflexes, and somewhat less surprising.

  4. I don't think consciousness mediates much, let alone performance in these types of task. I think this is evidenced by the fact that skilled typist's calibrated fingers knew what they were doing while their conscious experience had little to do with anything.