Visual feedback for training novel coordinations

The key feature of coordinated rhythmic movements is that not all coordinations are stable. Most other rhythms can be learned, however, which is why we can have jazz drumming. People have been training participants to perform novel coordinations (especially 90°, the least stable rhythm without training) for years now, and have been asking all the standard learning questions - how long does learning take? Does it transfer to other coordinations? 

The first real studies on learning were by Kelso and Zanone (Kelso & Zanone, 2002; Zanone & Kelso, 1992a, b, 1997). I briefly reviewed the results of these studies here, which have lead to to the dynamic pattern hypothesis. This account describes stable states as attractors in a state space defined by relative phase as the order parameter, and learning is the creation of a new attractor centred on the target novel phase. This account ran into problems quite quickly but is still alive and kicking in a modified form; stability is the governing principle now, and from this perspective the feedback displays used for training don't matter so long as they support stable action. 

However, from our perception-action standpoint, the feedback displays matter a lot, because these are what's providing the perceptual information about the coordinated movement. Early learning studies all used some kind of transformed feedback, which we could never use because it altered the overall perception-action dynamic. In order to look at action learning directly, we needed a new form of feedback. 

So I invented one.

Forms of Transformed Visual Feedback

1. Visual Metronomes

Figure 1. Visual metronome

The first form of feedback was the visual metronome, used by the Kelso & Zanone studies, and is still used by Zanone. All it is is a display of two dots who alternate their position at some frequency and some relative phase to each other. Figure 1 shows two dots alternating at 180°. Participants are instructed to (for example) flex and extend their index fingers to match the dots; the left dot for the left finger, etc. The net result is that they move at the required relative phase and frequency.

This effectively reduces the coordination task to two tracking tasks; people no longer have to actively produce a coordination, they simply have to track two signals and they end up producing a coordination. We used this trick in a previous paper (Wilson, Bingham & Craig, 2003) to have people produce coordinations they normally couldn't so they could make proprioceptive judgements about the movements. Coordination is not the same as two tracking tasks, however, and the key phenomena emerge from the coordination requirement. The upshot is that with this feedback, people aren't coordinating anything and thus it's no use to us. In addition, there is no motion in these displays and thus relative direction is not defined. 

2. Lissajous Figures

Figure 2. Lissajous plot

A second form of transformed feedback is the Lissajous figure. These are the result of plotting the displacement of two harmonic oscillators against one another. The resulting plot is specific to the relative phase; for movements at the same frequency, 0° produces a line of slope 1, intercept 0, 180° is a line with slope -1, and 90° is a circle. Other relative phases are ellipses of varying eccentricity (this figure on Wikipedia lays out the space, and this Excel file I made let's you play with the parameters to generate the resulting shape). You can also specify more complex frequency relations with more complex plots. 

You can display a template shape on a screen and ask people to move so as to make a single dot follow the shape. But this is also transformed feedback; there is motion but only one dot (so nothing to be coordinated and no relative direction), and again the task is reduced to tracing a shape. Work from Charles Shea's lab has explored this in detail and shown that Lissajous plots effectively make all coordinations equally easy and don't actually readily promote much learning; I'll review these in more detail next time. 

Coordination feedback

We needed a form of feedback that didn't perturb the task dynamic we're interested in studying, i.e. the one described by the model. I therefore used the normal display of two dots, and simply changed the colour to green when the person was moving at the target relative phase, +/- some error range. Colour has precisely no effect on coordination stability (Mechsner & Knoblich, 2004) and is also not part of the task dynamic described by the model. It can therefore serve as a task-neutral 'hot-cold' signal to participants, who must actively produce a coordinated movement to get the feedback. 

We therefore described and tested this feedback method, to make sure it works (Wilson et al, 2010; download). There are two ways to implement the feedback (described in the Appendix) but the net result is the same: the dots change colour to signal when you are within a given range of the target relative phase. We trained participants to move at 90° by starting the feedback error bandwidth quite wide and then making it harder to get the feedback over time by narrowing the window. We also gave a control group exactly the same amount of practice with these displays, but no feedback. The results are in Figure 3.

Figure 3. Movement stability data for the learning and control groups.

The result was unambiguous - participants improved their coordination stability at 90° to levels equivalent to 180°, but only when trained with the feedback. The control group failed to improve at all (due to the fact that prior to training, 90° is simply not clearly perceived). The feedback cued people when they were moving correctly, and in the Post-Training session (with the feedback turned off) they showed that this enabled them to learn how to actively produce a novel coordination. Note also how much more robust this improvement is than the gains with perceptual training - this is a perception-action task and while perception is a critical part, the movement itself matters too.

Summary

This method completes our methodological toolbox. The perception-action approach meant we had to invent numerous new methods to study coordination, including visual judgements, proprioceptive judgements, perceptual training, perturbation displays, and finally action training. This feedback method was only motivated by the perception-action approach; no one else thought the feedback mattered, despite increasing evidence to the contrary. We are now explicitly comparing performance with this feedback and the transformed versions, looking to see what, if anything, people learn about coordination using the other methods.

References

Kelso, J. A. S., & Zanone, P. G. (2002). Coordination dynamics of learning and transfer across different effector systems. Journal of Experimental Psychology: Human Perception and Performance, 28(4), 776-797.

Mechsner, F., & Knoblich, G. (2004). Do muscles matter for coordinated action? Journal of Experimental Psychology: Human Perception and Performance, 30(3), 490–503.

Wilson, A. D., Bingham, G. P. & Craig, J. C. (2003). Proprioceptive perception of phase variability. Journal of Experimental Psychology: Human Perception and Performance, 29(6), 1179-1190. Download 

Wilson, A., Snapp-Childs, W., Coats, R., & Bingham, G. (2010). Learning a coordinated rhythmic movement with task-appropriate coordination feedback Experimental Brain Research, 205 (4), 513-520 DOI: 10.1007/s00221-010-2388-y  Download

Zanone, P. G., & Kelso, J. A. S. (1992a). Evolution of behavioral attractors with learning: Nonequilibrium phase transitions. Journal of Experimental Psychology: Human Perception and Performance, 18(2), 403-421.
 

Zanone, P. G., & Kelso, J. A. S. (1992b). Learning and transfer as dynamical paradigms for behavioral change. In Tutorials in Motor Behavior II, Advances in Psychology (Stelmach, G., & Requin, J., Eds.). Amsterdam: North-Holland.

Zanone, P. G., & Kelso, J. A. S. (1997). Coordination dynamics of learning and transfer: Collective and component levels. Journal of Experimental Psychology: Human Perception and Performance, 23(5), 1454-1480.