A selection of problems with representation

As Andrew alluded to in the comments to his last post, I'm cooking up some replies to various questions that have been raised. Being relatively time poor at the moment, I'm going to take these in small bits, rather than attempt a more comprehensive reply. The first thing I want to do is consider a few of the major issues, as I see them, with the notion of representations. As a cognitive psychologist, I spent many years endorsing a representational framework, but over time I have encountered enough problems/limitations with this approach that I consider the idea of representations to be seriously flawed. As a result, I've spent the past couple of years figuring out how to do cognitive psychology without them (but that's another post). What I want to do here are go through some of the things that have convinced me to abandon representations.

1) We don't know what we're talking about when we talk about representations

This criticism goes beyond the fact that there are multiple, competing definitions of representation. It goes beyond the fact that most cognitive psychologists are never asked to seriously wrestle with their own definition of representation, to consider its historical origins, or to consider whether there are alternative approaches to cognition. This criticism is more basic and is summed up nicely by Larry Barsalou: "We have no accounts of how propositional representations arise in the cognitive system, either innately or through experience. We haven't the faintest idea of how biological mechanisms could produce abstract propositions" (Barsalou, 1993, p. 173). Regarding this quote, Linda Smith says "When we cannot imagine how our basic ideas about cognition can possibly be realized, we ought to consider the possibility that they are wrong" (Smith & Jones, 1993, p. 181).

There's More to Us Than Our Brains - So What Does The Brain Do?

I'm not that interested in the brain.

It's hard to be this way in modern psychology. Cognitive neuroscience is where it's at, and I think I come off as  a bit of a Luddite when I try to convince people fMRI is a bit of a waste of time. Not caring much about the brain is certainly a sociological reason why ecological psychology doesn't get taken very seriously; we're just the crazy people who don't think there are mental representations, based on some work from the 50s-70s. Surely modern imaging has shown us the activity of mental representations? Clearly, the brain is the source of all behavior! Popular science writing on psychology is all cognitive and representational; most of the psychology blogging I come across is neuroscientific. What else could it be?

I've certainly spent a lot of time waving the flag against the infiltration of neuro-talk into places it doesn't yet belong; but to be honest, as I get older, I've begun to worry that I'm trying to be 'fair and balanced' in the sense Fox News is fair and balanced: relentlessly playing up one side to offset a perceived imbalance elsewhere. What I actually want to do is be actually fair and balanced: I want my own discussions about these issues to be internally balanced and coherent, giving credit where credit is actually due. I want to start teasing apart a few issues I've conflated over the years, so that my strong concerns about the relevance of fMRI  and cognitive neuroscience work stop getting swallowed up in a general dismissal of the brain's role in our lives. The brain is clearly interesting, but it's not representing, and if not that, what is it doing?

This post is therefore a first swing at integrating a lot of the things I've been blogging about for a while and doing so in a way that leaves a sensible role for the brain. I'm going to need some neuroscientists to talk to, though; I'd appreciate it if people could spread the word on this a little, because there are just some things I want to go a few rounds on with people who know what they're talking about. 

Lissajous feedback and coordination stability

Understanding the perceptual information you provide people in a task is a critical element of the perception-action analysis. Last time I talked about the new form of coordination feedback I developed to allow us to train coordinated rhythmic movements without perturbing the task dynamic. Prior to this, the most common form of augmented feedback was the Lissajous plot - these are the result of plotting the displacements of two harmonic oscillators against one another, and the unique shape associated with each relative phase can be used as a template on the screen. People can then try to move so as to make a dot trace that shape.

Lissajous plots (have a play with them in this Excel file) are transformed feedback, because they take a coordinated movement and represent it on the screen as the motion of a single dot. This type of feedback has been used extensively to train people to perform novel coordinations, but until recently no-one had thought to investigate the consequences of transforming the information about relative phase. Kovacs, Buchanan and Shea have recently begun doing exactly this, and, in line with the perception-action approach developed by Bingham and pushed at every opportunity by myself, these authors have found that Lissajous plots completely alter the nature of the task, with serious consequences for the studies that rely on it.

Visual feedback for training novel coordinations

The key feature of coordinated rhythmic movements is that not all coordinations are stable. Most other rhythms can be learned, however, which is why we can have jazz drumming. People have been training participants to perform novel coordinations (especially 90°, the least stable rhythm without training) for years now, and have been asking all the standard learning questions - how long does learning take? Does it transfer to other coordinations? 

The first real studies on learning were by Kelso and Zanone (Kelso & Zanone, 2002; Zanone & Kelso, 1992a, b, 1997). I briefly reviewed the results of these studies here, which have lead to to the dynamic pattern hypothesis. This account describes stable states as attractors in a state space defined by relative phase as the order parameter, and learning is the creation of a new attractor centred on the target novel phase. This account ran into problems quite quickly but is still alive and kicking in a modified form; stability is the governing principle now, and from this perspective the feedback displays used for training don't matter so long as they support stable action. 

However, from our perception-action standpoint, the feedback displays matter a lot, because these are what's providing the perceptual information about the coordinated movement. Early learning studies all used some kind of transformed feedback, which we could never use because it altered the overall perception-action dynamic. In order to look at action learning directly, we needed a new form of feedback. 

So I invented one.

Rates of learning and the dynamic pattern approach

One of the interesting features of coordinated rhythmic movement is that people start out with a particular pattern to their performance - there is pre-existing structure to our attempts to coordinate these movements. This structure affects our ability to learn new coordinations, and the pattern of the effects reveals a lot about the cause of this pre-existing structure. 

However, the literature is split into two incompatible accounts of learning, and trying to fix this is part of my ongoing interest in this task. The first account is the dynamic pattern approach, which was pioneered by JAS Kelso, and championed by modelling (Gregor Schöner) and behavioural studies (Pier Zanone). I'm more interested in the latter aspect, because it's the motivation for the former. I've already reviewed how this account fails, but it's still alive and well thanks to some creative history, and needs to be tackled again. The second account, which I prefer, is the perception-action account (Bingham) which developed from empirical work on visual and proprioceptive perception as well as action measures, and embodied in a model.

We haven't explicitly tackled the rate of learning issue, although we will and there is already support for our account in the literature (Wenderoth et al, 2002). But it comes up regularly in the dynamic pattern behavioural work, so it's time to work out what's going on in their data.