"Moving Through Time" and embodied cognition

In which I am a bit rude about a rubbish paper and worry about how to kill papers like it.

The term 'embodied cognition' has been checked out of the library by a lot of different people, all of whom use it to mean something different. My least favourite definition comes from the cognitive literature, which considers embodiment to be about how internal, abstract cognitive function can be 'revealed' motorically (the Barselou (2008) version of embodiment). This definition is rooted firmly in the assumption of mental representation. For me, embodiment is only interesting if it contributes in a meaningful way to what cognition is, and thus I define embodied cognition in the other direction - embodied cognition is about how cognition is shaped by the kind of perceiving-acting organism we are.

A recent terrible example of the cognitive version is an article called 'Moving Through Time', which appeared in Psychological Science at the beginning of this year. I've bitched briefly about this before but it's on my mind again after reading the reviews for a colleague's attempt to kill this off.

The results are simple: the authors had participants either think about the past or the future, and they measured postural sway. On average, the postural sway trended forwards over the course of the trial in the 'future' condition, and trended backwards  in the 'past' condition. The authors concluded that the common metaphors we employ in language about mental time travel are grounded in this sensori-motor behaviour.

This got a fair amount of press at the time, which is a shame because it's a terrible paper. First, because this is science, let's look at the methods and results section:

Method: Postural sway was measured by placing one marker on the knee; this marker recorded a position time series.

Problem: The direction the knee moves is not a good indicator of postural sway. In fact, for larger movements, the movement of the knee is actually in the opposite direction to the head, and a primary function of the postural system is to maintain a stable platform for vision. So the head was, in all likelihood, moving in the 'wrong' direction for this account, although we don't know because this obvious data was not recorded.

Results: The data series was binned over time to look for a trend in the average position over time. Sway is a constant; what they were asking was whether this sway was tending to go in one direction or another over time. Refer to Figure 1. 

Figure 1 from Miles et al (2010).

The data clearly show two statistically reliable trends; in the 'past' condition people's sway headed backwards, and in the 'future' condition it headed forwards.

Problem 1: this is a textbook example of the difference between statistical significance and actual significance. While there was enough of a trend to cross the threshold for the test, the actual magnitude of the effect was 2mm in either direction. 

Problem 2: by measuring sway as a position value from the knee, they have a problem: the angle that this 2mm effect corresponds to depends on how high off the ground the knee is. A sway measured in mm doesn't really make sense, and given how small the effect is, this confound is likely to be a worry for them. 

Conceptually, of course, this paper is entirely flawed. The postural system has a very important job to do: it has to keep you basically upright (i.e. detect any sway that falls outside the safe range), and it also has to provide a stable viewing platform for vision to work optimally. In short, it doesn't have time to have your mental time travel getting in the mix, because this bears no necessary connection to the realities of your current postural condition. But, you cry, the effect is tiny but reliable - therefore it poses no threat to posture but is real! This might be true is the methods had been more appropriate, but the various confounds raise too many questions (always read the Methods section, dear reader). 

Why this effect may never die and why this matters

The reason this is on my mind is that some colleagues have tried to demonstrate the limitations of this study and are having real problems getting past the reviewers who think this effect is real. One problem is that they haven't quite run the right study. But the other problem is that they cannot replicate the basic effect (because it is, of course, rubbish). So the thrust of their paper is a failure to replicate, and worse, it's therefore a null result. In statistics you can't interpret a null result because there is more than one reason for you to have failed (there may be nothing to detect, you may have insufficient power, you may have run the experiment incorrectly, etc). In order to interpret a null result, you have to embed it in a positive result that identifies the reason why you got the null result (e.g. a control condition). 

This isn't impossible in this case, but it seems very very hard. The basic critique is not really that the result comes from the flawed methods or analysis; it's that the result is so meaningless it is probably a Type I error. This makes replicating the effect under any circumstances highly unlikely, and thus hard to run a study that shows why the null result in the replication came about. 

In the meantime, this paper gets blogged about, gets popular press, cited by people who don't understand embodied cognition and it never dies. Psychology is riddled with results like these and it's a real problem in the field caused by the lack of any over-arching theory; physics doesn't still have to address any of the experiments that 'proved' the aether, but psychology will always have to deal with studies that claim to show something equally incoherent. 

Why It Annoys Me

At it's heart, this perspective on embodiment assumes that perception and action are transparent filters through which one can observe the activity of an internal representation doing abstract, cognitive work. This idea is really apparent in the cognitive developmental literature, and you should read Thelen & Smith's excellent book on why it's rubbish. But the fact is that this work showed up in a top-ranked psychology journal without paying even lip service to the necessary methodological controls from the movement science literature, and now it's more noise the field has to contend with.

UPDATE (10 May 2016) John Stins and colleagues have published a failed replication of this paper.

References

Barsalou, L.W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645. Download

Miles, L., Nind, L., Macrae, C. (2010). Moving Through Time Psychological Science, 21 (2), 222-223 DOI: 10.1177/0956797609359333

Thelen, E., & Smith, L. B. (1994). A Dynamic Systems Approach to the Development of Cognition and Action. Cambridge, MA: MIT Press.