Connecting the conceptual dots in embodied cognition

UPDATE 9 June 2015: We published this critique. 

Around about the time we published our embodied cognition paper, we also reviewed a paper for that research topic, which happened to be an example of the conceptualisation hypothesis  style of 'embodied' cognition we aren't all that impressed by.

We had serious reservations about the paper (detailed below) and did not think it should be published. After several rounds of trying and failing to get the authors to acknowledge the problem, we withdrew from the review and the editor, Dermot Lynott then decided to side with the other reviewers (who had identified the same problem we had but who didn't think it was a fatal flaw). The paper (Dijkstra, Eerland, Zijlmans & Post, 2012) was therefore published.

A while back I drafted a commentary laying out the problems with this paper; the full text is here. The motivation for a comment was the same as for the one I wrote for Soliman et al (2014); this style of embodiment does not tackle the hard questions about mechanism that are crucial for an embodied account and this is a big problem. I cannot decide if the commentary is worth publishing; this paper is not that important, although it is a good example of this major issue for 'grounded' embodied cognition. Any thoughts on this would be welcome.

Leaning to the left still makes you think odd things about embodied cognition
The paper is similar in spirit to Eerland et al (2011) which claimed to find that leaning to the left makes the Eiffel Tower seem smaller. The experiment was simple. People were presented with politically neutral statements and asked to attribute these statements to one of 10 Dutch political parties which varied along the left/right political spectrum. While they were doing this, people had their posture manipulated in the same way was Eerland et al (2011). They stood on a Wii board which tracked their centre of pressure and used a feedback display to keep their COP aligned to a target. The target was set slightly (2%) to the left or right of the person's upright position.

The prediction was that when people were leaning left, this would make left wing parties more accessible and people would be more likely to attribute the neutral statements to left wing parties (and vice versa for leaning right). The result was that people were slightly more likely to attribute statements to left wing parties when leaning left (p<.01) and nearly more likely to attribute them to right wing parties when leaning right (p=.07). The authors claim support for their 'embodied' cognition hypothesis. (Note that as usual, the effects were tiny and asymmetrical in direction; this latter effect is apparently very common, which direction it works in varies from study to study, and no one has any viable explanation for why it happens. Again no kinematics were reported.)

There is a different problem, however. This pattern only showed up when the political parties were coded according to their actual political affiliation. When the parties were coded as a function of what the participants thought their affiliation was, the effect went away. This is because the participants had no real idea which parties were left or right wing; they got less than half of them right and scored very low in political knowledge or interest in general.

To us, this is a disaster for the authors. Their hypothesis was that leaning left should prime left wing parties, but this of course means leaning left should prime parties people think are left wing. This flavour of embodiment research is about how states of the body affect our access to our knowledge, and if you don't know that a party is left wing, there's no mechanism by which leaning to the left will prime it, even if it really is left wing in the real world. The authors' explanation was that the task they used to measure political knowledge was too hard, but we found this fairly unconvincing and, at the very least, this problem mandated a fresh data set with the problem fixed.

All three reviewers (us, Rick Thomas and Michiel Van Elk) identified this as a problem, but we were the only ones who thought it was a fatal blow. Everyone else thought it was acceptable to just publish an inexplicable empirical result because psychologists don't like ruling things out on theoretical grounds. This is a bit of a problem in and of itself, because now this result is in the literature and must be dealt with in any rebuttal of this kind of work. 

Summary
I like the ability to submit commentaries at Frontiers; I like the thought of being able to have this critique permanently connected to that paper so that people might actually see it. I also like the fact that I can then point to these commentaries as examples of how an ecological embodied analysis can be used to identify these issues and help weed out the nonsense. Do people think this is worth submitting? The paper in question is of little importance, and it's 18 months old now so we may have missed the window of opportunity as well. But it's such an egregious offender! Feedback here would be welcome.

References
Dijkstra, K., Eerland, A., Zijlmans, J., & Post, L. (2012). How Body Balance Influences Political Party Evaluations: A Wii Balance Board Study. Frontiers in Psychology, 3 DOI: 10.3389/fpsyg.2012.00536 Download

Eerland, A., Guadalupe, T., & Zwaan, R. (2011). Leaning to the Left Makes the Eiffel Tower Seem Smaller: Posture-Modulated Estimation. Psychological Science, 22(12), 1511-1514