Monday 19 May 2014

Connecting the conceptual dots in embodied cognition

UPDATE 9 June 2015: We published this critique

Around about the time we published our embodied cognition paper, we also reviewed a paper for that research topic, which happened to be an example of the conceptualisation hypothesis  style of 'embodied' cognition we aren't all that impressed by.

We had serious reservations about the paper (detailed below) and did not think it should be published. After several rounds of trying and failing to get the authors to acknowledge the problem, 
we withdrew from the review and the editor, Dermot Lynott then decided to side with the other reviewers (who had identified the same problem we had but who didn't think it was a fatal flaw). The paper (Dijkstra, Eerland, Zijlmans & Post, 2012) was therefore published.

A while back I drafted a commentary laying out the problems with this paper; the full text is here. The motivation for a comment was the same as for the one I wrote for Soliman et al (2014); this style of embodiment does not tackle the hard questions about mechanism that are crucial for an embodied account and this is a big problem. I cannot decide if the commentary is worth publishing; this paper is not that important, although it is a good example of this major issue for 'grounded' embodied cognition. Any thoughts on this would be welcome.

Leaning to the left still makes you think odd things about embodied cognition

The paper is similar in spirit to Eerland et al (2011) which claimed to find that leaning to the left makes the Eiffel Tower seem smaller. The experiment was simple. People were presented with politically neutral statements and asked to attribute these statements to one of 10 Dutch political parties which varied along the left/right political spectrum. While they were doing this, people had their posture manipulated in the same way was Eerland et al (2011). They stood on a Wii board which tracked their centre of pressure and used a feedback display to keep their COP aligned to a target. The target was set slightly (2%) to the left or right of the person's upright position.

The prediction was that when people were leaning left, this would make left wing parties more accessible and people would be more likely to attribute the neutral statements to left wing parties (and vice versa for leaning right). The result was that people were slightly more likely to attribute statements to left wing parties when leaning left (p<.01) and nearly more likely to attribute them to right wing parties when leaning right (p=.07). The authors claim support for their 'embodied' cognition hypothesis. (Note that as usual, the effects were tiny and asymmetrical in direction; this latter effect is apparently very common, which direction it works in varies from study to study, and no one has any viable explanation for why it happens. Again no kinematics were reported.)

There is a different problem, however. This pattern only showed up when the political parties were coded according to their actual political affiliation. When the parties were coded as a function of what the participants thought their affiliation was, the effect went away. This is because the participants had no real idea which parties were left or right wing; they got less than half of them right and scored very low in political knowledge or interest in general.

To us, this is a disaster for the authors. Their hypothesis was that leaning left should prime left wing parties, but this of course means leaning left should prime parties people think are left wing. This flavour of embodiment research is about how states of the body affect our access to our knowledge, and if you don't know that a party is left wing, there's no mechanism by which leaning to the left will prime it, even if it really is left wing in the real world. The authors' explanation was that the task they used to measure political knowledge was too hard, but we found this fairly unconvincing and, at the very least, this problem mandated a fresh data set with the problem fixed.

All three reviewers (us, Rick Thomas and Michiel Van Elk) identified this as a problem, but we were the only ones who thought it was a fatal blow. Everyone else thought it was acceptable to just publish an inexplicable empirical result because psychologists don't like ruling things out on theoretical grounds. This is a bit of a problem in and of itself, because now this result is in the literature and must be dealt with in any rebuttal of this kind of work. 

Summary
I like the ability to submit commentaries at Frontiers; I like the thought of being able to have this critique permanently connected to that paper so that people might actually see it. I also like the fact that I can then point to these commentaries as examples of how an ecological embodied analysis can be used to identify these issues and help weed out the nonsense. Do people think this is worth submitting? The paper in question is of little importance, and it's 18 months old now so we may have missed the window of opportunity as well. But it's such an egregious offender! Feedback here would be welcome.

References

Dijkstra, K., Eerland, A., Zijlmans, J., & Post, L. (2012). How Body Balance Influences Political Party Evaluations: A Wii Balance Board Study. Frontiers in Psychology, 3 DOI: 10.3389/fpsyg.2012.00536 Download

Eerland, A., Guadalupe, T., & Zwaan, R. (2011). Leaning to the Left Makes the Eiffel Tower Seem Smaller: Posture-Modulated Estimation. Psychological Science, 22(12), 1511-1514

11 comments:

  1. I find it fascinating that the "right" "left" distinction is still used. My understanding is that it arose from the arbitrary arrangement of the French Parliament circa the revolution. The distinction was between royalists and revolutionaries. The suggestion that it has anything absolute to do with rightness vs. leftness (dexterous vs. sinister) should be viewed as absurd from the start.

    As for the commentary... not sure. Given that is written already, there is no harm in sending it out. But spend much more time on it if it is not accepted.

    ReplyDelete
    Replies
    1. Eerland et al even talk about the origins of the left/right thing. It's all about the metaphors, man.

      Delete
  2. In some studies on a categorization/sorting task with US participants, participants spontaneously sorted politicians, social groups, and newspapers into clusters on the left and right side of the screen in such a way that corresponded to the left/right metaphor.

    This was not our question in the study at all, but it's kind of interesting that people do seem to do this without much prompting.

    Regarding the study you reviewed (above), I agree - it is totally perplexing that people's knowledge of L-R distinctions didn't play a role. Political Scientists - based on cognitive psychology from the 80's or so - expect that most political effects are due to people with high levels of political knowledge because they are the most likely to adopt the frames/representations/perspective of politics.

    ReplyDelete
    Replies
    1. It's beyond perplexing; it's a hint that the significant results they did get were nothing more than good luck.

      Delete
  3. Fully agree. I would also like to read more about COP dynamics. Apparently the task was to keep the square (the position of the own COP) within the confines of a larger circle. But how much larger? If the circle was large enough, the COP could just go its own erratic way, without a need for the 2% weight shift (which is tiny). In other words, we don't even know if subjects were actually leaning left or right!
    John

    ReplyDelete
    Replies
    1. I actually have half a mind to replicate one of these studies and analyse the kinematics. Could be a good student project next year!

      Delete
  4. Wanna do a joint student project?
    John

    ReplyDelete
    Replies
    1. I honestly don't know if I care. I'm all for replications but this topic really isn't that interesting, although it could be fun to write it up the way I think it should be reported.

      I have some Wii balance boards to get working this summer; if I get them ready in time for student projects we can try it out! Drop me an email, a.d.wilson@leedsmet.ac.uk

      Delete
    2. I just remembered why I'm not sure I care about replicating these. I'd want to do motion capture work to measure what people are doing to produce these 2% shifts in postural sway. My guess is the head is not moving, it's just a shift in weight. If only weight is shifting why would the left of anything become more accessible?

      Also as I type that I am reminded that the concept is ridiculous :)

      Delete
  5. There is something not quite right with the Frontier's review process. I recently reviewed a conceptually flawed paper (remain nameless). It could not be emended without canning the entire thesis. The lead author attempted to respond (repeatedly) and I reiterated my complaints again and again. The "editor" never intervened. I actually (in print) pleaded for the editor to take some action, but he/she simply allowed a futile and annoying "discussion" to continue. I finally took the only route still apparent and withdrew. The paper now appears in Frontiers.

    While I do value many aspect of the Frontier's model, this lack of editorial intervention needs to be addressed (even if it is only sporadic inattention).

    ReplyDelete