Connecting the conceptual dots in embodied cognition

UPDATE 9 June 2015: We published this critique. 

Around about the time we published our embodied cognition paper, we also reviewed a paper for that research topic, which happened to be an example of the conceptualisation hypothesis  style of 'embodied' cognition we aren't all that impressed by.

We had serious reservations about the paper (detailed below) and did not think it should be published. After several rounds of trying and failing to get the authors to acknowledge the problem, we withdrew from the review and the editor, Dermot Lynott then decided to side with the other reviewers (who had identified the same problem we had but who didn't think it was a fatal flaw). The paper (Dijkstra, Eerland, Zijlmans & Post, 2012) was therefore published.

A while back I drafted a commentary laying out the problems with this paper; the full text is here. The motivation for a comment was the same as for the one I wrote for Soliman et al (2014); this style of embodiment does not tackle the hard questions about mechanism that are crucial for an embodied account and this is a big problem. I cannot decide if the commentary is worth publishing; this paper is not that important, although it is a good example of this major issue for 'grounded' embodied cognition. Any thoughts on this would be welcome.

The Small Effect Size Effect - Why Do We Put Up With Small Effects?

Small effects sometimes matter - but psychology can do better

One of the things that bugs me about 'embodied' cognition research is that the effects, while statistically significant, tend to be small. What this means is that the groups were indeed different in the direction the authors claim, but only slightly, and that the authors had enough people showing the effect to make it come out on average. 

The problem with small effect sizes is that they mean all you've done is nudge the system. The embodied nervous system is exquisitely sensitive to variations in the flow of information it is interacting with, and it's not clear to me that merely nudging such a system is all that great an achievement. What's really impressive is when you properly break it - If you can alter the information in a task and simply make it so that the task becomes impossible for an organism, then you have found something that the system considers really important. The reverse is also true, of course - if you find the right way to present the information the system needs, then performance should become trivially easy. 

Psychology has become enthralled by statistical significance (to the point that we're possibly gaming the system in order to cross this magical marker). If your effect comes with a p value of less than .05, it is interesting, regardless of how small the effect is in terms of function. This is a problem, and we don't have to put up with it. If you ask a question about the right thing, you should get an unambiguous answer. If your answer is ambiguous, you may not be asking about the right thing. 

I want to remind readers of a couple of examples of nuisance small effects I've covered here before, then talk a little about some work which either broke or fixed the right thing, to highlight that we don't actually have to suffer from the tyranny of the small effect effect.