On why fMRI is bullshit

If you want to really confuse a psychologist, tell them you don't think there are mental representations mediating behaviour. Try it - they will simply assume you must be joking, because it has never occurred to them that it might be true. This, unfortunately, is the single biggest stumbling block in talking about Gibson to cognitive psychologists, because one of the radical ideas in ecological psychology is that there isn't any need to invoke representations.

The most common counter argument comes from modern neuroscience (specifically, neuro-imaging, and in particular functional magnetic resonance imaging (fMRI)). There must be representations, your cognitive friend will cry - we've seen them via fMRI! If I present a stimulus to a person in a magnet, you can literally see the brain light up. Clearly, the triumphant cognitive type will claim, Gibson is wrong when he says there are no representations because the brain is obviously up to something.

Well, yes - of course it is. The brain is a complex and expensive piece of biological equipment, so I certainly hope it's up to something. But this argument (and yes, I really have had this argument) simply rests, like all cognitive psychology, on the assumption of mental representation. The brain is up to something; mental representations are required; therefore the brain is representing.  Ecological psychology is therefore often accused of claiming that the brain isn't interesting, but this error comes from conflating 'brain activity' with 'mental representation'. Ecological psychology actually has yet to say anything much about the brain, but this is mostly because we are still figuring out what it might possibly need to be doing.

The fascination with the brain is almost entirely down to the development of fMRI techniques. Brain imaging with PET and EEG lacked the kind of spatial resolution required to start talking about which bit of the brain might be responsible for doing what kind of processing, but as soon as fMRI hit the scene cognitive neuroscience became the place to be. fMRI is convincing to people: look, they say, this bit of the brain really does become active when I present my stimuli, so it must be doing the things I think are required to solve my task. Interestingly, however, people will say this without ever actually thinking about what it is they just said and they will forget to question whether they understand their task at all.

Ecological psychology, when done carefully, is about solving this problem. Our research programme is to figure out the actual job description for the brain - remember, the assumption of mental representation is only justified under an assumption of poverty of stimulus. If there is no such poverty, then the job the brain needs to do becomes radically different (e.g. it no longer needs to enrich perception with prior knowledge). This is the reason why modern neuroscience isn't a slam dunk refutation of Gibson: none of their research correctly characterises the information in a task and so therefore they cannot possibly correctly interpret the brain's response.

This is a tremendous problem for neuroscience. They have managed to successfully present some kind of information in a format the nervous system is capable of responding to (light via the eyes, sound via the ears, etc). Given that the nervous system can detect it, it does respond; but if you haven't correctly described what went in, you cannot possibly interpret what came out.

Am I being picky? Is this just a problem for bad fMRI research, and people who really know what they're doing really do control their stimuli? I wish that were true, but it isn't. Neuroscience is currently premised on the idea that all the interesting work of our mental life is done inside the head; remember, visual perception begins at the eye for these researchers, and not out in the world in the interactions of light with surfaces as Gibson realised. Neuroscience is trying to explain everything about perception (it's contents, it's relation to things in the world) in terms of brain activity, and have simply failed to notice that a lot of the required work is actually not done anywhere near the brain at all. The optic array's structure is revealed by movement, for example; by eliminating movement (necessary for fMRI imaging) you've simply removed a large part of the information generally present when solving the task in question. The consequence is not simply 'controlling for movement', the consequence is 'fundamentally altering the task space'. This raises profound questions about the validity of fMRI research, i.e. it is not at all clear that researchers are even asking the question thinking they're asking.

This fundamental flaw in neuroscience is nicely illustrated by a story I tell a lot. When I was nearly finished at IU, the department was about the purchase an fMRI magnet and hire an imaging specialist. One of the job applicants was a high profile, in-demand researcher just completing a prestigious post-doc at NIH. This person was being flown all over the country by departments desperate to hire him because of how awesome everyone thought he was.

I went to his job talk, which was on some work he had been doing on the neural systems underpinning the perception of biological vs. non-biological motion. This is an interesting question: biological motion perception is a fascinating research topic that investigates the kinds of information we can gain simply from the way biological agents move (and it's a lot: age, gender, emotional state, etc etc; when done properly it is brilliant stuff). The important word in the phrase 'biological motion' is 'motion'; all the information is in the way things move in relation to one another, hence a point-light display can provide all the information with no confounding information.

Anyway, this job talk got off to a rocky start when he introduced his stimuli. His biological motion displays consisted of videos of someone dancing. Biological motion? Sure, but unnecessarily under-controlled, especially given that point-light displays are so readily available and solve all the key problems.

Then it got worse. His non-biological motion displays were videos of someone's hand holding a hammer and hitting a nail, with the hand digitally edited out. No hand = no agent = non-biological, for this person; this is, of course, entirely incorrect, because the video preserved the motion information. My PhD advisor and I looked at each other as if to say 'Do you want to tell him?', then he pointed out to the speaker that both of those displays were, in fact, poorly controlled biological motion displays; had he not thought to use point-light displays? The fMRI wunderkind had not, because he didn't know what those were.]

Let me restate this: a researcher studying biological motion perception had never heard of the single most important methodological tool for studying biological motion, and yet everyone thought he was fabulous because he had access to a scanner. [Edit: I misremembered this bit and it's not true; see the comment here.]

The worst part of this talk? He got differences in brain activity for the two stimuli type and proceeded to claim that the difference reflected how the brain processed biological and non-biological motion in different parts of the brain. And no one in the room besides my advisor and I thought this was wrong, even though the key flaw in his study had already been pointed out; he got differences, so the stimuli were clearly different, and the difference had been called biological vs. non-biological, so that is what the difference must be. The brain had spoken!

I tell this story to illustrate a few key points. First, fMRI research in general has no theory of information and therefore cannot be trusted to correctly interpret their results. Second, a room full of smart people believed the imaging data even when a fatal flaw in the method had been identified. The objection was ignored mostly because it came from the Gibsonian guy, and we all know those guys are crazy people who don't think the brain does anything! Third, this last fact tends to lead to fMRI research being sloppy with its stimuli in a way visual psychophysics research pre-imaging would never have found acceptable; pretty pictures have blinded psychologists to important questions of methodology and validity in a way that is simply shocking.

fMRI came along at precisely the wrong time for psychology. The dominant cognitive approach assumes that all the work of mental life is done inside the head by representations; imaging reveals that the brain is, indeed, up to something; and in a depressing example of the confirmation bias cognitive psychology triumphantly announces that fMRI has shown us the truth of their claims. Ecological psychology, dynamical systems and the various flavours of embodied cognition were only just starting to come together and demonstrate empirical successes, and we now have an uphill battle ahead of us.