Brains learn to perceive, not predict, the world

The current trendy way to think about brains is as predictive machines. Bayesian methods, free energy, generative models, and all the other things in Andy Clark's new book are where it's at. Brains have to predict because, in Clark's terms, we are 'surfing uncertainty' - one step from disaster as we control our behaviour using sensory information we had to spend time processing and that is therefore a few hundred milliseconds out of date. Our brains build models of the world that constantly generate active predictions about what sensation will tell us next, and our behaviour holds together to the extent that the models do.

The ecological approach provides a different job description for the brain (Charles, Golonka & Wilson, 2014). We are embedded in environments rich in information that specifies a wide variety of behaviourally relevant properties. Some of that information is prospective, that is, it is information available now that is about something in the future. Two examples are the information variables you can use to catch a fly ball; both are states of the current time that, when used, allow you to achieve a behaviour in the future (specifically, to be in the right place in the right time to catch that ball). Another example is tau and the various other variables for time-to-collision.

This post reviews a paper (van der Meer, Svantesson & van der Weel, 2012) that measured visual evoked potentials in infants longitudinally at two ages, using stimuli that 'loomed' (i.e. looked like they were going to collide with the infants). The data show that the infant brains were not learning to predict the world. Instead, neural activity became more tightly coupled to information about the time-to-collision. We learn to perceive, not predict, the world.

Thoughts on Ding et al (2015) "Cortical tracking of hierarchical linguistic structures in connected speech"

I happened to be reading Cummins (2000) paper “’How does it work?’ vs. ‘What are the laws?’ Two conceptions of psychological explanation”, when my Twitter feed announced that Chomsky was right and we do have grammar in our heads after all. The Twitter buzz concerned a new Nature Neuroscience paper by Ding and colleagues called “Cortical tracking of hierarchical linguistic structures in connected speech.” You can find it online here. Curious whether I needed to completely overhaul my understanding of language, I tracked down the paper and read it this morning. The method employed is sensible, the results are fairly clear, the analyses seem legit (though I’m not a neuroscientist). So, why am I not worried that everything I thought I knew about language is wrong?

Function vs Structure (A Purple Peril)

One of the apparently controversial things that I say is that psychology, as a science, needs to address function before it gets worried about structure; what is the brain trying to do, vs how is it doing it? This Peril lays out the argument in a little more detail. As always, this is my current thinking not my final thinking and I am happy as ever to hear arguments for and against this proposal.

Structure (the details of how a function is implemented) is important, there is no doubt. But I see two related arguments about putting function first, or at least giving it the driver's seat in our science.

It's Time to Relabel the Brain

Another day, another study finds that 'visual' cortex is activated by something other than information from the eyes:

A research team from the Hebrew University of Jerusalem recently demonstrated that the same part of the visual cortex activated in sighted individuals when reading is also activated in blind patients who use sounds to “read”. The specific area of the brain in question is a patch of left ventral visual cortex located lateral to the mid-portion of the left fusiform gyrus, referred to as the “visual word form area” (VWFA). Significant prior research has shown the VWFA to be specialized for the visual representation of letters, in addition to demonstrating a selective preference for letters over other visual stimuli. The Israeli-based research team showed that eight subjects, blind from birth, specifically and selectively activated the VWFA during the processing of letter “soundscapes” using a visual-to-auditory sensory substitution device (SSD) (see www.seeingwithsound.com for description of device).

There's lots of research like this. People are excited by mirror neurons because they are cells in motor cortex that are activated by both motor activity and perception of that motor activity. It's incredible, people cry - cells in a part of the brain that we said 30 years ago does one thing seem to also do another thing. How could this be??

I would like to propose a simple hypothesis to explain these incredible results and that is that we have been labeling the brain incorrectly for a long time. The data telling us this has been around for a long time too and continues to roll in, but for some reason we still think the old labels are important enough to hold onto. It's time to let go.

I'm going to go out on a limb and say it's a bit more complicated than this

Radical Embodied Cognitive Neuroscience - A Frontiers Research Topic

UPDATE: This topic is now live! We welcome all and any submissions that fit the remit. Please email Andrew if you are interested and I will add you to the contributors list, or you can also simply contribute via the Frontiers page. 

A couple of years ago, Sabrina and I were chatting about the brain and running into the problem that we just don't know enough about how it works. We realised that what we needed to do was host a conference, invite some useful people, and pick their brains for a few days. 

We've had two goes at funding such a conference; we've had a lot of interest from the academics we've contacted but no luck convincing anyone to give us any money. Over the process, however, we got Tony Chemero (author of Radical Embodied Cognitive Science) involved, and he recommended Louise Barrett to us (that's how we came across her excellent book). The four of us have been scheming for a while to try and make this a reality, and two things have developed.

First, we are going to host a workshop on Radical Embodied Cognitive Neuroscience, hopefully at the Lorentz Centre in the Netherlands. Before that can really be worth doing, however, we've realised we need a little more momentum, so we've advanced our plans and are about to announce a Research Topic at Frontiers in Psychology. We'd like to invite all interested parties to play.

The goal is simple: we want this to be a virtual conference, in effect, where people pose problems and offer solutions to the problem of developing a radical (non-representational) embodied cognitive neuroscience. We want real collaborations to come out of this, so we want people coming looking for ways to help and be helped. And we want to create a resource that we can point to to shape discussions at future workshops. 

What we need from you
I've pasted the text of the call we will run below. If you are interested in submitting something to  this, send us your name, affiliation and email address (either in the comments below or email us, psychscientists@gmail.com). At this point, this commits you to nothing; we just need a decent length list of people to initially invite to submit, to indicate that there is going to be enough interest. If you change your mind later there's no problem.

Any thoughts on the call, let us know. We all like that this call is short, direct and to the point; too many of these research topic calls are inflated by too much detail. But if there's any flags, let us and know.

If you can help us by promoting  this post on social media, that would also be excellent. We are looking to cast a wide net.

Shared neural resources for throwing and language: a whacky idea for an experiment

For some time now, there has been an hypothesis floating around in evolutionary biology that the human capacity for language emerged, in part, from the development of our ability to throw long distances with high speed and accuracy. There are a few reasons to think this, mostly correlational, inferential kinds of reasons, but they are accumulating.

We were chatting one day about how to test this hypothesis a bit more directly, and we came up with a whacky experiment. We'd like advice from neuroscientists with experience in brain stimulation techniques about whether this sort of thing is feasible. We'd also like to brainstorm the logic of this experiment and see if we can come up with a practical design that stands a chance of finding something. We then need collaborators; I can handle the throwing side (analysis, measurement, etc) but we don't know anything about TMS and would need an expert on board.

There are many other reasons why this might fail, though - I still need to do a detailed lit review on the throwing/language references I have. Our main problem is that we don't know the kind of obvious difficulties in doing TMS in this kind of context. We'd like to assemble a) an experiment and b) a research team to do the experiment if we can get it to make sense, and if it works we will submit the hell out of this to Nature :)

Patient DF uses haptics, not intact visual perception-for-action to reach for objects

Before functional neuroimaging techniques like PET and fMRI became common, what we knew about which parts of the brain did what came from neuropsychology. This is the study of patients with specific injuries to the brain, and the basic logic of the field is that if you have a patient with a lesion in area A who can't do task 1, then area A is involved in performing task 1. It gets a little more complicated than this, as you search for double dissociations, etc, but this is essentially it.

A surprising amount of what we think we know about the brain comes from neuropsychology; famous case studies such as HM have informed theories of memory so that they include short and long term storage, which are separable, and so on. These case studies can have a profound effect on research; my favourite story, though, was about a memory researcher who had a skiing accident and temporarily developed retrograde amnesia - he couldn't remember anything except that there was this guy in Connecticut (HM) who couldn't remember things either!

I always enjoyed classes in neuropsychology; the case studies are always fascinating. But they are deeply limited in what they can actually tell us about the brain. First, they are typically single patient case studies, which restricts how general the conclusions are. Second, they are data from damaged brains; the fairly linear assumption that some localised function has been subtracted out is simply not true, and the damage will have had complex effects on distributed functional networks.Third, the damage is never straight-forward, because these almost all come from accidents or strokes (HM's surgery being a rare example of more detail being known). This has not stopped the field being very excited by these cases, though, and from basing a lot of theory on these patterns of deficits.

In movement research, the most famous neuropsychology case study is Patient DF She suffered bilateral damage along the ventral stream of visual processing (James et al, 2003). The effect was visual form agnosia: she is able to control her actions with respect to objects, but cannot describe or recognise these objects verbally. Crucially, her accident did not damage her parietal lobe; specifically, the dorsal stream of visual processing was left intact. These two streams are well defined anatomical pathways leading out of primary visual cortex, and were first described by Ungerleider & Mishkin, 1982). DF's pattern of deficits led Mel Goodale and David Milner (Goodale & Milner, 1992) to suggest functional roles for these streams. The ventral stream, they suggested, was for perception - things like object and scene recognition. The dorsal stream, in contrast, was for perception-for-action, and used visual information for the online control of action. This perception-action hypothesis has been hugely dominant in the field, and the theory rests heavily on DF's shoulders.

Recently, Thomas Schenk (2012a) published some data which claims to show that DF's visually guided reaching is not normal if she doesn't have access to haptic feedback about the object. His data suggests that the only reason she succeeds at reaching while failing judgment tasks is that haptic information is only normally available in the former case. If correct, this is actually quite a shot across the bow of the perception vs perception-for-action work; naturally Goodale and Milner don't buy it, and have published a reply to which Schenk has then replied.

An invitation
I like seeing these arguments happen in the literature; but to be honest, the time scale is too slow. Schenk publishes, then Milner et al get to reply and Schenk gets right of reply to that. They may or may not iterate again and it's always left as 'we agree to disagree'. But these critiques have answers, and I think a blog comment feed might be the right place to work through the various cycles of suggestions and rebuttals until the obviously wrong things have been weeded out. It would also provide a place for other interested parties to weigh in. So if Schenk, Milner and Goodale (and anyone else!) feel like using the comments for this post or another made to purpose to bang around ideas until an obvious experiment or analysis pops out, please feel free!

Did language emerge from the neural systems supporting aimed throwing?

Aimed throwing is surprisingly uncommon in the animal kingdom. Humans do it par excellence, and otherwise it only shows up occasionally, even in our closest relatives. Chimpanzees will throw things (often faeces) but unlike humans don't throw things when hunting or trying to get food; when non-human animals throw things, it's usually part of a social encounter.

Throwing is a fascinating task for many reasons; I hope to blog some about the perception-action aspects of this task in the future as I prepare a couple of papers on the topic with my colleagues Qin Zhu and Geoff Bingham (who have previously done some excellent work on throwing to a maximum distance and the size-weight illusion; various papers available here). There are many fascinating questions about the perception of the affordances of throwing and distances to targets which we're starting to tackle empirically.

Biomechanically, throwing an object accurately over any distance requires the precise transmission of force from the large trunk muscles along a kinetic chain formed by the segments of the arm. The large trunk muscles generate forces the arm cannot, and this force is then transmitted by the motion of the arm; each segment weighs progressively less and so the force accelerates each one faster than the last. The end result is a hand moving at high speed. This requires careful timing; if the motion of the segments aren't coordinated carefully you will waste energy moving the limbs in ways that aren't helping the throw.

There has been some speculation for a while now that the neural mechanisms that help support this fine tuned coordination and control for throwing might also be just the kind of resources that could support the development of spoken language. Speech is a complex action that requires exquisite control over the coordination and timing of numerous elements, just like throwing. One hypothesis is that our ancestors began to develop the ability to throw long distances (this being favoured by natural selection processes because it enabled us to hunt and kill huge prey with much less physical risk to ourselves; e.g. Calvin, 1983). Evolution selected for neural resources that supported this activity, and this then opened the door to the possibility of complex spoken language. So do we speak the way we do because we throw the way we do?

Some Ground Rules for a Theory of Psychology

Add psychology to the list

A fairly common response to our theory post was 'here's my theory, which is designed to replace and fix all the others'. However, it's more a symptom of the problem I was discussing than a solution for everyone to have their own entirely separate theory which doesn't talk to any other work in the field (see above). One of my personal goals in science is to not be that guy. I want to see cognitive science become more integrated, not more fragmented. We have also been asked, however, and quite sensibly, what we think the solution to our problem is. The question then is how to propose a theoretical approach for psychology and cognitive science where we don't just reinvent the wheel.

Sabrina and I have been working on this for, well, the entire blog. It has been a place for our "brave attempt to think out loud about theories of psychology until we get some" since day one; we've been identifying problems but, just as importantly, solutions the whole time. The theory post identified the big picture problem we see in psychology; time to lay out some solutions.

Step one is to present a map of the blog, organised thematically to guide new readers to work we've already done here. This should also help map out the gaps in the approach, so we can focus on things to do next; feel free to point us to problems we can't yet address! (And yes, we know about episodic memory and language - we're working on it.) This post is not a comprehensive summary of past work - it's a map for you to use to find what we've done so far.

To summarise: in essence, and some minor details aside, we are advocating for Chemero's (2009) radical embodied cognitive science, with the addition of some elements he was missing (network science & task specific devices). Cognition is embodied, extended and held together by the direct perception of affordances and events; the result is a complex, nonlinear dynamical system that must be analysed as such. The brain is not the sole source of our behaviour, nor is it representing the world; it clearly plays a critical role in this system, though, and we propose that we'll need the tools of network science to describe what it's actually up to (Sporns, 2010). Methodologically, we must carefully characterise the task, the resources available to solve the task (which include brain, body and environment) and the information these resources create which can sustain the formation and control of an embodied solution. This method is Bingham's (1988) task specific device approach (the main piece Chemero was missing, I think).This approach applies to all and any behaviour you want to explain, including the hard stuff like episodic memory and language.

Critically, this approach, while new (and uncommon in insisting on a role for Gibson's ecological approach) isn't just something we invented: all these elements are active parts of modern cognitive science. The only new part is bringing it all under one roof, with the goal of getting on and getting some decent normal science under our belts.

Here's what we've covered so far. If you want more details on any point, click on the links!

Embodied solutions to neural delays: Information and Network Motifs

One of the bugbears of direct perception is the fact of neural delays. The transmission of signals through the nervous system takes time, and this means that there is a lag between something happening (at, say, the retina) and that event having consequences in cortex, let alone behaviour. In control theory terms, delays in a system can lead to instability in that system's behaviour as you are forced to make corrections that are then incorrect and must themselves be corrected.

It's typically suggested in psychology that these delays are compensated for via computational predictions; the nervous system 'perceives the present' by taking the lagged input and using it as the basis of a guess about what's going on now (e.g. Changizi et al, 2008, plus see this post from a while back). This is a problematic idea: if the perceptual control of actions is based on a guess compensating for a variable time lag, then the stability issue remains, not to mention the consequences for mis-predicting the future. Regardless, it's not really an option for a theory of direct perception, and I want to discuss a couple of options.

There's More Than One Way to Rhythmically Move a Lobster

I'm slowly working my way through Olaf Sporn's excellent book, Networks of the Mind. The purpose of this book is to introduce neuroscientists to network theory, and vice versa; I'm eavesdropping and tooling up on both. It's slow going only because it's pretty much all new territory to me, but I'm seeing a lot of potential in the overall approach to the brain, and this just confirms for me that Sporns understands what he does pretty deeply. 

Anyway, a while back, Bruce Hood tweeted the following:

Fact: in the digestive tract of the lobster, its nervous system can have 100k-200k different neural states that produce the same behavior

This struck me as an astonishing fact; to my mind, it throws the idea that hunting for the neural correlates of behaviour into serious doubt. At the very least, it needs to radically change what you expect to find. I finally found the reference for this fact in Sporns' book (Prinz, Bucher & Marder, 2004; download), and Sporns has some interesting context for this fact (which Hood undersells - see below) and why it's interesting, rather than soul-crushingly depressing. There are some interesting potential consequences relevant to my current ponderings on the brain, although it's definitely still at the 'these sound like the same sort of principle' level of analysis. 

What's the difference between perception and conception?

As Andrew has been tackling a new job description for the brain (part 1 and part 2), several comments have been made that suggest that his approach (and the ecological stance in general) might be fine for perception/action, but not for other types of tasks/behaviours. Later on in this post I also think about how we might be able to distinguish between association and direct access to meaning, which is another idea that has been coming up repeatedly in the comments (see here).

In this post I want to think about what makes perception different from conception (Andrew reviews William James' views on this distinction here). I will argue that both occur as networks of evolving neural activity (with perception, this network extends to the environment and the body), but these networks have different properties because they are driven by signals of differential stability. I do not think it is accurate to think of perception as something that happens in V1, for example, and conception as something that happens in SFG. If parts of the brain reverberate in a system along with parts of the brain that are directly interfacing with an energy array in the environment, it seems correct to refer to that whole system as perception. In contrast, if a network of activity appears to be relatively encapsulated with respect to external energy arrays, it seems correct to refer to that system as an act of conception. For now I will leave aside the problem of how conceptual networks develop, but I hope to return to this later.

Mirror Neurons, or, What's the Matter with Neuroscience?

One of the problems I face as I try to figure out what the brain is up to, if not representing, is that I can't rely on the neuroscience literature to back me up. The problem is that, while there has been a lot of data collected over the years, very little of it has been collected within an ecological framework. Neuroscientists are looking for how the brain represents information, not how it perceives it; they're looking to see where perception and action are integrated in the brain, not how the brain-body-environment system produces stable, functional behaviour. This matters because there's no such thing as theory-free observations - all data comes from this experiment rather than that experiment, and even simply reporting a result is laden with theoretical assumptions, even when these aren't explicitly identified. So until I can find a neuroscientist interested in collecting a little data (and I would love to hear from any such person!) I'm limited to laying out the consequences of taking the ecological route and critiquing what's out there already. 

The worst offenders, in terms of theory-laden data disguised as 'merely the observed facts', are mirror neurons. They are, I think, the text book example of what's the matter with neuroscience, and I thought it was about time to talk about them a little. For those interested, the most recent exposition of what mirror neurons are and do can be found in a recent (and currently open access) issue of Perspectives on Psychological Science: there's a brief introduction (Glenberg, 2011a), a detailed Q&A paper (Gallese et al, 2011) by five main figures in the field, and a summary (Glenberg, 2011b).(Update: recent meta analysis finds mirror type activity all over the brain, including the cerebellum!)

What Does The Brain Do, Pt 2: The Fast Response System

I want to continue thinking about the implications of the claim that the brain does not trade in representations. I'm not looking to defend this view here; we got into it a bit last time, I've talked about it here, and if you can't imagine what cognition without representation might look like, then you should read this post by Sabrina and then Radical Embodied Cognitive Science before worrying me with your lack of imagination. If you then feel like getting into it, Sabrina is tackling this topic in detail, beginning here.

If the brain isn't mentally representing, what is it doing? Last time, I got into the idea that the brain is part of a embodied cognitive system. It's in the middle of a rich information flow, with access to perceptual information about the world and ourselves, and it's a critical part of the action system, involved in our responses to that information. I talked about it as the fast response system in the set of inherent dynamical resources available for us to use to form task specific devices; I want to expand on that a little.

There's More to Us Than Our Brains - So What Does The Brain Do?

I'm not that interested in the brain.

It's hard to be this way in modern psychology. Cognitive neuroscience is where it's at, and I think I come off as  a bit of a Luddite when I try to convince people fMRI is a bit of a waste of time. Not caring much about the brain is certainly a sociological reason why ecological psychology doesn't get taken very seriously; we're just the crazy people who don't think there are mental representations, based on some work from the 50s-70s. Surely modern imaging has shown us the activity of mental representations? Clearly, the brain is the source of all behavior! Popular science writing on psychology is all cognitive and representational; most of the psychology blogging I come across is neuroscientific. What else could it be?

I've certainly spent a lot of time waving the flag against the infiltration of neuro-talk into places it doesn't yet belong; but to be honest, as I get older, I've begun to worry that I'm trying to be 'fair and balanced' in the sense Fox News is fair and balanced: relentlessly playing up one side to offset a perceived imbalance elsewhere. What I actually want to do is be actually fair and balanced: I want my own discussions about these issues to be internally balanced and coherent, giving credit where credit is actually due. I want to start teasing apart a few issues I've conflated over the years, so that my strong concerns about the relevance of fMRI  and cognitive neuroscience work stop getting swallowed up in a general dismissal of the brain's role in our lives. The brain is clearly interesting, but it's not representing, and if not that, what is it doing?

This post is therefore a first swing at integrating a lot of the things I've been blogging about for a while and doing so in a way that leaves a sensible role for the brain. I'm going to need some neuroscientists to talk to, though; I'd appreciate it if people could spread the word on this a little, because there are just some things I want to go a few rounds on with people who know what they're talking about. 

Chemero (2009) Chapter 8: Neurophilosophy Meets RECS

Chemero's book finishes with two chapters on some philosophical consequences of taking a radical, embodied approach to cognitive science. Chapter 8 is about the mind-body problem, and how various attempts to reduce cognitive science to, say, neuroscience, can be vigorously resisted via the RECS approach, without being dualist about the mind. There are many people who think cognitive science can be reduced to neuroscience (intertheoretic reduction), but one plank of any embodied approach is that this won't work. RECS is particularly committed to a more extended notion of cognition and so a strategy for resisting reduction is critical. Chemero's plan won't rely on the usual philosophical manoeuvres such as Martian pain mechanisms or zombies. Like me, Chemero is concerned that these create the impression that philosophers aren't tackling real problems; he wants the philosophical conclusions of RECS to be grounded in data, and I thoroughly endorse this approach.