The Ecological Approach, Explained to an 8 Year Old

About 3 weeks ago I got an email from a person who had found our blog via Robert Epstein's piece 'The Empty Brain'. The email said

I've had a good read this afternoon, and it has been informative to some degree, however ...
I have an 8 year old son, and due to questions we both have, we have had some very interesting laypeople's conversations about the nature of experience and "the mind" (is it a thing, a physical thing, a process?) as well as such things as memory, embodiment and perception.
It seems it would be really helpful for us (and by extension, possibly many others?) if you could summarise the broad strokes of your theory in some way in which an intelligent 8 year old (and his father!) could understand.
Would this be possible?

Ed Yong has taught me that good science communication doesn't have to be dumbed down, it just has to be pitched right, and while I am no Ed Yong, I say, challenge accepted! Let me know how it goes!

A Gibsonian analysis of linguistic information

This post is based on a talk I just gave at the Finding Common Ground Conference at the University of Connecticut. Please excuse the Power Pointy nature of some sections! You might need to Ctrl+ to see some of the images clearly.  I have made some changes from the original talk content on the basis of very useful feedback I received from other conference attendees.

What is the place of language in ecological psychology? Is language a type of direct perception? Is language comprehension direct perception? Does language have affordances?

In trying to answer these questions I discovered that some things we think of as being perceptual have a lot in common with the conventionality of language and that some language-related behaviours look a lot like perception (as typically construed). I end up suggesting that we move away from talking about 'perception' and 'language' as different types of entities and instead focus on information / behaviour relations in specific tasks.

'Embodied Cognition Is Not What You Think It Is' - the paper!

Whoops, we did it again - a paper based on the blog! This time we are in press at Frontiers in Psychology, in a Research Topic on embodied cognition, with a paper we somehow got away with calling 'Embodied Cognition is Not What You Think It Is'. 

This paper draws from a lot of posts on the blog on embodied cognition, perception-action and language. We have used this opportunity to tackle some key issues head on, and we like this paper a lot :) We cover all the important issues and we set up what we think is the way forwards for embodied cognitive science. In addition, it sets up the ground work that we want to build on with our own Research Topic on Radical Embodied Cognitive Neuroscience. We've laid out what we think is the task facing the brain; this is what the brain is engaging with, and so this is what we think neuroscience needs to work with in order to understand what the brain is doing.

It's the kind of paper that will either land with a splash or vanish without trace. We want it to make some serious waves, and we're hoping that we can encourage people to publish free Commentaries on it at Frontiers, to challenge us or pick up our challenges, and, most fun for all, to come work with us to take all this forwards! We want this to be the basis of an empirical research programme and we want you all to work with us on it :) At the very least, feel free to pepper us with questions; this paper is the start of something for us, not the end and we're interested in the response to this paper to frame the next step.

Learning the affordances for maximum distance throwing

Over the last couple of posts, I have reviewed data that shows people can perceive which object they can, in fact, throw the farthest ahead of time by hefting the object. Both the size and the weight of the object affect people's judgements and the distance thrown; however, only weight affects the dynamics of throwing (release angle and velocity are unaffected by changes in size). This rules out the smart perceptual mechanism proposed by Bingham et al (1989), which proposed that both size and weight changes affect hefting and throwing the same way. So how are people perceiving this affordance?

Using coordination to study learning across the lifespan

What happens to our ability to learn new movement skills as we age? There is surprisingly little research on this topic; a relatively recent review (Voelcker-Rehage, 2008) found only 25 articles about learning in old age, and no systematic programme of work. The answer to this question matters a lot; rehabilitation after events such as a stroke pretty much always entail (re)learning movement skills, and if our ability to learn gets worse with age, rehabilitation faces an uphill struggle. 

I have been studying coordinated rhythmic movement for some time now, and now we have a good handle on the task dynamic my colleagues at Indiana and I have begun using it to study the process of learning more generally. We decided to use it to look at learning in old age, to see what we could see.

This project grew out of a grant I had from Remedi when I was a post-doc in Aberdeen. I wanted to use coordination to look at learning post-stroke. One of the problems with studying this is finding useful novel tasks to learn - you need to give the stroke patients something they've never done before so you can be sure that any improvement is about learning, and not simply recovery of function. My thought at the time was that I could use any changes at 180° to assess recovery and changes at 90° to assess learning. We tested a huge number of patients and age matched controls, but the project didn't pan out because neither group (all aged around 65) couldn't learn to move at 90°. The question remained, what was going on? We now have the first of three papers on this question out in press. 

Language isn't magical (but it is special)

One of the most common comments about ecological psychology is that it's hard to imagine how it could apply to things like language. The sense is that language is a completely different kind of beast than perception-action and that it requires a completely different theoretical account (cognitive psychology). Andrew and I disagree. In this post I outline the similarities and differences between language and other types of perceptual information. The main idea is that language is indeed the same type of thing as perception-action, but there are key differences between them in the relationship between the information and what it means. These differences permit language to be flexible according to context, culture, and goals; to be expandable according to changing needs; and to be portable, allowing us to access information about things that are not currently in the environment. These properties make language special, but not magical.

An Ecological Approach to Language

Language is often held up as an example against the possibility of the radical (non-representational) psychology we advocate for. You might be able to explain perception-action without representations, people say, but we can't see how you'll ever be able to explain 'real cognition, like language' without them. It's finally time for us to begin chipping away at this criticism. In the next few posts I'll lay out a first draft of an embodied, ecological analysis of language use.

Psychologists usually assume that catching a fly ball and talking about catching fly balls are two different kinds of thing. I reject this assumption (it is just an assumption) and I am going to treat language use as the same kind of thing as other examples of embodied cognition. Treating language as just another instance of embodied cognition allows me to import the lessons learned from perception-action type tasks and apply these to language tasks. This will lead to very different questions about language use than are typical in the literature. The next post will describe what I mean by this in some detail.

Review: Louise Barrett's "Beyond the Brain"

There are a lot of books about embodied cognition. Like psychology itself, the idea that cognition might not all be in the head has fractured off into about 5 distinct flavours, most of them quite annoying and pointless.There are also quite a few books about the thesis of the 'extended mind' (Clark & Chalmers, 1998) ; the idea that cognition quite literally extends into our bodies and environments, in terms of the tools we use and the objects we interact with. It's getting quite hard to find a book on these topics that isn't a) simply rehashing old ground or b) trying to come up with it's own distinct flavour of embodied, extended cognition that fixes some problem of everyone else's. I find this sort of fracturing and endless bickering quite distracting and troublesome, for reasons best expressed by xkcd. I also agree with Tony Chemero, that much of this work is still, sometimes deep down, representational. I think this is because they fail to embrace a sufficiently useful theory of information, such as that proposed by Gibson. So I often find myself reading these books finding nothing new and far too much that's depressing.

Louise Barrett's book, "Beyond the Brain: How Body and Environment Shape Animal and Human Minds" is, refreshingly, not like this at all. Barrett is a psychologist who studies animal cognition and behaviour, and her book does a lot of things very, very well. I'm not planning on a chapter by chapter book club on this, although I may at some point; Eric Charles has posted a few thoughts on the book as well, here, here, here, and here. To cut to the chase: if you are new to the area of embodied cognition, read this book. If you're familiar with the literature but want a clear, well-structured presentation of many of the key ideas, then read this book. If you're bored with the same old examples and want some new, perhaps more convincing examples of embodied cognition in action, read this book. And if you have heard some of the arguments but still think behaviour really comes from the computational activity of our complex brains, then, for the love of science, read this book.

Prospective Control I: The Outfielder Problem

A couple of posts ago I raised the distinction between prediction and prospective control. I was trying to make the point that, if you are coupled to the right information, you don't need to be mentally simulating what's happening so you can run this simulation ahead and predict what's coming up. Prediction of this sort is invoked by representational cognitive scientists to cope with things like delays in the nervous system (e.g. Changizi's 'perceiving-the-present' framework). It's a risky business (if you make a mistake, you are suddenly controlling your behaviour according to an incorrect guess), and the better solution is prospective control. This is when you couple your behaviour to information in the world that doesn't tell you about the future, but that, if you use it for a while, will get you where you want to go.

People requested some more on this topic, and so here we go. People also requested something other than the outfielder problem, but I am going to start with this problem because it is still the best example, and I'll get into some more as we go to demonstrate it's not just baseball where this works.

Coordination dynamics and relative speed

The Bingham model of coordinated rhythmic movement makes three predictions. First, it predicts that movement stability is a function of perceptual ability, and we confirmed this in two ways (by showing how people can move stably at non-0° with transformed visual feedback (Wilson et al, 2005) and by showing that perceptual learning of 90° led to improved movement stability without practice at the movement task; Wilson et al, 2010). This prediction is also supported by recent work by Kovacs and Shea, who are busy demonstrating that transformed, Lissajous feedback breaks the classic pattern of movement stability in coordination tasks. The second prediction is that relative phase is specified by the relative direction of motion; we confirmed this by selectively perturbing various components of motion and showing selective effects on performance (Wilson & Bingham, 2008). 

The third prediction was that the detection of relative direction was conditioned on the relative speed; the latter was simply a noise term. de Rugy, Oullier & Temprado (2008) tested this prediction by using an amplitude manipulation to alter the relative speeds. Their data did not support the model predictions, and they concluded that the approach taken by the Bingham model was flawed. We recently replicated their experiment (Snapp-Childs, Wilson & Bingham, in press as of Friday; download) and identified numerous critical flaws in their design and analysis which invalidated their criticism.

There's More Than One Way to Rhythmically Move a Lobster

I'm slowly working my way through Olaf Sporn's excellent book, Networks of the Mind. The purpose of this book is to introduce neuroscientists to network theory, and vice versa; I'm eavesdropping and tooling up on both. It's slow going only because it's pretty much all new territory to me, but I'm seeing a lot of potential in the overall approach to the brain, and this just confirms for me that Sporns understands what he does pretty deeply. 

Anyway, a while back, Bruce Hood tweeted the following:

Fact: in the digestive tract of the lobster, its nervous system can have 100k-200k different neural states that produce the same behavior

This struck me as an astonishing fact; to my mind, it throws the idea that hunting for the neural correlates of behaviour into serious doubt. At the very least, it needs to radically change what you expect to find. I finally found the reference for this fact in Sporns' book (Prinz, Bucher & Marder, 2004; download), and Sporns has some interesting context for this fact (which Hood undersells - see below) and why it's interesting, rather than soul-crushingly depressing. There are some interesting potential consequences relevant to my current ponderings on the brain, although it's definitely still at the 'these sound like the same sort of principle' level of analysis. 

Mirror Neurons, or, What's the Matter with Neuroscience?

One of the problems I face as I try to figure out what the brain is up to, if not representing, is that I can't rely on the neuroscience literature to back me up. The problem is that, while there has been a lot of data collected over the years, very little of it has been collected within an ecological framework. Neuroscientists are looking for how the brain represents information, not how it perceives it; they're looking to see where perception and action are integrated in the brain, not how the brain-body-environment system produces stable, functional behaviour. This matters because there's no such thing as theory-free observations - all data comes from this experiment rather than that experiment, and even simply reporting a result is laden with theoretical assumptions, even when these aren't explicitly identified. So until I can find a neuroscientist interested in collecting a little data (and I would love to hear from any such person!) I'm limited to laying out the consequences of taking the ecological route and critiquing what's out there already. 

The worst offenders, in terms of theory-laden data disguised as 'merely the observed facts', are mirror neurons. They are, I think, the text book example of what's the matter with neuroscience, and I thought it was about time to talk about them a little. For those interested, the most recent exposition of what mirror neurons are and do can be found in a recent (and currently open access) issue of Perspectives on Psychological Science: there's a brief introduction (Glenberg, 2011a), a detailed Q&A paper (Gallese et al, 2011) by five main figures in the field, and a summary (Glenberg, 2011b).(Update: recent meta analysis finds mirror type activity all over the brain, including the cerebellum!)

There's More to Us Than Our Brains - So What Does The Brain Do?

I'm not that interested in the brain.

It's hard to be this way in modern psychology. Cognitive neuroscience is where it's at, and I think I come off as  a bit of a Luddite when I try to convince people fMRI is a bit of a waste of time. Not caring much about the brain is certainly a sociological reason why ecological psychology doesn't get taken very seriously; we're just the crazy people who don't think there are mental representations, based on some work from the 50s-70s. Surely modern imaging has shown us the activity of mental representations? Clearly, the brain is the source of all behavior! Popular science writing on psychology is all cognitive and representational; most of the psychology blogging I come across is neuroscientific. What else could it be?

I've certainly spent a lot of time waving the flag against the infiltration of neuro-talk into places it doesn't yet belong; but to be honest, as I get older, I've begun to worry that I'm trying to be 'fair and balanced' in the sense Fox News is fair and balanced: relentlessly playing up one side to offset a perceived imbalance elsewhere. What I actually want to do is be actually fair and balanced: I want my own discussions about these issues to be internally balanced and coherent, giving credit where credit is actually due. I want to start teasing apart a few issues I've conflated over the years, so that my strong concerns about the relevance of fMRI  and cognitive neuroscience work stop getting swallowed up in a general dismissal of the brain's role in our lives. The brain is clearly interesting, but it's not representing, and if not that, what is it doing?

This post is therefore a first swing at integrating a lot of the things I've been blogging about for a while and doing so in a way that leaves a sensible role for the brain. I'm going to need some neuroscientists to talk to, though; I'd appreciate it if people could spread the word on this a little, because there are just some things I want to go a few rounds on with people who know what they're talking about. 

Visual feedback for training novel coordinations

The key feature of coordinated rhythmic movements is that not all coordinations are stable. Most other rhythms can be learned, however, which is why we can have jazz drumming. People have been training participants to perform novel coordinations (especially 90°, the least stable rhythm without training) for years now, and have been asking all the standard learning questions - how long does learning take? Does it transfer to other coordinations? 

The first real studies on learning were by Kelso and Zanone (Kelso & Zanone, 2002; Zanone & Kelso, 1992a, b, 1997). I briefly reviewed the results of these studies here, which have lead to to the dynamic pattern hypothesis. This account describes stable states as attractors in a state space defined by relative phase as the order parameter, and learning is the creation of a new attractor centred on the target novel phase. This account ran into problems quite quickly but is still alive and kicking in a modified form; stability is the governing principle now, and from this perspective the feedback displays used for training don't matter so long as they support stable action. 

However, from our perception-action standpoint, the feedback displays matter a lot, because these are what's providing the perceptual information about the coordinated movement. Early learning studies all used some kind of transformed feedback, which we could never use because it altered the overall perception-action dynamic. In order to look at action learning directly, we needed a new form of feedback. 

So I invented one.

Task Specific Devices and the Perceptual Bottleneck

I've been wanting to blog this paper, Bingham (1988; download link), for some time, and I've had the excuse to be reading it this week as I develop a grant. There's a lot here, and many of these brief points are worth posts in and of themselves. My goal here was to create a walk through of the paper, and I hope to dive into some of these issues in more detail.

This paper comes from Geoff Bingham, my PhD advisor at IU. And, like most of the good things Geoff has taught me over the years, this paper is a gift that keeps giving as I come to grips with what's in it. What it does is lay out a methodological problem (the massive redundancy and complexity of the human action system), proposes a solution (studying task-specific devices) and firmly embeds the idea that these devices are intrinsically perception-action devices (by discussing the so-called perceptual bottleneck). In effect, it lays out a way to be a productive scientist studying a hugely complex system without shying away from the complexity. This paper blew my fragile little mind when I first read it, and I'm still pulling good ideas from it today.

This paper is what I think the science of perception-action should look like. It's the piece I think Chemero (2009) is missing for his radical embodied cognitive science, and it contains (oddly without a lot of specific references) all the key ideas that have come up on this blog in a single coherent frame work (e.g. Gibson & specification; Turvey et al on the symmetry principle). Frankly, if you want to study perception-action systems from a dynamical systems perspective, this is what you have to acknowledge is the lay of the land and these are the beginnings of the tool kit you'll need.

Perception, Action & Dynamical Systems

Over Easter I visited the Center of Functionally Integrative Neuroscience at Aarhus University in Denmark, courtesy of the Interacting Minds group. I gave a talk, got the tour, and met some of the faculty and students - some interesting opportunities for future collaborations, I hope - thanks for the hospitality!

I wanted to lay out the basics of the talk I gave. I took the opportunity to present some ideas that have been developing as I work on this blog, reading Chemero and working on coordination experiments. There is a core of people in Aarhus interested in things ecological, as well as dynamical systems, so it was a good audience to try these ideas out and they seemed to go over well. This is also the sketch of a paper Sabrina and I are going to work on over the summer.

The take home message of the talk was simple - dynamical systems is the right kind of mindset for cognitive science, but it is not a theory of behaviour. Dynamics merely provides the right kind of modelling tools - the form of the model must be based on hypotheses about the specific kind of dynamical systems we are or else they are merely an exercise in data-fitting. Ecological psychology is the right theory, and the Bingham model of coordinated rhythmic movement is currently the only example of a genuinely perception-action dynamical systems model. My thoughts here are largely from my response to Chapter 4 of Chemero (on 'the dynamical stance') and Chapter 5, his initial attempt to use dynamics to serve as a guide to discovery which I think fails and which Chemero then replaces with ecological psychology. The description of Bingham's model comes from here.

F*cking affordances - how do they work?

Over on Bounds of Cognition, Ken has been doggedly pursuing what he thinks is a critical problem with the concept of affordances as described by Gibson (1979) and expanded on by Turvey, Shaw, Reed & Mace (1981; hence TSRM). I feel the need to spend some time consolidating my responses and some ideas in one place; every time I try to lay out why the problem is ill-posed or a potential route out, Ken just says I'm clouding or avoiding the issue or throwing out red herrings.

I actually think Ken has identified one very useful critique of the affordance concept: the lack of care with which we attach '-able' to words. But Ken isn't just making a methodological point; he thinks he's shown that affordances cannot structure light in a way that can specify the affordance, and that's the argument that needs to be tackled.

Saddle up: this is going to take some time, and I'm not going to solve everything. But to get a little ahead of myself, the answer to the problem is that it's complicated.

The Size-Weight Illusion is Functional, and It's About Throwing

My colleagues, Geoff Bingham and Qin Zhu, have recently published some fascinating data which has emerged from their work on the uniquely human skill, long-distance throwing. This is a novel and rich perception-action task which Bingham and Zhu (and recently, me) have been investigating for some time, with many interesting results. I'll get onto blogging about this project once I've caught up with the coordination studies and have had some time to get my head around the data I'm helping generate.

I wanted to blog about this new paper, though, because it's an exciting result which deserves all the attention it gets. The result is about the size-weight illusion, one of the most robust illusions around. As I've talked about before, illusions are a concern to ecological psychologists only in that they suggest the task has been incorrectly characterised. This paper presents data that suggests the size-weight illusion is actually functional, and that it reflects the readiness of the human perception-action system to throw objects long distances.

This paper has seen some activity in the popular press already (e.g. here and here): Geoff's hoping for the NYT Science section too! 

UPDATE: Geoff being interview on NPR

Identifying the Visual Information for Relative Phase

Bingham's model predicts that the information for relative phase is the relative direction of movement. The first direct test of this hypothesis was the experiment that followed on from my learning study, in which we systematically perturbed the various candidate information variables to see which affected performance in the perceptual judgement task.

I like this study a lot, if I do say so myself. It's a serious attempt to make a strong test of the model's predictions, and we invested a lot of time in the methodology. This is also that rare paper that benefited from a vigorous review process; the end result is, I think, a clear, careful, and detailed presentation of a critical result for the perception-action approach Geoff and I are developing.

Readers interested in the issue of how you can scientifically study information from an ecological perspective should certainly read the paper (Ken, that's you :); it's my go-to reference for how I believe this has to be done. The main lesson - it's hard to do this properly, but the rewards, in terms of unambiguous data, are clear.

Perceptual Learning Stabilises Action: A Test of the Bingham Model

Bingham's perception-action model was initially inspired by perceptual judgement studies (using vision and proprioception). The HKB phenomena are movement phenomena, however; simply noting that the same qualitative pattern is seen in different judgement and action studies is a good first step but only suggestive, at best. We therefore next took simultaneous judgement & action measures from a movement task where we manipulated the feedback display (Wilson et al, 2005a). For instance, when the display showed 0°, movement was stable, even when the movement was at, for example, 90°. Perception of relative phase was driving the stability of the movements.

On the basis of all this data, the model predicts that the reason 0° and 180° are easy is that the information specifying that you are moving this way is easily perceived. There is provisional evidence to support relative direction of motion as the specifying information (Bogaerts et al, 2003; Wilson et al, 2005b; Wimmers et al, 1992) with relative speed acting as a noise term. This variable certainly predicts the observed pattern, as the relative direction of motion is only stable at 0° and 180°. It is maximally variable at 90°, which would explain why movements here are also maximally unstable. The model is therefore explaining the problem with moving at 90° as a problem detecting the information required to maintain the coordination; as we saw in the case of friction, no information means unstable behaviour.

The model therefore makes a critical prediction. If we could improve people's ability to perceive 90°, they should gain the ability to move at 90° without any practice at the movement itself. All previous learning studies had entailed training people to move by having them move, with the help of various forms of transformed feedback methods (visual metronomes or Lissajous plots; more on this when I discuss feedback). The prediction, that movement stability should improve with improved perceptual ability, is a strong test of both the model and the modelling strategy in general, and the experiment to test it was the first half of my dissertation.