About 3 weeks ago I got an email from a person who had found our blog via Robert Epstein's piece 'The Empty Brain'. The email said
I've had a good read this afternoon, and it has been informative to some degree, however ...Ed Yong has taught me that good science communication doesn't have to be dumbed down, it just has to be pitched right, and while I am no Ed Yong, I say, challenge accepted! Let me know how it goes!
I have an 8 year old son, and due to questions we both have, we have had some very interesting laypeople's conversations about the nature of experience and "the mind" (is it a thing, a physical thing, a process?) as well as such things as memory, embodiment and perception.
It seems it would be really helpful for us (and by extension, possibly many others?) if you could summarise the broad strokes of your theory in some way in which an intelligent 8 year old (and his father!) could understand.
Would this be possible?
What Does Cognitive Science Want to Know?
The basic question in cognitive science (psychology plus related fields like linguistics, computer science, neuroscience, philosophy and more) boils down to this: "Why did that person do the thing they just did, in the way that they did it?" There are two basic answers to this question; the computational and the ecological approaches.
The Computational Approach
If you look at what goes into the person from the environment (sensations) and look at what comes out (behaviours), they don't look anything alike. Sensations (input) clearly have to be taken, altered, processed until a behaviour (output) can be generated.
The main thing sitting in between sensation and behaviour is the brain, and it's clearly built in ways that enable it to do this kind of processing, transforming work. The mainstream view in cognitive science is therefore that people do what they do because the brain makes us. Cognitive science therefore needs a way to talk as precisely (read: mathematically) as possible about those brain-based processes.
The ideas that makes this possible come from computer science. Pioneers like Claude Shannon and Alan Turing invented the maths that made it possible to take (almost) any input and process it into (almost) any output. All the maths that make it possible for your computer to take some input (e.g. a mouse click) and transform that into an action (e.g. taking you from one web page to another) were now available to carefully describe the processes of the brain.
We therefore end up with this basic hypothesis: the form of our behaviour is caused by computational mental and neural processes that transform input into that output, and we need all this because our perceptual contact with the world (through sensations) is not rich enough to explain the form of the behaviours we can get up to.
 |
| James J Gibson |
The Ecological, Embodied Approach
It turns out, however, that there was a psychologist (James J Gibson) who spent the better part of 40 years figuring out that our perceptual contact with the world is, in fact, amazingly rich and detailed. People realised that Gibson's theory might offer a way to think about perception that could replace the need for mental representations and computations; suddenly there was another option.
The key to the ecological approach is that the brain is not the only place where the good stuff happens. Our environments offer some opportunities for action and not others (Gibson called these 'affordances'), our bodies enable us to do some things and not others, and the way our bodies perceive and act in their environments allows all this to change in interesting, complex but not random ways.
As soon as it becomes possible that the form of our behaviour can be caused by something other than just the brain, everything changes. The job of the brain changes from "process input into output" to "link body and environment through perception and action". The brain is no longer necessarily a computer, because computation is no longer necessarily the thing that has to happen to get to behaviour. The difference between the two approaches to behaviour is captured by the slogan, 'Ask not what is inside your head; ask what your head is inside of'.
A Simple Example
How does a baseball outfielder catch a fly ball? In terms we've been using so far, we want to know the form of the actions the outfielder takes and how that relates to various ways of solving the problem.
The computational strategy is to detect the initial motion off the ball from the bat and then to predict where it will land. You can technically do this because the physics of projectile motion (motion caused by an initial force and then left to run without any more help) is fairly straight forward and you can predict where the ball will go pretty quickly. This account predicts that the form of the outfielder's behaviour will be to run in a straight line (the shortest distance) from where they are to where they need to be.
There are two ecological strategies. Fly balls follow a curved path and they change speed (they slow to a stop at the top, then speed back up as they fall). The two strategies are to move so as to cancel out one aspect of this motion. Optical Acceleration Cancellation predicts that the outfielder's behaviour will be to run with varying speed that tries to offset the acceleration of the ball. Linear Optical Trajectory predicts that the outfielder's behaviour will be to run along a curved path that tries to cancel out the curvature of the ball's path. Weirdly, it turns out that if you succeed at either of these, you will arrive in the right place at the right time to catch the ball.
The data unambiguously support the ecological strategies. Outfielders never simply run to the predicted landing location; instead they run along curved paths at varying speed in various combinations of the two perception based strategies. The form of their behaviour maps directly onto the form of the perception of the environment.
Whenever the details of the perceptual coupling to the environment have been worked out and tested, human behaviour always shows the various tell-tale signatures of online perceptual control, rather than mental prediction. While there are many tasks still to solve, so far so good.
I hope that helps; comments and questions welcome, I'm keen to fine tune this as much as possible!
Dear psych S.
ReplyDeleteCouldn't the ecological approach (example baseball) be the result of repeated, and endless, re-computations (corrections to initial prediction)?
No. The behaviour dos not look like this is what is going on; no straight line running
DeleteI disagree. For many problems, there is more than one way to do it (different algorithms for the same task). You start and end in the same places but the path may differ. Some ways of doing stuff may be more/less useful depending on circumstances. Look up all the sorting algorithms for an example.
DeleteBy showing ball catchers not running in straight line you don't dismiss computation as such, you only show that humans do not use "precompute-and-move-to-the-destination algorithm". We use a different way to catch a ball - one that is easier for our monkey brains & bodies. More "ecologically suitable".
Repeated, endless re-computations and corrections is exactly what is taking place.
The point of ecological approach is that we rely on (quick) feedback from the environment quite a lot while doing our computations.
Repeated, endless re-computations and corrections is exactly what is taking place.
DeleteWhat is being computed?
Where and how to move to catch the ball.
DeleteHow else would you engage muscles if your central nervous system wouldn't compute it first?
Why compute when you can perceive?
DeleteIsn't perception of the environment one of the inputs for computation? I mean what drives the catcher to the end goal of motion if not based upon information about speed, height, trajectory (perception) and experience (LT memory)?
Delete@Andrew
DeleteUnless you claim that you can always link sensations to actions exactly 1:1 without any interaction between signals whatsoever, you'll end up with some kind of computation (transforming input to output that is different from the input). Or "signal processing" if you dislike the word "computation" - I guess that our notions of it are a bit different.
For me, the point of ecological/embodied approach is that these computations might be quite straightforward because we can use neat algorithms that are enabled and supported by the properties of the environment and our bodies.
In short: the ecological approach doesn't dismiss the computational approach. It refines it by trying to explain out HOW the computing is done.
Unless you claim that you can always link sensations to actions exactly 1:1 without any interaction between signals whatsoever, you'll end up with some kind of computation (transforming input to output that is different from the input).
DeleteOne of the overlooked findings in all the perception-action research is that once you identify the right information variables, the action tends to unfold in a way that matches how the information unfolds. In essence, as a general rule in the perceptual control of action, the primary job of the nervous system seems to be to preserve the spatiotemporal structure of the information variable so that it shows up in behaviour. See work by Audrey van der Meer, for example.
So what exactly are the required transformations that the brain is adding to the mix?
Isn't there still transformation because perception and action are two different things? They unfold at the same time, but the brain must tie them together and isn't that a transformation?
Delete@Andrew
Delete"... the action tends to unfold in a way that matches how the information unfolds."
I'm still an undergrad and I'm just now coming across this, so I hope this response to your work doesn't come across as too old-hat. I want to go to graduate school to study linguistics, and I'm in the process of thinking about what kind of work I want to do. What little I've read about dynamical systems and embodied cognition has been fascinating. I'm still trying to decide what to think about computationalism, and I'm hoping you can give me some feedback.
In the baseball example, is it really so clear that the action unfolds like the information? My first instinct here is to emphasize a distinction between the behavior itself, and the action as we perceive it. By the behavior itself, I mean only the motor responses to the stimulus: pumping the legs while modifying their speed while also stretching out the hands. By the action as we perceive it, I mean the path the outfielder takes to the ball.
From the perspective of an observer watching someone catch a ball using an ecological solution, I would perceive the outfielder's path to unfold very much how I perceive the ball's path to unfold.
But say I'm the outfielder herself, rather than an observer. I don't perceive my own action in the same way a scientifically-minded observer perceives it; I simply behave. And the pattern of leg muscle contractions that move my body in the correct direction looks nothing like the sighting on the ball that I am trying to maintain.
In between the perception of the ball's path and the motor response to it, it seems like there must have been transformation, because they're so different. But I could also imagine how, to an observer, there could still appear to be no transformation because the outfielder's embodied context (i.e. skeletal muscle physics, the traction of her cleats on the grass) transforms her behavior back into an action that corresponds with the ball.
This is what my intuition tells me, anyway. I haven't read nearly enough about the concepts you're talking about, and I definitely want to. For now, does any of that sound reasonable? And if not, how would you amend it?
Thanks for the well-pitched explanation!
-Gabriel Brinton
Hi Gabriel
DeleteThis is an excellent question. Let me see if I can explain what I mean.
Yes, the 'perception' bit and the 'action' bit do not, at one level, resemble each other. What I mean, though, is that the spatial-temporal dynamics of the behaviour matches the spatial-temporal dynamics of the information. By "spatial-temporal dynamics" I simply mean "the way these things change over space and time".
So let's take the Linear Optical Trajectory case. Informationally, what you have to do is move so as to cancel out a spatial change in the ball's trajectory, from a parabola to a straight line. Action wise, how you do that is you run in such a way to make that change happen. The way your running unfolds (eg when you change to make a correction etc) is directly tied to the way the information is changing (eg when the trajectory shows a curve). See the curvature, move to offset it.
One trick here is that the system is not working to make you do one movement. It's working to produce an outcome, a function. The details of the movement emerge in real time as the system does whatever it has to, given what it's currently doing, to preserve the function. But the overall form of the behaviour (not the muscle activity, but the behaviour) is shaped by the shape of the information.
Does that help?
Hi Andrew,
DeleteYes, thank you for your reply; it helps a lot! It sounds like what you're saying is very much in keeping with the idea expressed elsewhere on this blog that spatiotemporal dynamics occurring outside the brain are actually included in what we call "cognition." Is that characterization correct?
I like your use of the word "resemble," because I think it might explain Jan's insistence that transformations are happening. Part of the reason I made that distinction between the action and the behavior was to cut away the spatiotemporal stuff look as narrowly as possible at what the brain in particular is up to, because that's the part of the outfielder's ecological strategy that we still don't understand. And just seeing that lack of resemblance between perception and action, at a first impression, was enough to make me go, "Huh, well something ~transformative~ is obviously happening here." Do you agree with that? If I'm reading Jan correctly, I think he and I both had the same thought process there.
I'm tempted to answer your last question to him ("What exactly are the required transformations ...?") in light of that: The transformations are those that make the optical array resemble a motor signal, and they're required because it doesn't already resemble one. What particular transformations they are strikes me as a research question. But I suspect from what I've read elsewhere on your blog about representations that the concept of transformations may also have some baggage that causes you to eschew the term. What might that baggage be?
Also, I should mention I've really been enjoying Sabrina's series on ecological language, though I'm waiting to comment on those until I'm a little more confident on the concepts involved.
-Gabriel
spatiotemporal dynamics occurring outside the brain are actually included in what we call "cognition." Is that characterization correct?
DeleteYes. Nicely put.
Part of the reason I made that distinction between the action and the behavior was to cut away the spatiotemporal stuff look as narrowly as possible at what the brain in particular is up to, because that's the part of the outfielder's ecological strategy that we still don't understand. And just seeing that lack of resemblance between perception and action, at a first impression, was enough to make me go, "Huh, well something ~transformative~ is obviously happening here." Do you agree with that?
This is actually a nice summary of the mistake standard cognitive science makes, as I see it. You cannot understand the brain without knowing about the perception-action stuff, and when you try, all you see is a huge gap between sensation and motor control that has to be filled somehow. Once you start looking at ecological information, you see it's structure in behaviour (coordination dynamics, outfielding, etc) and also in the brain (van der Meer's stuff). So yes, I think this move is likely the source of the intuition - but it's a bad move, ecologically.
The transformations are those that make the optical array resemble a motor signal, and they're required because it doesn't already resemble one. What particular transformations they are strikes me as a research question.
So there may not be transformations that have to be implemented. For example, a running outfielder might just be what you get when you couple a fielder with their particular dynamics to that information variable. See this post for more on what I mean here.
the concept of transformations may also have some baggage that causes you to eschew the term. What might that baggage be?
The primary baggage is that the concept lives in one ontology (theory about the nature of the world; here, the informational processing ontology) and may simply be an error, if the ecological ontology is more accurate. It is not a necessary feature of cognition; it's something that's only required if signals require active transforming, rather than the more dynamical view in the post I linked above.
>>This is actually a nice summary of the mistake standard cognitive science makes, as I see it.
DeleteHuh, that is not what I was going for. I was thinking about how we should proceed after we feel we have characterized the task correctly. I like the "Ask what your head is inside of" quote, and your arguments in the same vain elsewhere on your blog make a lot of sense; it's hard to understand what the brain is up to before we understand what it's up against. As a chemistry major, I also value scientific rigor in the conclusions we draw.
And I think I'm beginning to understand what you mean by the information processing ontology as computational cognitive psych conceives of it, and why an alternative is necessary.
But say we do have a pretty good understanding of the task and the resources and the information and the etc. Does ecological psych tell us that we shouldn't study what's inside your head, even then? That sounds less like a difference in theories or methodologies, and more like a difference in research interests.
It seems to me that, whatever it ends up looking like, a viable ecological alternative to the info processing ontology has to support scientists who are interested in studying what the brain is up to. You've argued before here that people often misunderstand Gibsonian psych as claiming that the brain isn't up to anything, and I see how that might be a difficult misperception to surmount. At the same time, talking about "a fielder with their particular dynamics" just begs us to ask what those particular dynamics are, and one thing the information processing ontology is really good for is, it gives us some vocabulary to speculate with. The ecological ontology's vocabulary so far doesn't seem to afford brain-up-to-what-speculatability, and I think it needs to, in order to really challenge the dominant ways of doing psychology. After all, speculation is a necessary function of even the most rigorous science. How can we do that without referring to information processing terminology?
This has been stimulating a lot of really interesting conversations with my neurosci and psych major friends. As always, thanks for your response!
I'm not an 8-year-older but I really needed this.
ReplyDeleteI will take that as a compliment and a hint that I hit the right spot :)
DeleteWell - I suppose the trajectory of a baseball is somewhat dependent on the air it flies through (temperature, wind) and how this interacts with details - which are not easy to read - of the ball's movement (such as spin). This means that the trajectory is not so easily computed from only the angle and speed of the ball - environmental condition and the interaction with these needs to be taken into account as well and they only unfold as the ball traverses it's course. Thus you need an ongoing perceptual activity to correct your position.
ReplyDeleteA different example - billiard - is perhaps not so straight forward? Getting ready to make a shot typically involves moving round the table, taking in different perspectives, but it also involves considering the exact angle and power you wish to hit the ball with.
Wouldn't a dual theory better account for this? Where you have both the embeddedness of moving round the table and a more theoretical reasoning about strength and angle? I don't suppose ecological psychology would claim, that we culturally can't develop and learn to use our brains to something like holding a representation of something in our imagination? Certainly that is what much of school is about. Of course the representation would be reminiscent of actual motor-perceptual experiences but it would nevertheless be another way of planning action than the in-the-moment of running to catch a ball?
Good post, by the way - great you take the time :-)
The problem with the dual model is twofold:
Delete1. There's no evidence for it. Whenever computational and ecological solutions are tested directly against one another, the computational solution is no where in sight in the data.
2. You are proposing that the nervous system implements two radically different kinds of solution. This is actually quite a big claim, and comes with problems like "when do I use this one vs that one?". So while I get the urge to combine, it's important to remember it is not a trivial thing to do.
Billiards; I bet if you looked at eye movements and the kinds of warm up movements that happen as people heft their cues etc, you would see a lot of action designed to parameterise and calibrate an upcoming action. I don't see huge trouble here for the ecological approach.
Thanks for the questions, glad you enjoyed the post! :)
Thanks for the answer. I feel the need for a follow-up comment, though :-) First 2), then 1) and finally billiards :-)
ReplyDelete2) Perhaps we could call them 'ends of a continuum' instead of opposites? I think you sort of sidestep the issue I try to raise: The humans I've talked to generally agree that there are situations where their cognition is more dependent on movement than at other times. Planning how to solve a math puzzle, before putting the pencil to the paper, is an example from one end of the spectrum. Catching a ball on a windy day is from the other end of that spectrum. It is intuitively difficult to see these two situations as intrinsically similar and entailing the exact same processes.
1) There are situations in which movement is severely restricted but where cognition is intact. A strong example is the locked-in-syndrome where people are unable to move but still able to function intellectually.
Billiards: Such a 'locked in person' could in theory successfully play a game of virtual billiard by adjusting angle and strength of a virtual cue using a brain-computer interface. Perhaps disregarding minimal eye movement this would eliminate the use of warm up movements etc.
If you could accommodate the explanation you started out with, so that it explicitly tackles situations which are not absolutely dependent on movement, I think it would sink in better... with me at least... and perhaps other 8 year olds as well ;-)
No one is saying brains don't do things. The trick is they are not doing everything. Much of the form of our behaviour comes directly from the task and our perceptual contact with it.
DeletePlaying billiards is not the same task as virtual billiards, and there is literally no way to make them the same task. So the fact you can rig up a virtual game here tells me nothing about performance in the real version.
Notice that absence of movement does massively alter the things you can do. If you could test action control in such a person, you would find all kinds of calibration problems, etc that are the consequence of not being able to move. So sure, some stuff can happen without overt movement; but it's unstable and limited.
Don't get too caught up on the outfielder problem. It's just an example to clarify the different approaches.
Nice post Andrew. How would you explain the differences between Behavior Analysis (B. F. Skinner's Radical Behaviorism) and Gibson's Ecological Approach? I'm aware the explanation of the outfielder's behavior would be quite different from a Skinnerian approach, but how would you describe the philosophical, epistemological differences, as you did between the Computational Approach and the Ecological Approach?
ReplyDeleteThank you!
First, I love and respect Skinner's work. The man was not quite right but by god he was onto something and he was not wrong about so many things.
DeleteWhat Skinner lacked is dynamics and ecological information; he was mapping out structure and contingencies in the environment and showing how that all showed up in and accounted for behaviour. But he was expressing his description of the environment in inadequate terms (he needed dynamics) and he didn't have a theory of perception (he needed information).
Basically he was sufficiently right in his belief that the structure of behaviour often comes from the structure of the environment that he was able to generate reliable effects. He had the right idea but an inadequate formal vocabulary.
Gibson was very much at the behaviourist end of things; he thought of himself as a molar behaviourist like Tolman. He just went a few steps further into the mechanistic details of how it all actually worked.
First of all, the 8 (now 9 year old) and his Dad were very pleased to see their request answered.
ReplyDeleteSecondly, it did make sense - all of it (though some explaining of the language was needed for the 9 year old).
The questions which now come up are:
1) Where does thought come from? Is it just a stimulus response?
2) What about imagination / dreams / daydreams / emotions?
3) Could you give us some more examples of where " the details of the perceptual coupling to the environment have been worked out and tested"?
And thanks once again for this post - it's made for a very interesting afternoon's conversation!
Glad it helped!
Delete(1) and (2); no idea. They are big questions and the ecological approach has not tackled everything. We have a first draft idea of what might be going on in the brain here, although it's still first draft and also a little technical. The hypothesis is that we think you can build more abstract thought than required for catching a fly ball out of interactions with information and a nervous system handling that interaction. There are MANY devilish details to be worked out!
(3); check the rough guide for the posts on coordinated rhythmic movement, especially about Geoff Bingham's perception action model (we summarised a lot of this in this paper). I did my PhD on this with Geoff and we've done a ton of empirical work mapping this all out in a lot of detail.
Also, if you have the time; what language didn't work? My kids are 3 and 5 so I didn't have an 8yr old to run things by :)
DeleteI loved this and I will be coming back here.
ReplyDeleteIt seems to me that, in each of a series of moments, the outfielder is doing whatever brings him/her closer to a desired feeling -- maybe it could be called "the feeling of what's worked in the past in this context" or
"the feeling of being closer -- vs further away -- from the moment of catching this ball." I'm wondering if, instant by instant, behaviour depends on fluctuations in that feeling.
I wouldn't think that physical movement is necessary; when I was enthralled by algebra in high school I got good at resolving equations quickly because I had a feeling, based (I assume) on past experience of similar feelings and actions, that a particular step had a good chance of bringing me closer to a desired state (the feeling of satisfaction / achievement of having resolved the equation.)
But it does seem likely that a body is necessary; otherwise where does the input of "feels good" or "feels bad" come from?
Is there overlap here with "implicit learning" and "tacit knowing"? Or are these just labels referring to "things that are hard to explain"?
I always look forward to your blog.
ReplyDeleteHow does the ecological versus non-ecological contrast differ from dead-reckoning versus trial and error?
Also, your outfield baseball example does not allow enough for uncertainty. Considering the interplay between a pitcher and a batter would force the analyst to consider the indetermiancy of inter-brain interaction as well as probabilities based on past experience and differential ability. There is an enormous amount of historical data that could be mined to test alternative analyses against a large set of variables.
Hiya! I also think your explanation was very clear and very winning. In reading some of the comments, I think what might need more rhetorical emphasis is that the strategy of MOVING in relation to perceptual data is fundamentally different from COMPUTING. Then, when we watch fielders actually do this, it looks very much like they are using the former. This seems quite coherent to me vis-a-vis the initial contrast between models of cognition, and I wonder if, ironically, resistance to embodied cognition might be the old cartesian dualism making a curious comeback. That is, have we swallowed the bitter pill that mind is a product of evolution, but now are refusing to honor the feet? I'm reminded of Pinker's claim that there will never be a Decade of the Pancreas. But hasn't the stomach recently been called a "second brain"?
ReplyDeleteIn the sport of curling, the person delivering the stone must adjust the force or speed and the judgement as to whether it is right for the shot being played gets described in terms of sensation - "feel" is the word typically used. Seems more to fit in the ecological approach than the computational.
ReplyDeleteYour statement: The job of the brain changes from "process input into output" to "link body and environment through perception and action", was the 'A-ha' moment I've been looking for for so long. Thank you!
ReplyDelete