People requested some more on this topic, and so here we go. People also requested something other than the outfielder problem, but I am going to start with this problem because it is still the best example, and I'll get into some more as we go to demonstrate it's not just baseball where this works.
The task
The first step in answering such a perception-action question is a task analysis. What, precisely, is happening in the task and what are the optical consequences of what's happening? In baseball, outfielders are often required to catch fly balls. These are balls hit high in the air, and the outfielder has to move from where they are to where the ball will land in the time it takes the ball to get there from the bat. The ball is typically moving pretty quickly, and is a long way away from the fielder, so it is very small. This last point means some types of time-to-contact information (like tau) aren't available, because the variation in the apparent size of the ball with changes in distance are below threshold. You can, however, learn to catch fly balls pretty reliably; the question is how.
The critical feature of a fly ball is that it moves through the world along a parabola; it is an example of simple projectile motion.
 |
| Figure 1. Simple projectile motion follows a parabola |
Potential solutions
1. Prediction
The prediction solution suggests you take the initial angle and velocity and use those to predict the parabolic path, assuming constant ball size and mass (fair, given it's a baseball) and constant air density, drag and gravity (less fair; drag varies non-linearly with things like spin and wind). If you have access the parabola, you simply read off where the ball will intercept the ground and go there to wait. However, even though you can estimate the ball's behaviour fairly well with some sensible assumptions about drag, etc (Saxberg, 1987a), people don't seem to use this strategy at all (Saxberg, 1987b).
2. Prospective control: Optical Acceleration Cancellation (OAC)
If people aren't acting on predictions generated by an internal simulation of the physics, they must be coordinating their actions using information available during the flight of the ball. The ball is too far away for information such as 'looming' to be of any use. However, there are certain optical consequences when the fielder moves relative to something travelling along a parabola that enable the fielder to refine that movement so as to intercept that object.
 |
| Figure 2. Optical Acceleration Cancellation (OAC). From McBeath et al, 1995 |
This strategy has some appeal; it is an intrinsically perception-action strategy (you are required to move so as to produce a specific spatial-temporal pattern in the optics), there is continuous information about whether you are moving correctly, and you end up doing the right thing without ever knowing where the ball is going to land until you intercept it. However, while there is some support for it (most recently Fink, Foo & Warren, 2009) there are some problems. The first is that the evidence suggests people are not sufficiently sensitive enough to optical accelerations to be able to implement the strategy. Second, the geometry that produces the solution only works properly when your path and that of the ball are aligned; in baseball, this is often not the case. Third, as Figure 2 shows, you actually have to cancel out the acceleration of the ball along the slanted projection on the left; this means you have to cancel the acceleration of tan(α), and not α per se (where α is the angle between the ground and the ball as seend from your vantage point). Like tau, OAC is sound in principle, but the lawful relation between the world and the optics simply doesn't range over a suitably wide scope to allow stable behaviour.
3. Prospective control: Linear Optical Trajectory (LOT)
OAC is on the right track, but isn't good enough to support the observed behaviour (namely outfielders catching all kinds of fly balls). McBeath et al (1995) proposed a different strategy that shares the important features of OAC (continuously available information that allows you to control your behaviour with respect to your goal); this strategy is to move so as to make the ball appear to move, not along a parabola, but along a straight line. This strategy is called Linear Optical Trajectory, or LOT (Figure 3)
 |
| Figure 3. Linear Optical Trajectory (LOT). From McBeath et al, 1995. |
Which strategy do people use?
It's looking good for LOT, but it is always an empirical question what information is being used, especially when there are alternatives (Gibson, 1979). The strategies entail different paths to the same location: prediction and OAC suggest that you will run along a straight path at constant speed; LOT suggests you will accelerate then decelerate along a curved path. Prediction also suggests you will simply run to a given location, whereas the prospective strategies suggest you will move according to the information, not the end location.
McBeath et al (1995) collected some simple data from 2 participants; they tracked the fielder's path and also recorded with a camera the optical trajectory that results from these paths. The paths were curved and entailed accelerations, while the optical trajectories were linear, on 75% of the trials. This suggests outfielders typically rely on LOT. McBeath et al also suggested that the fact skilled baseball players often run into walls and catch on the move, rather than rushing ahead of the ball, also work against the predictive strategy.
It's not yet over for OAC (see Fink et al, 2009); and people continue to run experiments showing that either LOT or OAC dominate in varying conditions. However, all the data support the idea that one of these prospective methods is being used, and prediction of any kind is simply not being made.
Summary
The reason I use the outfielder problem is that it is a complex, real world task that is amenable to the kind of detailed optical analysis required to identify potential sources of visual information for prospective control. That analysis has revealed at least two potential perception-action strategies in which people move so as to make the optic array contain a particular state of affairs, i.e. information. These strategies can be distinguished on the basis of the paths they predict people take (McBeath et al, 1995) and how those paths respond to certain perturbations (Fink et al, 2009). Even dogs can use these strategies (Shaffer et al, 2004), and demonstrating that non-human animals solve a task the same way we do is sometimes a useful way to help rule out the kind of complex internal representations of physics psychology thinks humans, but only humans, have. The outfielder problem in particular, and work on prospective control in general, shows how complex behaviour that looks like it requires a complex mental life can emerge by the operation of a simple rule over time (and Eric's discussing these topics with respect to Louise Barrett's book 'Beyond the Brain' on his blog).
References
Fink, P., Foo, P., & Warren, W. (2009). Catching fly balls in virtual reality: A critical test of the outfielder problem Journal of Vision, 9 (13), 14-14 DOI: 10.1167/9.13.14 Download
McBeath MK, Shaffer DM, & Kaiser MK (1995). How baseball outfielders determine where to run to
catch fly balls. Science (New York, N.Y.), 268 (5210), 569-73 PMID: 7725104 Download
Saxberg, B. V. H. (1987a). Projected free fall trajectories I: Theory and simulation. Biological Cybernetics, 56, 159-175. Download
Saxberg, B. V. H. (1987a). Projected free fall trajectories I: Human experiments. Biological Cybernetics, 56, 177-184. Download
Shaffer, D. M., Krauchunas, S. M., Eddy, M. & McBeath, M. K. (2004). How dogs navigate to catch Frisbees. Psychological Science, 15(7), 437-441. DOI Download
Posts
does a distinction need to be made here about expert and non-expert performance? Were the people in the McBeath study practiced outfielders? Presumably if they were North American, it would be likely... I could certainly get my head around the notion that we learn to use prospective mechanisms in situations where there is enough dynamic information (i.e. situations where we have optic flow and a moving target), because there suddenly become too many variables to predict accurately. But any simple, static reach to grasp task (which, I could contend, is not as uncommon as ecological psychologist types would have us believe), it seems hard to suggest that we could learn (or inherently) a prospective mechanism that would be easier than a predictive 'sort of like what i did last time' effect.
ReplyDeleteGavin,
ReplyDeleteAs you surely know, the Gibsons' split their intellectual turf. For some reason, the field has never successfully put things back together. You are quite right that we would not expect a novice to use an ideal strategy for intercepting balls, and that some process of perceptual attunement must be a key part of a novice becoming an expert. As a whole, eco psych people need to come up with better ways to conceptualize developmental processes that might effect their experimental results.
Andrew,
I really liked the presentation, but I'm still suspicious about the terms (i.e., the words "predictive" and "prospective"). I am also sympathetic to Gavin's point that we should see a range of strategies that people use to solve various tasks under various conditions. That is, even if we assumed the existence of higher-order variables specifying the course of action needed to solve all possible tasks, wouldn't we still expect to see both types of control in practice?
So, learning. The subject's in McBeath's study were indeed experienced. Not experts, but they'd thrown a baseball or two around. This is, of course, critical: no one is saying that you simply respond to the information because it's there, you must come to differentiate that information from the global flow of the optic array. This isn't a critique of Gibson, though: requiring learning isn't a weakness.
ReplyDeleteOn 'static reach to grasp', and 'doing what you did a minute ago'. First, there is plenty of information available about size, shape, distance, etc. This is all dynamic, flow based information that is available because you aren't, ever, static. Your eyes move in your head (and this is enough flow for you to easily discriminate a lot of information, even when otherwise in a bite board). Your head moves, as does your body, etc etc. So there's plenty of flow, and invariants over that transforming flow are, by definition, information about the object for the control of prehension.
it seems hard to suggest that we could learn (or inherently) a prospective mechanism that would be easier than a predictive 'sort of like what i did last time' effect.
My standard, slightly rude reply to this these days is 'your lack of imagination should not be interpreted as my problem'. You should look at Joe Anderson's and Geoff's new paper in EBR on locomoting to reach; there's plenty of information available, and people switch as they approach the object as a function of the stability of the information. The walking, the reaching, the switch between information variables, is all controlled by the availability and stability of the various information sources. The moral of the story is, identifying information is hard but it can be done if you ask the right questions,which you won't do if you assume prediction.
And 'on the basis of what you did a minute ago'; my question is, on what basis did you just do that? Especially given you probably succeeded?
I really liked the presentation, but I'm still suspicious about the terms (i.e., the words "predictive" and "prospective"). I am also sympathetic to Gavin's point that we should see a range of strategies that people use to solve various tasks under various conditions. That is, even if we assumed the existence of higher-order variables specifying the course of action needed to solve all possible tasks, wouldn't we still expect to see both types of control in practice?
ReplyDeleteIf you mean 'both predictive and prospective', my answer is 'no, why would we?'. Why would the brain go to all the trouble of implementing a noisy, unstable solution to a problem it is currently solving much better with information?
I'm more inclined to agree that we'll see multiple informational strategies, to the extent that multiple sources of information that specify what you need to know are available. Hence, while LOT seems to dominate, OAC works just fine sometimes too.
I'm more inclined to agree that we'll see multiple informational strategies, to the extent that multiple sources of information that specify what you need to know are available.
ReplyDeleteWell... yeah... but... I have a nagging feeling (that I am quite willing to be proven wrong about) that supporting that statement will require you to give up the word 'specify' or play fast and loose with the term 'information'. For an hard core eco-psych type, like yourself, those two options should amount to the same thing.
Look... lots of decisions in the course of an animals life are made based on imperfect environmental evidence. You could call the what-they-are-responding-to 'cues', or 'non-specifying invariants', or whatever you want, but organisms do not always couple their behavior to specifying variables. Often, organisms find themselves in circumstances where specifying information can't be accessed, other times they don't have the time, and still other times they just don't have enough experience in similar situations to be attuned right.
Now, as a crazy behaviorist, I'm happy to claim that organisms are always responding to some identifiable environmental variable (with as much dynamics as you want thrown in). However, about half the research that made up the classical field of Ethology was aimed at determining what aspects of the world organisms respond to, and lots of the answers such research leads to are weird. The same perceptual attunement that makes a baby gull correctly beg-food from its parent, also makes the gull beg-food from a yellow stick with a red dot. That might not be the best example, but hopefully you get the drift. The week-old gull is not basing its behavior on impossible-to-lead-it-astray specifying information. Is the baby gull's begging predictive or prospective? I'm not sure how such an example fits into the dichotomy, but the baby gull is definitely performing a perceptually guided task. How can we, as ecological psychologists, talk about the range of options we know much exist in nature?
(If not obvious, that last line is a somewhat desperately-honest question.)
Part of the problem is this: it's not a complete description of the situation to say 'information x specifies state-of-the-world y'. Law statements like this require a description of the scope over which the law applies. So 'OAC specifies your future path to intercept a ball' needs the additional constraints 'when that ball is following a parabola and your future path is aligned with the arc of that parabola'. LOT is more stable because you can relax the second constraint. But both OAC and LOT can be said to specify your future path, if you aren't being precise enough. The scope of the lawful relation between OAC and interception is narrower than that for LOT; outside that scope, OAC no longer specifies, although it is still an approximate solution.
ReplyDeleteSo you can have multiple variables specifying the same thing, although their scope often varies. Gibson was happy with this; this is why he insists that identifying the actual affordance properties and information being used is an empirical business.
Now, how well attuned you are to a variable is not part of the scope. The variable either does or does not specify something, regardless of whether you can detect it. This has to be the case, or else perceptual learning would have no target to head for. So your (in)ability to detect a specifying variable is only a critique if you weren't ever able to learn the invariants. For tasks you have had some decent exposure to, I'm happy to bet you'll have attuned to the actual invariants (although 'decent exposure' might be a long time - see how long it takes to learn to walk).
So, you're right: there is an interesting question of what you are doing in the meantime. Rob Withagen thinks you can ecologically perceive non-specifying variables, and I need to read some stuff before I get into that.
The same perceptual attunement that makes a baby gull correctly beg-food from its parent, also makes the gull beg-food from a yellow stick with a red dot. That might not be the best example, but hopefully you get the drift. The week-old gull is not basing its behavior on impossible-to-lead-it-astray specifying information. Is the baby gull's begging predictive or prospective?
ReplyDeleteNeither: the information it's attuning to isn't about the future. 'Prospective' is a sub set of 'informationally coupled' that deals with controlling behaviour with respect to the future.
Now, it's also not responding to proper information. Barrett talks about this: evolution can wire up a preference to bias the fast learning that is imprinting, but it can't wire in the ability to detect an invariant. The bias is there to speed up the learning process, it's not the outcome of the learning.
Also, specifying doesn't actually mean 'impossible to lead astray'. The problem of the evil psychologist reveals that you can often create the information without the actual physical state of affairs (see Fink et al above, who used VR to simulate fly balls). However, this kind of reverse engineering says nothing about how likely it is for two states of the world to produce the same information; this is the heart of Runeson's excellent smack down of the Ames Room.
Ken Aizawa and I had this out in some detail over the shark example from Turvey et al (1981), beginning about here and going on for quite a long time.
On 'static reach to grasp', and 'doing what you did a minute ago'. First, there is plenty of information available about size, shape, distance, etc. This is all dynamic, flow based information that is available because you aren't, ever, static. Your eyes move in your head (and this is enough flow for you to easily discriminate a lot of information, even when otherwise in a bite board). Your head moves, as does your body, etc etc. So there's plenty of flow, and invariants over that transforming flow are, by definition, information about the object for the control of prehension.
ReplyDeleteBut none of the info that you can get from movements of the eyes or head will give you any prospective insight into properties such as the weight of an object. At best, you can guess the weight based on how it looks (size/material), or you can guess based on how heavy the last thing you lifted was, if there are no apparently useful visual cues to weight. There's reams of papers out there which show that your gripping and lifting forces reflect how heavy you expect something to be, rather than how heavy it was (presumably, this can only be described as predictive), often leading to errors. With new looking stuff, we have to learn how to lift it every time. Once you've had the opportunity to interact with something, you can then presumably change your strategy to something more prospective-seeming, but why? If the object hasn't fundamentally changed its properties, then it's easy to remember your successes or failures from previous interactions and use that information. In short, I'm more than happy to accept that many parts of any task could readily be described by the use of prospective information, but (and this is my general bugbear for all eco psych) to suggest that this is how all actions are controlled under all situations, and we just have to try and think outside the box to figure out how, even in the face of glaring parsimony, seems over the top. I can't see how the hunt for an all-or-nothing solution for human behaviour is going to lead to anything other than frustration and an intellectual dead end...
Gavin,
ReplyDeleteI agree with you... mostly.
One of the problems is in demarcating the types of problems that ecological psychologists are interested in. There are definitely certain types of tasks for which the ecological explanations work more naturally. People who want to expand ecological explanations to all possible situations are going to have a problem - either they are going to start ignoring / denying certain types of real situations, or they are going to start fuzzying up their terms to they can apply to all situations. Some things that people do are not well described as "perceptual tasks", such as the examples you give, where all perceptual cues have been intentionally nullified by the experimenter. In those cases, an ecological-style explanation (right or wrong) will seem contorted and strained.
I should add the disclaimer that it seems so painfully obvious to me that the core of ecological psychology is a certain subtype of psychological problem, that I am continuously surprised that other ecological psychologists think differently. The failure too keep a "core" in mind when discussing these issues seems to me to have caused much confusion, and led people to see eco-psych as more very abstract and mystical, when it is really very concrete. It is a problem for the field.
There are several things going on here. Some specifics:
ReplyDeleteThere's reams of papers out there which show that your gripping and lifting forces reflect how heavy you expect something to be, rather than how heavy it was (presumably, this can only be described as predictive), often leading to errors.
On what basis have you set up an expectation? What was the information that created that expectation, and how does it relate to what happens next?
That expectation that's been established somehow, then, might more usefully be described as calibration. Calibration of a dynamical system is not predictive; it was based in information and, perhaps, some intrinsic dynamic characteristics of the measurement device. A skilled actor always comes 'pre-calibrated', you are never a blank slate starting from nothing. If that calibration reflects properties of the thing to be lifted (which it will start to as soon as you lift it) then you will appear to have correctly predicted; if not, you will seem to have made an error.
So I deny that it can only be described as predictive. That's a big part of this: establishing that there's something else it might be (here calibration).
Why calibration? Because treating the perception-action system as a non-computational dynamical system provides calibration as a mechanism that you can use to solve these kinds of problems. This, then, will either work or it won't, to the extent that we're right to treat the perception-action system this way.
Once you've had the opportunity to interact with something, you can then presumably change your strategy to something more prospective-seeming, but why? If the object hasn't fundamentally changed its properties, then it's easy to remember your successes or failures from previous interactions and use that information.
Why use information instead of a guess? Force control is actually a good example: the inertial changes created as you move an object are typically highly non-linear. The frictional forces between your fingers and the object alter in all kinds of unpredictable ways as you move, for example. So why switch? Because prediction won't work.
n short, I'm more than happy to accept that many parts of any task could readily be described by the use of prospective information, but (and this is my general bugbear for all eco psych) to suggest that this is how all actions are controlled under all situations, and we just have to try and think outside the box to figure out how, even in the face of glaring parsimony, seems over the top.
ReplyDeleteI always liked Fox Mulder's idea of Occam's Principle of Limited Imagination as a reminder that simplicity is relative to the system you are studying.
The perception-action approach is a theory of perceptually controlled actions. The only problem is, as Eric notes, finding out what counts as perceptually controlled and what doesn't. We all push this boundary in various ways, with varying success - but there are certain core cases where it is simply an example of an action that we are trying to explain, and having a theory means starting there and ruling out other alternatives until you break your theory by not being able to find evidence for your account. Turvey's admonition that if you haven't found it yet you just haven't looked hard enough is just a reminder to psychologists not used to not being allowed to give up really easily that they have to work it a lot before you give up such a well supported and coherent theoretical account.
I can't see how the hunt for an all-or-nothing solution for human behaviour is going to lead to anything other than frustration and an intellectual dead end...
This, of course, is not what we're doing. The perception-action approach is having great success pushing the boundaries of what it's able to account for but of course it hasn't explained everything, nor does it claim this. What it is, and what I keep pushing for, is a theory that you should apply first before throwing up your hands in defeat because you can't see how the system just did that thing without prediction.
"OAC is sound in principle, but the lawful relation between the world and the optics simply doesn't range over a suitably wide scope to allow stable behaviour."
ReplyDeleteI'm afraid I have to disagree with you there, Andrew. The OAC idea as proposed by Chapman was two dimensional (Chapman was no fool) and I implemented a version of his model computationally a long time ago (Tresilian, 1995, QJEP). The analysis showed that an implementation of Chapman's ideas could work even when it is impossible to detect the value of the optic acceleration and there are significant time delays in the control loop (a well understood source of instability). That is, the simulated fielder got to the right place in 2D despite the time-delays, poor acceleration detection, limits on running speed and acceleration and so forth. The solutions seem to be stable (i.e., no instability was found using reasonable sets of model parameters).
Despite all this, I don't believe that this is how running to make a catch actually works. I prefer a predictive account along the Saxberg lines (see Tresilian, 1995), but I confess to having no empirical basis for this preference (but Fink et al. did not persuade me to change it).
The analysis showed that an implementation of Chapman's ideas could work even when it is impossible to detect the value of the optic acceleration
DeleteHow is that possible??
If OAC is more stable than I thought, though, that's kind of interesting. One of the critiques of Fink et al is the perennial one for Warren's lab, namely they seem to be the only people who end up finding optical acceleration is used for anything.
(Also, hi James! Nice to hear from you :)
I belatedly answer your question "How is that possible?"
ReplyDeleteThe model could not detect the value of the acceleration (i.e., it could not say whether the acceleration was 3.4 degrees/s/s or some such thing), it could only detect whether the acceleration was in the direction of increasing angle or in the direction of decreasing angle. So it could detect the sign but not the magnitude of the acceleration. This was sufficient to get the catcher to the right place at the right time.
Ah...that makes sense. Of course you can cancel acceleration with just the sign. Thanks :)
DeletePerhaps this question has been answered already, but two additional questions dealing with the outfielder problem.
ReplyDelete1) The coordination between two outfielders. Meaning that when a ball is hit between two players closest enough for one to predict he can catch it and the other will go into a "safety" position to catch the ball behind the primary outfielder if he were to drop the ball. This ultimately I believe would require some element of prediction and therefore representation. This is most evident in the following link: http://www.youtube.com/watch?v=mM_XYs_bAU4&t=1m10s
2) Second this has to do when there is very limited direct sensory information, or that information comes at the beginning and at the end. The best example of this is the "The catch" via Willie Mays in the 1954 World Series. See this video: http://www.youtube.com/watch?v=7dK6zPbkFnE
Basketball examples of this would be a quick turn baby hook from the block with a baseline spin. Often this can be done without looking at the hoop and the ball is released prior to looking at the basket. This adds to the speed of the shot and the decreases the likelihood that it can be contested. In this example. I would be interested to know your speculation regarding such a scenario. Also given the basketball scenario the dynamic system is controlling the projectile towards a fixed target goal.
It seems to me that embodied cognition at its best intimately ties action to perception. Yet there may be overriding representation schemes which may control overarching goals of actions, re: turn on catch mode, or turn off catch mode (given the outfielder example). That these modes most likely have a representational component. There is also a circular feedback loop which can control compute the probability that the ongoing action will be a success and therefore control and conserve energy expenditure. For example the outfielder who starts running hard but then pulls back, knowing his teammate is in a better position. This it seems would be governed by a representational scheme.
I would be interested in your feedback in how the above scenarios may demonstrate a coupling between embodied cognition and representation.
Good questions.
Delete1. Coordination between players
Both players are watching the same ball event from different points on the field. One typically is better placed to catch the ball, so one of them will clearly perceive that they can catch it while the other will perceive that they can catch it but will need to work to get it. They can also perceive the fact that there's another fielder in the mix; outfielders patrol given regions, and they can see each other and are possibly calling to each other (when I played cricket whoever called for the catch got to take it).
Like all things in perception, there are threshold issues; if they are both perceiving a very catchable ball they may both go for it (I've seen fielders hit each other hard this way).
2. Limited information
Events have structure that extends over space and time. Once you are trained to perceive the relevant information for that event, perceiving part of the event can keep you informed about other parts of the event. For example, think about watching someone walking behind a picket fence; as they swing in and out of view, you are always perceiving them, you are not alternating between perception and memory.
Gibson talks about this; when you, the trained observer, perceive an object from one side you have access to part of the perspective structure in the optic flow that specifies the size and shape etc of the object. EB Holt has some nice thoughts on this too.
Fly balls have a nice, simple structure that unfolds in a fairly compulsory fashion; once the initial conditions are set, the landing point is set and that trajectory is specified by the first part of it. So a (highly trained) fielder can access the whole event from a partial view.
There is, of course, an element of luck to these kinds of catches :)
So this is all still about perception. LOT and OAC are the most stable solutions because they allow online corrections. Catching without looking is still possible if and only if you have perceived enough of the event structure to have been informed about the whole event structure. How much that is is a function of the event dynamics (simple events like a fly ball requires only some information) and the perceiver's expertise (practice lowers thresholds, etc).