Task Specific Devices and the Perceptual Bottleneck

I've been wanting to blog this paper, Bingham (1988; download link), for some time, and I've had the excuse to be reading it this week as I develop a grant. There's a lot here, and many of these brief points are worth posts in and of themselves. My goal here was to create a walk through of the paper, and I hope to dive into some of these issues in more detail.

This paper comes from Geoff Bingham, my PhD advisor at IU. And, like most of the good things Geoff has taught me over the years, this paper is a gift that keeps giving as I come to grips with what's in it. What it does is lay out a methodological problem (the massive redundancy and complexity of the human action system), proposes a solution (studying task-specific devices) and firmly embeds the idea that these devices are intrinsically perception-action devices (by discussing the so-called perceptual bottleneck). In effect, it lays out a way to be a productive scientist studying a hugely complex system without shying away from the complexity. This paper blew my fragile little mind when I first read it, and I'm still pulling good ideas from it today.

This paper is what I think the science of perception-action should look like. It's the piece I think Chemero (2009) is missing for his radical embodied cognitive science, and it contains (oddly without a lot of specific references) all the key ideas that have come up on this blog in a single coherent frame work (e.g. Gibson & specification; Turvey et al on the symmetry principle). Frankly, if you want to study perception-action systems from a dynamical systems perspective, this is what you have to acknowledge is the lay of the land and these are the beginnings of the tool kit you'll need.

Some terms
Part of the goal of this paper is to lay out a framework for studying perception-action systems. With that comes some terminology, and these are the key ones I think I've relied on the most here:

Human Action System (HAS): the full system, the one capable of walking and throwing and prehension and,... 

Task-Specific Device (TSD): a specific example of one thing the HAS can do, which you then study.

Inherent dynamics: the resources available to the HAS from various internal systems, e.g. the nervous system. 

Incidental dynamics: the task-specific resources available to the HAS out in the world.

Resource dynamics: all available resources, both inherent and incidental.

The human action system is complicated

One of the key ideas in motor control is the degrees of freedom problem (Bernstein, 1967*). A degree of freedom (df) is anything in a system which is free to vary, and the problem is the sheer number of them in the human action system. If something can vary then it must be controlled, and if there are too many things to control then control becomes impossible. 

However, we clearly are able to control our actions. The suggestion (by Bernstein) is that we temporarily solve the control problem by reducing the number of dfs, assembling them into synergies. Take a simple example: hold your index finger in front of you and move it around, using all the knuckles and joints. Now hold it straight - 'freeze' the degrees of freedom in all the joints except where it joins your hand. The reduced system is easier to control, although at the cost of some flexibility. This trade-off is acceptable, however, if the lost flexibility is irrelevant (or even beneficial) to performance in a task.

Bingham's first point is this: the 'freezing degrees of freedom' idea is fine, but can't guide research because we can't specify the initial set of dfs the system has to work with. The dfs of the human action system (HAS) come from multiple subsystems, each with their own (often nonlinear) properties, so we can't get a description of the unfrozen system to begin with. Bingham notes that the set of degrees of freedom describing the inherent dynamics of the HAS come from at least four distinct subsystems (the link-segment system, the musculotendon system, the circulatory system and the nervous system, plus the respiratory and nutritional systems, although he suggests these latter are less immediately coupled to the performance in the moment). These must be mobilised and coupled to the incidental dynamics of the particular task at hand. In the case of throwing, which I will blog about in much more detail later, these incidental dynamics would be the physics of projectile motion. So the full, unfrozen system with all it's degrees of freedom is the wrong place to start doing science.

Task-Specific Devices

The proposed solution is to start at the other end and study task-specific devices. The research strategy involves studying the organisation and composition of particular examples of the human action system at work, and work from there to describe the kinds of resources the overall system has access to for building such devices. In other words, figure out how a given task is achieved, and by doing so, come up with a list of things you know the system as a whole is capable of, which can then inform your study of related tasks.

This is not straight-forward, but it is much more achievable than trying to understand the HAS in it's entirety. So this approach is methodologically useful. More interestingly, I think, is the fact that it mirrors the solution adopted by the HAS itself; the system's job is to form synergies where the trade-off between control and flexibility benefits, rather than impairs performance. Different tasks present different problems which require different solutions and hence different trade-offs; the human action systems solutions are, in fact, task-specific.So it's not just methodologically useful, but I think it makes us think correctly about what the HAS is actually up to - it's not merely a stance.

This solution also enables us to keep studying systems, and not just isolated components. A simple example demonstrates why this is important. When the force-velocity relation of muscles was studied using cadaver tissue, they appeared to not be capable of producing the forces typically seen in common tasks. These apparent limits aren't actually a problem, because when the muscles are actually used by an intact actor they are used as part of a system which changes what the limits are. In the case of force production, muscles can slowly store energy in tendons, energy which can then be explosively released or used to amplify force production in a task. Studying the components in isolation does provide some information; however, the real question is how these components work together to produce the behaviour we see.

The properties of task-specific devices

The goal of the HAS is to temporarily assemble a small subset of the available dynamic resources (both inherent and incidental) into a low dimensional device that is controllable and solves the task at hand. These are task-specific devices, and Bingham then discusses eight properties of these TSDs. 

1. A task-specific device is 'task specific'
Forming a TSD entails a reduction in the complexity of the HAS, and this reduction must be driven by the functional demands of a specific task. Research questions here include how are the various resources coupled, and, most importantly, how do task constraints have their effect? How do we come to select one rather than another TSD? Because this is a perception-action approach, we are constrained in the kinds of acceptable answers, which must include information but won't allow, say. rampant speculation about mental representations which magically do all the work. Some more constraints come from the properties listed below.

2. A task-specific device is 'smart'
TSDs should be locally optimal solutions to the task at hand; the end result should be an efficient use of the resource dynamics. This is related to Runeson's use of the term 'smart' and his example of the polar planimeter; in order to be optimal you have to be able to take advantage of local shortcuts, features of the task which allow locally robust solutions (at the cost of not being a general problem solving device anymore). Smartness and task-specificity are very tightly coupled ideas. Smartness poses complicated research problems: in order to evaluate optimality, we need an understanding of the alternatives, and we begin to head back to needing a description of the full resource dynamics of the HAS.

3. A task-specific device is 'deterministic'
A system is deterministic if you can predict it's future state based on it's current state and the upcoming control and perturbing inputs (i.e. what it's about to do and why). One advantage of such a system is in terms of controlling it: a deterministic system can be controlled intermittently, which helps when the system is moving faster than, say, the delays in the nervous system. You can control such a system 'online' but by simply setting it running correctly and tweaking it only occasionally. This kind of control is stable and economical, as are the movements it produces. Research questions here revolve around the question, what kind of deterministic processes can the HAS assemble?

4. A task-specific device is 'softly assembled'
To be 'softly assembled' is to be built out of whatever resources are available to solve the task at hand, resources which can be removed and used again later as part of a different TSD. This is in contrast to a device which is built to do one than and can only ever do that thing. The HAS has to move from TSD to TSD over time as demands change, and the same resources from one TSD can often be used in the second, albeit perhaps in a different role. Take a crude example: I can use my hand to pat someone on the head or punch that head as hard as I can. The resource (the hand) is the same, but it has been placed in a different relation to the other elements of the task (e.g. the head).

Assembly is critical: you have to be able to build your TSDs as required. But softness is also critical: you must be able to re-purpose your resources as required. Research questions here centre around the assembly process; how are the components placed in the appropriate relation to solve a task, and how are they then dis- and re-assembled to solve the next one?

5. A task-specific device is 'controlled'
However the resource dynamics are softly assembled, this assembly must be sufficiently stable to be controlled. Specifically, it has to resist small perturbations and not respond to them wildly and disproportionately (like a butterfly effect). There are numerous kinds of stability described by dynamical systems, usually described as some kind of attractor. The form of this attractor provides clues to the dynamics of the system, e.g. limit cycle stability entails a nonlinear dynamical system. Research questions involve using perturbation methods to uncover the stability of the TSD in question.

6. A task-specific device is 'scaled'
One of the key features of a given task is the scale at which it is occurring; is it very fast? Does it involve a lot of force to be produced? Scale is the issue of setting the correct parameters to a movement - e.g. not just 'moving to pick something up' but 'moving at a particular, appropriate speed'. Questions here ask how this scaling is achieved, and whether two similar movements at a different scale are simply scaled versions of one another, or different types of task entirely?

7. A task-specific device is 'assembled over the properties of both the organism and the environment'
The TSD approach was extended and embodied before those words got trendy, and had a very clear rationale for being so. Bingham drew a distinction between inherent and incidental dynamics, for convenience. But they are of the same type, and are accessed and used in precisely the same way; the system makes no a priori distinction between these two dynamics, and essentially works at the level of resource dynamics.

The critical questions here are about perception, and introduce the idea of the perceptual bottleneck. The world is described dynamically; in terms of time, position and it's derivatives, and mass. The human perceptual system (HPS) is only able to contact these properties via kinematics (the above, minus mass). This is the bottleneck: the necessity of finding a unique mapping from dynamics to kinematics. The study of perception requires us to find these mappings and establish whether they are being used.

8. A task-specific device is 'potentially modifiable to a new purpose'
Task specific devices do overlap, to the extent that the task demands overlap. Hammering and punching both require you to achieve peak force at the point where you intercept the surface in question, and thus the composition and organisation of these TSDs might be expected to overlap somewhat. Coaching often describes doing a sports action in terms of doing something different but more familiar to the athlete, with the goal of accessing the overlap to the benefit of the sports action. Research questions can then be about transfer of learning, and the informational basis for this transfer.

The perceptual bottleneck

Every single one of these properties entails perception. In effect, this scheme clearly lays out the justification for the claim that there isn't a perceptual system and an action system, but that there can only be a perception-action system. In this scheme, then, perception must inform the organism about the behaviour of resource dynamics and this information must be preserved over two mappings. The first is from the dynamical world to the kinematic energy array (e.g. the optic array) and this presents us with the identification problem. The second is from this array to scaled information for the control of action, and this presents us with the sclaing problem. This is essentially the Turvey-Shaw-Mace scheme for information that Chemero described when discussing the symmetry principle.

1. The identification problem
The dynamical world must be specified in kinematic arrays. How is this possible? Mathematically, kinematics can provide a specific solution to the differential equation describing the dynamical event. Roughly speaking, kinematics can specify dynamics but only over a limited scope. This will hopefully be familiar territory; Turvey et al (1981) laid out the idea that the ecological laws governing perception have scope, and this is simply another way of describing why this matters. Then, usefully, detailing the scope over which an information variable can inform about the dynamics of an event lands us right back into TSD territory - so part of the TSD analysis is solving the information question.

2. The scaling problem
The second mapping is from the arrays, which have no intrinsic units associated with them, to the organism, who needs the information from the array to be calibrated, i.e. to be perceived on some (action relevant) scale. This topic came up last time with the question of how distances outside reach space are perceived; the exact solution is unclear, but whatever the units are (eye height, effort) they must be action-relevant and this calibration is part of the act of perception, not a cognitive process after the fact (Witt et al, 2010).

Bingham mentions body scale solutions (e.g. eye height) but notes these fail because the relationship between this body scale and the scale of the event in the world would then also have to be determined. As I mentioned about eye height, while it is available it is not immediately obvious what it means with respect to, say, your ability to walk a given distance. So the solution to the scaling problem in perception, suggests Bingham, must rely on scaling relations that the event and the organism share. His example is gravity: the force affecting a ball is the same force affecting the hand reaching to catch the ball, and the correct scale to perceive the information on can (in principle) be derived from this common feature. This is, I think, a detailed way of describing the problem with body scaling that myself and Chemero have identified, namely the fact that body scale is really only occasionally a proxy for the real scale, 'capacity to perform the task'.

Summary
So, there's a lot here. It turns out studying perception-action systems properly is hard, and time-consuming, and requires more than just measuring some stuff. But this approach provides a rigorous and useful framework for actually getting some science done, and is the framework I'm trying to make more explicit in my own work these days; it's always been there (I was trained by Bingham, after all) but it's time to tease some of this out for the more recent literature.

____

*Nikolai Bernstein (1896-1966) was a Russian physiologist who was the first person studying motor control to really confront the degrees of freedom problem head on. His work was largely unknown in the West until his book, The Coordination and Regulation of Movements was published in English in 1967. His insights form the basis for the 'action' part of the perception-action approach, in the way Gibson's are the basis of the perceptual side of things.

References  

Bernstein, N.A. (1967). The co-ordination and regulation of movements. Oxford: Pergamon Press.

Bingham, G. (1988). Task-specific devices and the perceptual bottleneck. Human Movement Science, 7 (2-4), 225-264 DOI: 10.1016/0167-9457(88)90013-9  Download

Chemero, A. (2009). Radical Embodied Cognitive Science. Cambridge, MA: MIT Press. Amazon.com, Amazon.co.uk, MIT Press e-book


Turvey, M. T., Shaw, R. E., Reed, E. S., Mace W. M. (1981). Ecological laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981) Cognition, 9 (3), 237-304. DOI

Witt, J. K., Proffitt, D. R., & Epstein, W. (2010). When and how are spatial perceptions scaled? Journal of Experimental Psychology: Human Perception and Performance, 36(5), 1153-60. Download 

0971370d80183a9c9f12565d311f01f9