The Task Dynamics of Reaching-to-Grasp

In the last post, I reviewed the basic form of the reach-to-grasp task and the basic spatial and temporal structure of the resulting reach-to-grasp action. I'm shortly going to review three papers by Bingham about where all this structure comes from, but first I wanted to sketch out the task analysis those papers will rely on. 

The question at hand is, in the context of reaching-to-grasp an object, what are the relevant object affordances? What follows is derived from Mon-Williams & Bingham (2011), which I will review fully in the next post. I've tried to fully flesh it out, though, to be as complete as possible. The goal is to lay out the likely relevant task dynamics; this leads to specific predictions about which manipulations should affect which parts of the reach-to-grasp action.

Overall, reaching-to-grasp an object is a task dynamic made up of three separable other task dynamics; transport, then grasp, while avoiding collision. Each of these three dynamics pick out a different set of object properties (affordances) and organism properties (effectivities) that must be made to complement each other. When the whole set is brought together, a reach-to-grasp happens.

Reaching as Two Targeting Tasks

The first feature of a reach-to-grasp movement is that it is two targeting tasks. The hand must be moved so as to intercept a target (transport), and the fingers & thumb must be targeted to land in a specific location on the object (grasp). 

There's a few reason to think these are separable task dynamics. First, you can transport without grasping, and grasp without transporting. Second, they pick out different object properties (different affordance properties). Third, these properties affect different measurable parts of the reach-to-grasp action. Fourth, there is a distinct break in the movement; the smooth reaching movement gets the hand to the target, the wrist comes to a stop, and then the fingers land. 

To successfully transport the hand to the target, the system needs to perceive where the object is (heading in 3D space, and (with no obstacles) straight line distance, in an egocentric frame of reference). Evidence that targeting task dynamics matter include the fact that reach kinematics are scaled by distance (peak speed increases with required distance, for example). 

To successfully land the fingers and thumb and grasp,  the system needs to perceive the landing area; where it is (but that's part of targeting the whole hand) and how big it is (smaller landing area requires a more precise targeting). Evidence that these targeting dynamics matter includes movement time increasing for smaller objects (longer deceleration phase; Bootsma et al, 1994), and the existence of two distinct phases in the reach movement (hand speeds up then comes to zero, and then the fingers are landed). 

Reaching as Obstacle Avoidance

This is somewhat counter-intuitive, but part of successfully reaching-to-grasp is not initially hitting that target! If you reach to grasp an object and don't form a wide enough aperture, your fingers will hit the object and maybe knock it over, or damage it, or otherwise interfere with successfully grasping it (Rosenbaum et al, 1999). This is a distinct constraint from targeting, and different dynamical properties of the object matter. To figure these out, we need to consider the degrees of freedom in the hand's motion that can lead to collision or no collision. 

First, the size of the object clearly matters for the obstacle avoidance part, you have to be able to fit the object into the gap created by the hand. But it's not simply the width of the object! Reaching entails targeting an aperture formed by a finger and thumb, and that aperture has a length but also an orientation in 3D space (van Bergen et al, 2007). The rigid wrist-thumb-forefinger system can pitch, roll, and yaw and therefore so can the aperture.

Figure 1. Pitch, roll, yaw

A reach involves placing a finger and a thumb on either side of an object; it effectively only interacts with two dimensions at a time. For a reach-to-grasp using the vertical sides, pitch variation has no collision implications (it would matter for a grasp using the top and bottom, or if you are an airplane). 

Using the vertical sides, yaw matters, because it reduces the effective length of the grasp aperture (foreshortening) and may also move the finger or thumb to within the bounds of the object. You can eliminate the concern about yaw by forcing people to grip the object on points laid out in the fronto-parallel plane, or you can investigate it's effects by varying the landing locations allowed (that tends to get studied in the context of shape perception, e.g. Lee et al, 2008). 

For the standard lab reach-to-grasp task, the big issue is roll. If your hand is tilted, the functional width of the object is the length of the diagonal (effectively, the hypotenuse of a right-angled triangle; see Figure 2). Scaling your grip aperture to the width will be too short unless your tilt is zero. Given that reaching involves transport but also targeting for grasp constrained by the need to avoid a collision, this diagonal (which Mon-Williams & Bingham call the Maximum Object Extent, MOE) is the relevant size property. 

Roll creates a related issue about size, this time for the grasp targeting component. Roll must not be allowed to exceed the angle defined by the target landing areas, or else the fingers will miss. So it's again not object size per se that matters, but this angle; this should constrain the orientation of the hand and it defines the MOE.

To summarise: 

1. Targeted Transport of the Hand: the relevant object property is its location (distance and heading in 3D egocentric space). With no obstacles present, this distance is the straight-line distance from the hand start location to the target. You could simply pick a speed and only ever use that, but that's inefficient. What you want is to accelerate to the highest controllable speed. Control is affected by both the transport dynamics (hand inertia, and the resulting requirements to accelerate but then also decelerate the hand in the given space) and also the targeting dynamics (speed-accuracy issues). Target distance should affect the timing and magnitude of peak speed, while target size should affect the magnitude of peak speed. 
2. Targeted Grasping of the Object: this occurs after the hand has come to rest; the fingers hover momentarily with a Terminal Grip Aperture (TGA) slightly larger than the object before being placed. By this time, the finger-thumb axis is aligned with the target, so the relevant object property is the object width (and not the maximum object extent). TGA should scale with object width.
3. Collision Avoidance: a collision happens when the hand hits the target in any way other than a targeted grasp. A reaching hand has formed an aperture, which is a gap; so that gap must be big enough to go around the object safely, under the worst case scenario. The worst case scenario that still produces a successful grasp is when the hand has rolled such that the fingers land on the most extreme parts of the landing site; the effective object size becomes the hypotenuse of a triangle, with the width along one edge and the height of the landing area the other edge. Two properties of the target are relevant now; the effective size (the Maximum Object Extent) and the angle at which that MOE lies (actually twice that angle, because roll can go left or right). The Maximum Grip Aperture (MGA) should scale not with object size, but with MOE; that scaling will include a safety margin; and the orientation of the grip aperture should be constrained by the MOE angle (no variability allowed beyond that range). 

To enable them to study all these elements, Mon-Williams & Bingham developed these objects (see Figure 2)

Figure 2. Objects from Mon-Williams & Bingham (2011)

The task is to reach-to-grasp these objects by placing a finger and thumb on each of the two knobs. These can be varied in area and width, to alter the accuracy requirements and alter the maximum object extent and angle. The object itself can also vary in its width. 

We now have a range of things to measure (see the last post) and a range of properties we have reason to manipulate (from the task dynamical affordance analysis). Over the years there have been about a million experiments in this task, but in the next post I will begin by reviewing experiments by Bingham explicitly motivated by the affordance analysis.