Lecture 25: The Mechanical Basis for "Getting About Among Things" (Turvey, 2019, Lectures on Perception)

This chapter is about the mechanical consequences of interacting with the environment; what happens when we come into physical contact with things. Importantly, though, we are going to focus on the informational consequences - what information is created by mechanical interactions, what does it enable us to perceive, and what is the medium for this information? Broadly then, the topic of this literature is dynamic/effortful touching. 

Talking about this topic will enable us to make a few key points. First, dynamic touch is a commonplace activity, namely the kind of activity an ecological psychology takes as it's main topics. Second, it engages with the ecological hypothesis that information is specific to the dynamics of what it is about, and not specific to the medium in which it lives. Visual information and haptic information, for example, are the same kind of thing, and are sometimes even the same information variable. The important thing for a theory of perception is the information, and not the details of the medium, nor the anatomy that happens to pick it up. 

Turvey then spends most of the chapter reviewing key empirical findings from the ecologically motivated investigation of dynamic touch. The key perceptual variables are not things like mass or weight, for example, but the moments of inertia; how an object resists changing state. This leads to tasks that involve hefting or wielding (changing the state, essentially), and experimental manipulations that separate out things like mass from mass distribution and the inertia tensor. 

Several things emerge. People can perceptually identify different properties via similar movements, by attending to different properties of the object. This demonstrates that intentionality matters (you can ask people to judge different things and that drives attention to different variables), and that in any task there are multiple variables available, defined across various scales and relations between properties (the task is multi-fractal). These mechanical properties are also the basis of perceiving things about ourselves, as well as what objects we are mechanically interacting with; we perceives things about our limbs this way as well as things our limbs are dynamically touching. There's no in-kind difference between perceiving ourselves and other things, it's all grounded in informationL. Finally he notes the case of weight perception, and how it is not based on the weight of objects, but on their inertia (this connects to Gibson's point, that the variables of perception must be empirically discovered and confirmed, not assumed from one description of the task at hand). This connects to things like the size-weight illusion and reveal it's only an illusion given the assumption of the wrong variables at play. 

Turvey then discusses a fascinating case, a man called Ian Waterman who lost all proprioception and had to effortfully replace that perception of the environment with vision. The details of his ability to do so and the limits on what he could do are a fascinating story for ecological psychologists (some clips from a documentary here, and there's a great book about it too). 

Turvey briefly reviews his hypothesis that the medium for all this mechanical interaction perception is the body considered as a tensegrity structure (Turvey & Fonseca, 2014). The hypothesis is that the way the body deforms under mechanical pressure provides a medium for the same kinds of structure as the optic array does; it has the right kind of features to support implementing informationL. Analogous to the optic array, the key is the right level of description - the optic array begins with optical solid angles, for example, rather than points or rays of light. 

Turvey then notes briefly that hearing is also about the detection of mechanical events, just not necessarily ones in which our body was involved. He points to the work of Gaver (1993a, b) on the kinds of events that can be specified in the acoustic array, and how we can 'hear what a struck object looks like' because of the nature of the informationL involved. 

Where is the Haptic Information? (A Purple Peril)

Haptics (or proprioception) is the sensory modality built into our bodies; it's provides constant information about the state of the body and things it is in mechanical contact with, such as tools. Many ecological psychologists (myself included) have investigated haptic perception and it's role in the control of action, but unlike the optic array, we have basically zero work identifying what the relevant information variables look like. 

I first investigated haptic perception in the context of coordinated rhythmic movements (Wilson, Bingham & Craig, 2003). Geoff had run studies showing that visual judgements of different relative phase varied in stability in the same way that the production of those relative phases does. This suggested that the movement phenomena were being caused by the way relative phase is perceived. This was vision, however, and the movement phenomena obviously involve motion of the body and the haptic system. This involvement was typically explained in terms of muscle homology and neural crosstalk effects. Our study had people track manipulanda that moved up and down one of three mean relative phases with various levels of phase variability added, and had them make judgements of that variability (replicating the visual studies). We found haptic perception of relative phase, as measured by those judgements, behaved just like visual perception of relative phase - we inferred that the information, the relative direction of motion, can be detected by both systems and has the same effects. 

I am moving back into the haptic information world for two related reasons. 

First, I want to replace the muscle homology/neural crosstalk stories with a haptic perception story. The effects these theories account for are very large and reliable, and Geoff's perception-action model currently only applies to visual information. Specifically, muscle homology applies to relative phase defined in an egocentric (body centred) frame of reference, while Geoff's model applies to relative phase defined in an allocentric (external) frame of reference. Relative phase is clearly detected in both frames of references; when they are pitted against one another experimentally, both matter and the egocentric effects dominate (e.g. Pickavance, Azmoodah & Wilson, 2018).

Second, I have become interested in individual variation in the variables used to perceive relative phase. Based on his data, Geoff's model predicts relative phase is perceived via the information variable relative direction of motion, the detection of which is modified by the relative speed of the oscillators. In Wilson & Bingham (2008; blog post), we showed this was true in 7 out of 10 untrained participants judging 0° and 180°. The other three became unable to judge these phases when we perturbed another candidate variable, relative position. This experiment also showed that people trained to perceive 90° had improved because they had switched to this variable, but we were not expecting people at the other relative phases to be using this variable. I'm finally getting back into experiments probing the prevalance of this individual difference in visual information use and the consequences for perception-action stability (briefly: there's a lot of variation and it matters!). As part of the above project, I want to do the same kinds of studies on haptic perception too. 

My problem here is, there is essentially no information in the literature on the nature of haptic information variables. This Peril lays out my current hypothesis about where to look; please, dear God, come help me!

A Test of Direct Learning (Michaels et al, 2008)

Direct learning (Jacobs & Michaels, 2007) is an ecological hypothesis about the process of perceptual learning. I describe the theory here, and evaluate it here. One of the current weaknesses is little direct empirical support; the 2007 paper only reanalysed earlier studies from the new perspective. Michaels et al (2008) followed up with a specific test of the theory in the context of dynamic touch. The study was designed to provide data that could be plotted in an information space, which provides some qualitative hypotheses about how learning should proceed.

There are some minor devils in the detail; but overall this paper is a nice concrete tutorial on how to develop information spaces, how to test them empirically and how to evaluate the results that come out. The overall process will benefit from committing more fully to a mechanistic, real-parts criterion but otherwise shows real promise.  

Affordances are Not Relations, Part 1: Chemero (2009)

Affordances are on my mind right now as I develop the throwing research programme, and a major commitment of that work is that affordances are (dispositional) properties of the environment picked out by organisms in the context of tasks. This commitment has become important enough that it's time to get into developing specific arguments against the various 'affordances are relations' papers that are out there. I am working towards a paper summarising my objections to the relations account that also strongly advocates for the properties account on the grounds it enables a lot more science. This will be an occasional series of posts as I read and draft my arguments; as always, feedback welcome.

In this first post, I want to draft a response to 'Affordances 2.0', from Chemero's (2009) book Radical Embodied Cognitive Science. I previously blogged this chapter in two parts here and here. 

How do we perceive which objects afford throwing the farthest?

Previous work has established that people with throwing experience can perceive the affordance of 'throwability'. If you let these people heft objects with a range of sizes and weights, they will confidently select the one they think they can throw the farthest, and they tend to be correct. It's a very natural task, one you have probably done yourself on a beach or lakeside looking for stones to throw into the water. 

This is only the first, and relatively easy step in any ecological task analysis. Once you've identified an affordance property and established that people are sensitive to it, you need to identify the information supporting this perception. For throwing, this has not been done, and while the paper I'm reviewing here doesn't solve the problem, it does rule out a highly likely contender for the source of the information that has implications for a lot of other research.

Cracking the Tough Nut of Chimp Tool Use

A paper just out in PLoSOne reports that chimpanzees, given some experience and enough of a weight difference, prefer to use heavier hammer stones when cracking hard nuts. This is apparently quite exciting: this is the first study to isolate weight as a property relevant to the task of cracking open a nut.

This caught my attention because weight is not actually the only key property that determines nut cracking success. A heavy hammer is great, but it will eventually become too heavy to lift, and for a given size stone there may very well be an optimum weight (similar to how people choose very specific combinations of size and weight when asked to throw objects to a maximum distance; Zhu & Bingham, 2011). In the current experiment, when the only difference between the objects was weight, the chimpanzees often went to the heaviest stone because it took the fewest strikes and least time to crack the nut. But is this just an artefact of the current experiment? And if so, can an ecological approach find ways to find out just how chimps choose their tools?

Individual Variation in the Use of Perceptual Information (Specification IV)

If it is the case that perception requires the use of specifying variables, then there should be no individual variation in what information variables people use. However, as we've already seen, such variation exists: the dynamics of a collision event produces multiple kinematic patterns in the optic array, and people judging the mass ratio of colliding balls use all of these, only one of which actually specifies which ball is heavier. Even with training, people do not always find the specifying variable. 

This is an example of how the mapping between a property in the world (mass ratio) and the optic array can be one-to-many, with consequences for perception. Figure 1 on this post shows us that there is another mapping to investigate, namely the one from a perceptual array to the organism. Can this mapping also be one-to-many?  Withagen & van Wermeskerken (2009) suggest that it can, and that again training does not necessarily help.

Chemero (2009) Chapter 7: Affordances, etc (Pt 2)

Last time I went over affordances-as-dispositions, and Chemero's first swing at affordances-as-relations. Affordances can't be dispositions, claims Chemero, because

1. Dispositions manifest when the conditions are met; this is compulsory. But I am not currently trying to effect all the affordances in my vicinity, so they can't be dispositions. Relations are functions, and thus support malfunctions.
2. Dispositions require complements - for perception-action, the complement of an affordance is an effectivity. But what exactly is this? Body scale (e.g. leg length)? Actually, it's more likely in terms of ability (per some unpublished experiments Chemero has run); people's judgements of stair climbability are a relation between the riser height and the person's ability to step that high.
3. If affordances are properties that are directly perceived, then when two people perceive the same affordance their minds will overlap: the problem of two minds. Relations solve this problem by making the overall relation which the directly perceived affordance is part of unique to each observer.

This would be all well and good, except that 

1. Affordances and effectivities are complex dispositions, and the conditions for being realised can be a long list. In addition, I can only be one kind of effecting device at a time, so when seated I am literally not capable of complementing the climbing affordances of my stairs at that moment in time.
2. Noting that 'body scale' is an imperfect proxy for an effectivity, and then claiming that this means nothing is an effectivity makes no sense. In addition, 'abilities' are equally approximate. The issue (being careful what you claim is the actual complement of the affordance) is valid but applies equally to dispositions or relations.
3. The solution to the problem of two minds that Heft outlined and Chemero thinks supports his case lies in making the act of perception relational, not the thing perceived. The affordance does not, itself, need to be a relation.

So far, nothing has convinced me that affordances need to be relational. But to round the story out, I want to finish the chapter and address the final tweak Chemero adds: Affordances 2.0.

Tools and Brains and Embodied Cognition

Arguing about embodiment with Ken Aizawa over the last few days has opened up a lot of topics that I hope to cover over the next little while. But it also primed me to notice this article at Scientific American by Patrick Haggard and Matthew Longo, summarising a recent paper adding to the growing literature on the neuroscience of tool use. I like this work, and it got me thinking how this relates to the embodied cognition literature; Ken, I'd be interested to hear your thoughts on this on.