The Theory of Affordances (Gibson, 1979, Chapter 8)

The most influential part of this book is the theory of affordances. We still argue a lot about what these are and how they work, because they are slightly weird (at least from the traditional point of view). So it was interesting to go back and read Gibson's original approach. I was struck by how much sense it makes in the context of the previous chapters; the book really builds and develops and this chapter should be discussed in that context I think.

Environments are the surfaces that separate substances from the medium. But these configurations of surfaces aren't simply there; they offer possibilities for action. They afford things to the animal. Gibson asks 

How do we go from surfaces to affordances? And if there is information in light for the perception of surfaces, is there information for the perception of what they afford? Perhaps the composition and layout of surfaces constitute what they afford. If so, to perceive them is to perceive what they afford. This is a radical hypothesis...

pg 119

In this chapter, Gibson does not offer a single straight-forward definition; that's not really how this works. Instead he lays out examples, and uses those to constrain the concept. As usual, ecological categories have fuzzy boundaries and this is ok!

Lecture 22: Ontology at the Ecological Scale (Turvey, 2019, Lectures on Perception)

After a book length set of preliminaries, we are finally getting to the meat of the positive case the ecological approach has to make. That case of course sits on top of a rejection of the Cartesian ontology that has been driving everything else; this chapter describe the first part of the ecological replacement, specifically affordances. 

Effectivities for Reach-to-Grasp Actions

I just reviewed the affordance properties that produce the spatial structure in reach-to-grasp actions, and there's an unquestioned assumption lurking in that analysis. Luckily, Mon-Williams & Bingham (2011) actually did question it, so I wanted to cover that part of the paper here.

The assumption in the analyses I described last time is that the unit of action is the aperture between the finger and thumb, and not the fingers themselves. Bingham refers to this as an opposition vector (Iberall, Bingham, & Arbib, 1986; van Bergen et al, 2007). In some ways, this is a weird idea; the action system working to control a space between limbs, and not the limbs! Smeets & Brenner (1999) proposed that grasping is actually about the two limbs. Mon-Williams & Bingham tested these hypotheses and found evidence in favour of the opposition vector.

I want to walk through this in a little detail, though, as of course identifying the relevant elemental variables is part of an UCM analysis, and affordance research helps here too. The task analysis that reveals affordance property options also points to effectivity property options (at least it should - these are complementary after all!). But another part of the UCM approach is that it can, in principle, test hypotheses about elemental and performance variables, so I want to lay this out as well.

The Task Dynamics of Reaching-to-Grasp

In the last post, I reviewed the basic form of the reach-to-grasp task and the basic spatial and temporal structure of the resulting reach-to-grasp action. I'm shortly going to review three papers by Bingham about where all this structure comes from, but first I wanted to sketch out the task analysis those papers will rely on. 

The question at hand is, in the context of reaching-to-grasp an object, what are the relevant object affordances? What follows is derived from Mon-Williams & Bingham (2011), which I will review fully in the next post. I've tried to fully flesh it out, though, to be as complete as possible. The goal is to lay out the likely relevant task dynamics; this leads to specific predictions about which manipulations should affect which parts of the reach-to-grasp action.

Motor Abundance & the Affordances for Reaching-to-Grasp

Movements are never the same twice, even when you are trying to do that same thing over and over. Variability is an inescapable fact of trying to organise and run a complex system such as a human body. But there is more than one source of variability in movement; there's noise, and then there's redundancy, and these are not the same thing. 

Our movement systems are redundant; specifically, they always have more degrees of freedom available than are ever required to perform a given task. This means that there is always more than one way to perform any given task, and this can range from slight variations to complete reorganisations. 

Redundancy is a feature, not a bug. It means that we can reliably achieve a task goal in the face of perturbations that range from trial-to-trial fluctuations in execution up to surprises like tripping or the sudden appearance of an obstacle. However, it poses two related control problems. First, a problem of action selection: given that there are many functional organisations of degrees of freedom that could solve that task, which do we choose, and why? Second, a problem of action control: once we have our degrees of freedom organised, we still have some left over that need to be actively controlled; how do we do this, and why do we control them the way we do?

Trip Report from the Uncontrolled Manifold

I've spent the past few months getting a new paper to the 'complete first draft' stage (you can find a copy here in the meantime; it's still got some work to do though). It's about affordances, using targeted long-distance throwing as the task, and it's my first dip into the world of the uncontrolled manifold. I collected this data over five years ago, and it's been deeply satisfying to actually use it after all this time.

Part of what's taken so long is that I've had to learn the details of the uncontrolled manifold analysis. I blogged some about it here and here but this was the year I finally had the time and data to actually get into the maths. I still really like it as an approach to analysing human movement, but learning the details and trying to figure out how to get affordances into it has raised a lot of interesting questions about how it gets used right now and what this all implies for how we think movement is controlled. I'm raising a bunch of these issues in the paper but I wanted to sketch some out here for comment.

Broadly (and this shouldn't be a surprise to anyone really) I've realised that UCM is only a method, not a theory, and it's therefore not able to serve as a 'guide to discovery' about movement control. However, it's being used as if it can, and to be honest I was quite shocked at how carelessly it's being used in the literature. 

Is Direct Perception Plausible? Ecological Information

We're in the home stretch of working through how direct perception is, at least, an option, and how the ecological approach in particular attempts to make it work. We've talked about what direct vs indirect means, the kinds of properties direct perception needs to be out there in order to work, and affordances/effectivities-as-dispositions as the specific properties ecological psychology claims are out there and fit the bill. 

I ended that last post by highlighting ecological psychology had one last thing to do in order to be plausible, and that is to have a way to bring affordances and effectivities together into a kind of contact that allows them to work together. That contact can't be mechanical, or simply physical proximity, because almost all of the things we perceive and act with respect to are not in that kind of contact. The ecological solution is informational contact, and so this post will build on the pieces I've assembled to identify what kind of thing ecological information has to be in order to work. 

Is Direct Perception Plausible? The Case for Affordances-as-Dispositions

The first post in this series laid out the rules for what makes a theory of perception direct or indirect. In order to avoid having to require organisms to somehow figure out behaviourally relevant properties (indirect perception), direct perception requires that such properties are out there already, ready and able to be detected. Such properties are a bit weird - by definition, they must be properties of objects that include some reference to the organism doing the detecting (specifically, they are impredicative properties). 

The second post laid out some reasons to think that such circularly defined properties are legitimate options for the physical world. Properties like this are required by the mathematics of the quantum scale; and so, while a bit weird, are possible. Organisms don't work with quantum properties, though, so if they only happen at that scale, this doesn't help a theory of direct perception. The key to making impredicative properties work is measurement. At the quantum scale, all properties are uncertain until measurement collapses the uncertainty of the thing measured. At the ecological scale, properties aren't uncertain in this way, but which properties are 'primary' still depends on the measurement device; to a polar planimeter, for example, the higher order property 'area' is simple and lower order properties such as 'length' must be figured out. The reverse is true of a ruler. 

We have a set of pieces now. Impredicativity requires measurement to affect which properties are immediately available to the measurer, this is possible at the ecological scale, and with the right measurement device higher-order properties can be immediately available. This establishes an in-principle case in favour of direct perception as an option. Next, if we can identify the higher-order form of behaviourally relevant properties, we will identify what kind of measurement device is required for these to be directly available. 

The main ecological hypothesis is that higher-order behaviourally relevant properties, which we call affordances, take the form of dispositional properties. This post will walk through how this works and how it fulfils what's required to support direct perception.

(As many of you know, not everyone is happy with the dispositional ontology, and propose instead that affordances are relations. I am not going to lay out the parallel case for how this ontology fulfils the requirements I have been laying out, for the simple reason that I do not think it can do any such thing and I have yet to see any successful attempt to make it work. I talk about this in this paper, currently still living in review limbo.)

Are Affordances Plausible? Updating Some Intuitions

In the previous post, I laid out the basic distinction between direct and indirect theories of perception. The basic issue is how to get to behaviourally relevant properties. Such properties of the world have to be partly about the organism, and not just the world; not just 'mass', which is something about the world, but 'moveability' which is something about the world, relative to the organism. If there are properties like these out in the world, and we can perceive them, then perception can, in principle, be direct. If there aren't properties like these there to be perceived, we would have to invent them, and perception would have to be indirect. The ecological hypothesis has to begin by finding behaviourally relevant properties out in the world, ready to be perceived without having to be invented; we need to find affordances. 

The problem is that this seems to be madness, right off the bat. It seems bizarre to think that things separate from the organism could possibly have properties that have anything to do with the organism. Why would they have such things? Surely things can only have properties that are about themselves and make them what they are? 

This post will talk about some pieces of information that make this at least plausible; the next post will talk about the specific ecological hypothesis about affordances that tries to implement that plausibility. To get to that plausibility, I am going to have to update your intuitions about what is simple, and what sorts of things can be physically real, and along the way introduce some useful vocabulary I will do everything I can to use clearly. 

Lecture 4: Simulative, Projective, and Locality Assumptions (Turvey, 2019, Lectures on Perception)

This lecture is a brief history of the common assumptions made in theories of perception about how things 'over there' can cause us to have a given perceptual experience. The simulative and projective elements can be quickly dealt with; the big claim in this lecture is that the right notion of causation for perception is non-local, as it is in quantum mechanics. (Note: Turvey is not saying perception is a quantum process. He's just going to use it as a framing to explain what non-local causation is, and he will rely on the rigorous empirical testing it has passed in physics to say it is a viable notion of causation for a physical system.)

You Cannot Perceive a Relational Affordance (A Purple Peril)

One of the more enduring arguments in ecological psychology is about the best way to formally describe affordances. The two basic approaches are that they are dispositions (Turvey, Scarantino, me) or that they are relations (Reitveld, Kiverstein, Chemero). The argument has mostly settled down into just agreeing to disagree, but I am still convinced that the relational analysis is critically flawed and I want to try and either get them to solve the problem or end the debate once and for all. I've reviewed this in a bunch of places (e.g. here, here, and here)  but this post is just setting out my challenge once and for all; you cannot perceive a relational affordance, and there is as yet no good story about how to learn new affordances.

My problem stems from this Gibson (1979) quote (we all have our favourite, but this one seems to cut to the heart of it)

The central question for the theory of affordances is not whether they exist and are real but whether information is available in ambient light for perceiving them.

Right now, the affordances-are-relations camp have no story for how these can structure light (or other energy media) and therefore create information about themselves. They are therefore, as currently formulated, not even in principle perceptible. This means affordances-as-relations is of zero use to the ecological approach. 

Bruinberg et al (2018) tried to address this problem, but as I blogged here their solution is not ecological information and it reveals that these authors do not as yet understand what information actually is. My challenge is therefore this: tell me a story in which affordances-as-relations are able to create ecological information in energy arrays, and might therefore be learned, and the debate will be back on. Until then, affordances-as-dispositions is the only account that formalises the right properties and the debate is over. 

Affordance Maps and the Geometry of Solution Spaces

I study throwing for two basic reasons. One, it is intrinsically fascinating and I want to know how it works. Second, it's become a rich domain in which to study affordances, and it is really forcing me to engage in great detail with the specifics of what these are.

My approach to affordances is that they are dynamical properties of tasks, which means that in order to study them, I need to be able to characterise my task dynamics in great detail. I developed an analysis (Wilson et al, 2016) to do this, and I also have a hunch this analysis will fit perfectly with the motor abundance analyses like UCM (Wilson, Zhu & Bingham, in press). I have recently discovered that another research group (led by Dagmar Sternad) has been doing this whole package for a few years, which is exciting news. Here I just want to briefly summarise the analysis and what the future might hold for this work.

General Ecological Information Does Not Support the Perception of Anything

One common critique of the ecological approach is how can we use perception to explain behaviour that is organised with respect to things in the world that aren't currently in our area? How do we plan for future activities, or how do we know that the closed fridge has beer? 

A recent attempt to get ecological about this comes from Reitveld & Kiverstein (2014) who propose a relational account of affordances that enables them to talk about opportunities for more complex behaviours. This account has developed into the Skilled Intentionality Framework (e.g. Bruineberg & Rietveld, 2014), where skill is an 'optimal grip' on a field of task-relevant, relational affordances. 

I have always had one primary problem with this programme of work - I don't believe that they can show how these affordances create information and thus can be perceived. I discuss this here and here, and there's comments and replies for Rietveld and Kiverstein there too. You can indeed carve the world up into their kind of entities, but if they don't create information then they cannot be perceived and they are irrelevant to behaviour. 

I was therefore excited to see a new paper from the group called 'General ecological information supports engagement with affordances for ‘higher’ cognition' (Bruineberg, Chemero & Rietveld, 2018; hence BC&R). There is a lot of excellent work in here; but their proposal for a general ecological information is, in fact, neither ecological nor information. It is a good way of talking ecologically about conventional constraints on behaviour, but it doesn't make those perceivable and so the main thesis of the paper fails. 

What Limits the Accuracy of Human Throwing?

Throwing a projectile in order to hit a target requires you to produce one lot of the set of release parameters that result in a hit; release angle, velocity (speed and direction) and height (relative to the target). My paper last year on the affordances of targets quantified these sets using a task dynamical analysis.

There is one additional constraint; these release parameters have to occur during a very short launch window. This window is the part of the hand's trajectory during which the ball must be released in order to intercept the target. It is very easy to release slightly too late (for example) and drill the projectile into the ground.

How large is this launch window? It is surprisingly, terrifyingly small; Calvin (1983) and Chowdhary & Challis (1999) have suggested it is on the order of 1ms. Those papers used a sensitivity analysis on simulated trajectories to show that accuracy is extremely sensitive to timing errors and this millisecond level precision is required to produce an accurate throw.

Smeets, Frens & Brenner (2002) tested this hypothesis with dart throwing. If this intense pressure on timing the launch window determines accuracy, then throwers should organise their behaviour and throw in a way that makes their launch window as tolerant of errors as possible. They replicated the sensitivity analyses on human data to see if people try to give themselves the maximum error tolerance in the launch, or whether they were trying to accommodate errors in other variables.

What they found is that the launch window timing is not the limiting factor. Their throwers (who were not especially expert) did not throw so as to minimise the sensitivity of the launch window timing to errors. Quite the contrary; they lived in a fairly sensitive region of the space, and then didn't make timing errors. They did throw so as to reduce the sensitivity to speed errors, however, and errors in the targeting came from errors in the spatial path of the hand that the system did not adequately compensate for, rather than the timing of the hand's release. (The authors saw some evidence that the position, speed and direction of the hand trajectory were organised into a synergy, which aligns nicely with the motor abundance hypothesis).

I would like to replicate and extend this analysis process using more detailed simulations and data from better throwers. I've become convinced it's a very useful way to think of what is happening during the throw. I also think these results point to some interesting things about throwing. Specifically, while timing and speed must both be produced with great accuracy, the system has developed two distinct solutions to coping with errors. Timing errors are reduced by evolving neural systems that can reliably produce the required precision. Speed errors have been left to an online perception-action control process which adapts the throw to suit local demands. The latter is the more robust solution; so why was timing solved with brain power?

Expectations and the Size-Weight Illusion

The size-weight illusion (SWI) occurs when people are asked to judge the weights of two different sized but identically weighted objects. The smaller object is judged to be heavier. There are a variety of explanations for this illusion (see Buckingham, 2014 for a review). I'm going to be reviewing some papers on it as I develop some experiments connected to my throwing research.

One set of explanations is 'bottom up', i.e. perceptual. Amazeen & Turvey, 1996 suggested that people do not perceive weight but inertia (this is the dynamic touch hypothesis about the inertia tensor) and Zhu & Bingham (2011) have proposed the illusion is not the misperception of weight but the correct perception of throwability (I obviously quite like this one, and have discussed it here). Interestingly Zhu et al (2013) have since shown that the inertia tensor does not explain the throwing related SWI!

The second set of explanations is 'top down'. The basic hypothesis is that the sensorimotor system expects larger things to weigh more than smaller things, within a class of 'things'. This expectation has been learned over time via experience of the real world in which this is basically true. Large mugs weight more than small mugs, even if large mugs weigh less than small anvils.

There are two interesting papers that have looked at the top-down hypothesis.

Affordances are Not Relations, Part 1: Chemero (2009)

Affordances are on my mind right now as I develop the throwing research programme, and a major commitment of that work is that affordances are (dispositional) properties of the environment picked out by organisms in the context of tasks. This commitment has become important enough that it's time to get into developing specific arguments against the various 'affordances are relations' papers that are out there. I am working towards a paper summarising my objections to the relations account that also strongly advocates for the properties account on the grounds it enables a lot more science. This will be an occasional series of posts as I read and draft my arguments; as always, feedback welcome.

In this first post, I want to draft a response to 'Affordances 2.0', from Chemero's (2009) book Radical Embodied Cognitive Science. I previously blogged this chapter in two parts here and here. 

The Affordances of Natural vs Designed Environments (A Purple Peril)

I had a conversation with Eric Brymer (@Ericbrymer) a few weeks ago. Eric is a Reader at Leeds Beckett and is an ecologically minded psychologist interested in the effects of the natural world on mental health. We chatted affordances for a while, and despite it being a very interesting chat, I really wasn’t sure if I had anything to say about the differences between the natural and designed worlds. 

But I have not been able to stop thinking about this topic, and now I think there’s something fairly cool here.

Scarantino (2003) “Affordances Explained”

Turvey, Shaw, Reed and Mace (1981) laid out an ontology of affordances; a formal account of the kind of things they are. They described them as dispositions, properties of the world constituted by sets of anchoring properties that offered an action to an organism whose dispositions could complement the affordance. Making affordances dispositions makes them real, makes them pre-date the interaction with the organism, and accounts for their odd ‘not doing anything until interacted with’ kind of existence. I am firmly Team Affordances are Dispositions and I have yet to meet an alternative account that supports a science of affordances or even allows them to be perceived.

The literature on dispositions was somewhat limited in 1981, but in 1998 Stephan Mumford published the definitive work on what they are and how they work. I always hoped someone with the necessary philosophy chops would use this work to strengthen the foundations of affordances (I even almost talked a philosopher into doing it!) but it turns out I’m covered. Andrea Scarantino (2003) published ‘Affordances Explained’ and did much of the necessary work, and there are some very useful things in the analysis. This post is me working through this material, translating from the technical philosophy into words I can understand better.

The Field is Full, Just Not of Affordances - A Reply to Rietveld & Kiverstein

I recently posted about relational accounts of affordances and how one way to summarise my objections to them is that they cannot support mechanistic models of cognition. I came to this after reading Rietveld & Kiverstein's 'Landscape of Affordances' paper and chatting to them both at EWEP14. Eric and Julian have been kind enough to send through some detailed comments (beginning here and split over three comments due to character limits). This post is me replying to these comments as a way to get them somewhere a little more visible. I haven't gone point by point, I've just pulled out the key stuff I wanted to address; read their comments for the whole thing. I appreciate their willingness to get into this with me; their account is becoming wildly influential and their papers and feedback are helping me immensely as I work to articulate my concerns. 

To preview: my fundamental objection remains the same and as yet unanswered - while it is indeed possible to identify relations between 'forms of life' and 'socio-cultural environments' there is, as yet, no evidence that these relations create perceptual information. If they do not create information, they are not ecologically perceived, and they cannot figure in the online coordination and control of behaviour. And if they can't do that, then they sure as hell aren't affordances.

So my challenge to Reitveld & Kiverstein (R&K) is this - work up an example of an affordance that fits their definition and not mine and that creates information. Then we can test to see whether people act as if they perceive that affordance and can try perturbing the information to confirm how they are perceiving it. Then, and only then, do we have a ball game.

The Bliss of Motor Abundance

The fundamental question in psychology boils down to "why did that person do that thing they just did?". Welcome to my new favourite example.

Figure 1. I am pretty sure this move totally makes sense

I saw this on Twitter the other day and I couldn't stop watching it. What the ever-loving hell is that cat doing? Then I realised I had a hypothesis, and that it was a fun example of an ecological affordance analysis. So here we are, at last, with a cat-themed blog post. I'll review this manouver, claim that it's all about effecting affordances, and then chat a little about complex actions like this.