The Field is Full, Just Not of Affordances - A Reply to Rietveld & Kiverstein

I recently posted about relational accounts of affordances and how one way to summarise my objections to them is that they cannot support mechanistic models of cognition. I came to this after reading Rietveld & Kiverstein's 'Landscape of Affordances' paper and chatting to them both at EWEP14. Eric and Julian have been kind enough to send through some detailed comments (beginning here and split over three comments due to character limits). This post is me replying to these comments as a way to get them somewhere a little more visible. I haven't gone point by point, I've just pulled out the key stuff I wanted to address; read their comments for the whole thing. I appreciate their willingness to get into this with me; their account is becoming wildly influential and their papers and feedback are helping me immensely as I work to articulate my concerns. 

To preview: my fundamental objection remains the same and as yet unanswered - while it is indeed possible to identify relations between 'forms of life' and 'socio-cultural environments' there is, as yet, no evidence that these relations create perceptual information. If they do not create information, they are not ecologically perceived, and they cannot figure in the online coordination and control of behaviour. And if they can't do that, then they sure as hell aren't affordances.

So my challenge to Reitveld & Kiverstein (R&K) is this - work up an example of an affordance that fits their definition and not mine and that creates information. Then we can test to see whether people act as if they perceive that affordance and can try perturbing the information to confirm how they are perceiving it. Then, and only then, do we have a ball game.
Can Affordances be Relations?

Contrary to what Andrew Wilson writes in his blogpost, on our account it is not the case that affordances do not exist before they are perceived. 

This has been my main problem with relational accounts proposed by Chemero and others for doing relational affordances, and it remains an issue with their accounts. But that said:

Affordances on our definition are relations between aspects of the (socio)material environment and abilities available in a form of life.  So they are defined independently of any particular individual but not independently of the abilities available in the (socio-cultural) practices that make up the form of life. When an individual dies or is born, (almost) nothing changes to the affordances available in the practice. But when a practice changes or dies out, this does change the affordances available. Learning takes then place via a process of education of attention to the affordances that are already available in the ecological niche. This learning is typically scaffolded by more established practitioners. 

This works better; the relation is not implemented by specific things but by categories of things; affordances are a relation between organisms of a given type ("form of life") and their broad environment (physical but also social and cultural). You can indeed define such relations and they would indeed exist if the form of life and the socio-cultural-physical environment also existed.

But it's here that they walk right into my main objection, which the mechanism thinking made clear to me. You might be able to legitimately define such relations, but this does not make them real parts or processes that can take part in an actual act of perception and action. On top of this possibility, there is reason to believe such relations actually aren't such real parts because I have yet to hear an account of such a relation that can interact with energy and produce information and I have no idea what that would look like. This means that while such relations can be defined, because they cannot be perceived they cannot take part in the online coordination and control of behaviour and it makes exactly no sense to call them affordances. We should reserve that word for perceivable things; Gibson himself noted that defining potential affordances is easy, but that the central question for ecological psychology is whether there is information available for them.

Another quote from R&K emphasises the information problem they have:

Another key difference between Wilson & Golonka and us is their divide between affordances and ecological information. Our relational account of affordances defines affordances relative to a form of life/ecological niche. In the human form of life, the landscape of affordances is very rich and includes possibilities for what people have typically characterized as forms of ‘higher’ cognition, and this includes for example the affordances of language. For us perceiving is “relevant affordance related action-readiness”, there is therefore no need to restrict affordances to information that is specifying and lawlike. Instead the concept of affordances can apply to everything in the (sociomaterial) environment that people can skillfully engage with.

This tells me that R&K do not actually know what ecological information is. Even if it isn't being used as a specifying variable (Golonka, 2015) ecological information is a thing that is distinct from the thing that created it. Organisms do not interact with task dynamics, they interact with information about those task dynamics, and that information is not identical to the underlying dynamics (although it can specify). This analytic separation of world and information is utterly central to the ecological approach. Perceiving is not “relevant affordance related action-readiness” because without information, there is no point of contact to those affordances for the organism. 

Mechanistic Models
I'll jump here to this comment about mechanisms:

Finally, Andrew Wilson suggests that relational affordance-based accounts will have problems giving mechanistic explanations. While we have no problem with mechanistic styles of explanation we doubt that it makes sense to try to fit all ecological and dynamical styles of explanation into this box. Mechanistic explanation has its limitations when it comes to accounting for systems that resist compositional analysis which is often the case with self-organizing systems that exhibit non-linear causal behaviors. We therefore think it is important to seek methodologies that complement mechanistic explanations and we are certainly not afraid of causal explanations. 

This comes up a lot so I wanted to head it off here. Sabrina and I are not looking to jam ecological psychology into the mechanism literature just for fun. Sabrina found the mechanism literature and she quickly realised it was a formal framework that perfectly fit the way we talk about what the ecological approach can do. We're pursuing it because it's working really well (once you ground the mechanistic model at the level of information), and not because we think mechanisms are somehow the only thing worth doing. The mechanism cart is firmly behind the ecological horse.

We know there's objections to mechanistic modelling in cognitive science because such models do struggle with nonlinear interactions, etc (although we are a little underwhelmed by these arguments so far). Even mechanism people tend to think cognitive science won't ever make to mechanisms for these reasons. But when Sabrina presented her analysis in Warsaw to a crowd that included Bechtel and Craver, they were both really into it and see some real possibilities there. Our current view is therefore, if we have a real shot at mechanistic models, why shouldn't we? They're awesome!

Given all this, the fact that relational accounts of affordances cannot support mechanistic models while a properties account can (because only the latter creates information) is a real negative for me. R&K's 'landscape of affordances' framework just stops being any use to me; it can't generate any experiments for me, it can't explain any of my data, and it can't help me build mechanistic explanations of that data. I want to do better than this because I know that we can. 

Relations are Required to Get to 'Relevant' Affordances

Crucially, the papers by Wilson and Golonka presuppose the solution to one of the main problems in cognitive science: the origin of relevance in the concrete situation. It does so by presupposing “tasks” rather than explaining why one cares about one task rather than another in the particular situation. In other words, it is presupposed that people care about certain affordances, for example an obstacle to avoid, like a pole with a stop sign one navigates around (p243 in Golonka 2015). But note that someone in a tank would not care at all about a certain pole with a sign, so this aspect of the environment would not invite avoiding to him or her. We think what matters are relevant action possibilities: affordances for which the individual has some readiness to act, and that influence the self-organization of what we call the field of relevant affordances (Bruineberg & Rietveld, 2014). 

So this idea is actually quite important; the affordances an object offers to an organism at any given moment in time seems to depend on the what the organisms is up to and currently capable of. This underpins a lot of the momentum behind relational accounts of affordances; people like Chemero worry that if affordances are dispositions we'd be effecting them every time we came anywhere near them, even if we didn't want to. Take salt; it has no choice but to dissolve in water because as soon as the dispositions meet, they do their thing. Right now there are chairs and stairs and obstacles screaming affordances at me, and people worry that if affordances are dispositions I'd be trying to effect them all, right now, at the same time, etc. 

Luckily, the concern about dispositions is entirely unfounded. Salt will indeed immediately dissolve in the presence of water, but only so long as the relevant supporting conditions are met (e.g. the water temperature must be in a suitable range, etc). I am quite a high-dimensional device that has to softly assemble itself into something that can complement an affordance disposition, so while I can do this, my high-dimensional state at any given moment is only going to complement a small and task relevant set of affordances. 

I've been working on articulating this as clearly as I can for a while. My take right now is this:

1. Objects have lots of properties (e.g. length, mass, colour, inertia, position relative to me, position relative to the door, etc etc etc). This is a large set.
2. Some of these properties can be higher order relations (bear with me) between those simpler properties. So for example, the size-weight relation of an object can be defined and it is a third property, separate from the size and the weight.
3. A subset of this large set of properties are action relevant, specifically they have consequences for how you interact with the object. For a thrower the colour isn't going to affect your throwing dynamics much but the size-weight relation will.
4. Most if not all of these action relevant properties will create information in various combinations of optic/acoustic/haptic or other arrays, by virtue of being properties of objects that can physically interact with those arrays.
5. Only the subset of properties that are action relevant are affordances. 
6. These action relevant properties are best described formally as dispositions, in that they dispose the object to be used some ways and not other ways (see my prehistoric objects paper for an example; I didn't just show you could throw those spheroids, I showed they were highly disposed to be projectiles for humans rather than anything else).
7. Affordances are dispositional properties.
8. At a given moment in time, a perceiving-acting organism comes along and is only 'ready' to interact with some and not other affordances (stepping over vs throwing a rock). The organism picks out that Relevant Subset of the Affordances Subset by placing itself in a specific relation (bear with me) to the object which makes some but not other affordances able to be implemented. 

I think a lot of the problems here come from knowing that relational stuff is happening but not knowing where. There is relational work happening. Affordances are properties but these can be higher-order properties defined as a relation between simpler properties (e.g. the size-weight relation for optimum throwing). However, that does not make the affordance a relation. The relevant relation is the particular episode of perception and action that occurs in that moment. When I interact with an object I literally enter into a relation with it (I come at it from a particular direction, I get to a particular distance, these change in particular ways over time) and the information I have access to is structured by this (relatedly, this was also basically Gibson's solution to the 'Problem of Two Minds' according to Heft). 

 It's also worth noting that even if my account falls down at some point, the 'field of relevant affordances' as described in that paper still doesn't help. Because the affordances they are talking about are relations that create no information, there is no way for the organism to detect the set of affordances of an object, let alone the subset of relevant ones. There's also therefore no way for the organism to be the thing that defines that subset. A theory of perception needs a theory of information or else nothing else makes sense.

Summary

The central question for the theory of affordances is not whether they exist and are real but whether information is available in ambient light for perceiving them.

Gibson, 1979/2015, pg 132

At the risk of just being another guy with a convenient Gibson quote, this is the one that sticks with me, and as I proceed with my task dynamics work it is always with this in mind.  I don't see it here.

I actually quite like a lot of the R&K framework. I think they are developing a way to talk about complex and interesting things, and I think they are doing well engaging with work by Friston, Merleau-Ponty, Wittgenstein and Gibson. A lot of our work is about finding ways to talk carefully about these things and I see the value in their work. I just don't think what they are describing are affordances.

Is it just a semantics thing? No. If affordances are going to be of any use to an ecological science of psychology they had better do something useful, and if they cannot be perceived then they can't be anything to an organism and they won't let me build a science of perception and action. Until there is an account of how a field of affordances can structure an array of light, this account will remain at best a functional level description of some things we might want to talk about. That's fine, but it's not enough, and we can do better.