The affordances of objects and pictures of those objects

People interested in how perception and action affect cognition have begun talking about affordances. This should be great news; the ecological approach suggests that affordances are the properties of the world that we perceive that enable us to control our actions, so if you are interested in how action can ground, say, memory or language, then discussing affordances should enable real progress.

The term 'affordance', however, is a technical term, and it refers to very particular properties of an organism's environment. There are methods for experimentally identifying exactly how these properties are composed, and there are methods for testing our perception of them. If you aren't using these methods, and if you aren't using the term correctly, then you aren't studying affordances.

A recent example of this kind of work is Pecher et al (2013; pdf). They were interested to see whether object affordances play a role in working memory for those objects. They ran two experiments which used pictures of objects with varying affordances for manipulability, and found that in each case memory for these objects was not affected by an interfering motor task, suggesting affordances are not used to structure working memory.

There is a key problem here, and that is that pictures of objects do not afford the same things as the objects themselves. Interact with a coffee cup, and a picture of a coffee cup; they provide very different opportunities for interaction. You cannot drink out of a picture of a coffee cup; you cannot hold it by the handle because it has no handle! (See Snow et al, 2011 for a demonstration of this in an fMRI context).

The fact that these things afford different things is not intrinsically a problem; the affordances of these things and the different information these different affordances produce is, after all, how we can perceive that they are different things. It is a problem, however, if you are trying to study how the affordances of coffee cups might affect your memory for coffee cups by using pictures of coffee cups. You aren't studying the affordances of coffee cups, and you shouldn't actually get to say that you are studying the affordances of coffee cups.

Studying affordances is hard. They are odd things; they are properties of the world (e.g. the spatial extent of an object) as measured by the action capabilities of the organism (e.g. hand span) during the performance of a particular task (e.g. prehension). This means that you can only identify their exact composition by running experiments involving the objects, an organism and a task. An outstanding example of how to do this is Mon-Williams & Bingham (2011). They had people reaching and grasping objects that varied in size along various dimensions. They then systematically investigated the effects of these size variations on the spatial structure of the prehension movement (structure such as the maximum grip aperture). 

They identified that the relevant action capability was the opposition axis. Imagine a line between your thumb and forefinger. This line has a length and an orientation. People are navigating their hands by setting those two parameters, and they alter how they grasp objects as the relationship between the opposition axis and the maximum object extent changes. The MOE is the longest possible distance across the object. If you tilt your hand as you reach for an object, you can increase the effective width your hand has to span. This, not width, is what the perception-action system is measuring with respect to the opposition axis. 

This set up holds for a certain range of prehension tasks (one handed grasping of objects within a size range that can be spanned by the thumb and forefinger). This holds, therefore, for grasping something like a coffee cup, and the MOE measured with respect to the opposition axis is the affordance for this object to a person. This will therefore not hold for a picture of a coffee cup. Pictures are thin, almost 2D objects and do not fall within the relevant range. So using pictures of cups to study the affordances of cups will simply not work.

So what do pictures of objects tell us about what those objects afford? This is a very complex question with no answer as yet, because no one to my knowledge has looked. Gibson (1979) struggled with this question; he acknowledged that there is something important here, took an initial swing at an analysis and left it there. If anyone knows of anyone who has picked this problem up properly, I would love to know. It's an important question and would help provide an ecological way into accounting for what's going on in, say, the sentence-picture verification task. We discuss this briefly in our recent Frontier paper - there is as yet no account of what information a picture of an object might contain about that object's affordances and no one relying on picture of objects has ever done the necessary work. It would be exciting to see, because it would impact many lines of research (I'm looking at you too, fMRI).

Summary
Pictures of objects do not afford the same things as the objects themselves. Two things that are not the same thing are, it turns out, not the same thing, and if you want to study affordances it's time to front up to this indisputable fact of the matter.

Mon-Williams, M. & Bingham, G.P. (2011). Discovering affordances that determine the spatial structure of reach-to-grasp movements. Experimental Brain Research, 211(1), 145-160. 

Pecher, D., de Klerk, R., Klever, L., Post, S., van Reenen, J., & Vonk, M. (2013). The role of affordances for working memory for objects Journal of Cognitive Psychology, 25 (1), 107-118 DOI: 10.1080/20445911.2012.750324 Download

Snow, J., Pettypiece, C., McAdam, T., McLean, A., Stroman, P., Goodale, M., & Culham, J. (2011). Bringing the real world into the fMRI scanner: Repetition effects for pictures versus real objects. Scientific Reports, 1, 130.