People interested in how perception and action affect cognition have begun talking about affordances. This should be great news; the ecological approach suggests that affordances are the properties of the world that we perceive that enable us to control our actions, so if you are interested in how action can ground, say, memory or language, then discussing affordances should enable real progress.
The term 'affordance', however, is a technical term, and it refers to very particular properties of an organism's environment. There are methods for experimentally identifying exactly how these properties are composed, and there are methods for testing our perception of them. If you aren't using these methods, and if you aren't using the term correctly, then you aren't studying affordances.
A recent example of this kind of work is Pecher et al (2013; pdf). They were interested to see whether object affordances play a role in working memory for those objects. They ran two experiments which used pictures of objects with varying affordances for manipulability, and found that in each case memory for these objects was not affected by an interfering motor task, suggesting affordances are not used to structure working memory.
There is a key problem here, and that is that pictures of objects do not afford the same things as the objects themselves. Interact with a coffee cup, and a picture of a coffee cup; they provide very different opportunities for interaction. You cannot drink out of a picture of a coffee cup; you cannot hold it by the handle because it has no handle! (See Snow et al, 2011 for a demonstration of this in an fMRI context).
The fact that these things afford different things is not intrinsically a problem; the affordances of these things and the different information these different affordances produce is, after all, how we can perceive that they are different things. It is a problem, however, if you are trying to study how the affordances of coffee cups might affect your memory for coffee cups by using pictures of coffee cups. You aren't studying the affordances of coffee cups, and you shouldn't actually get to say that you are studying the affordances of coffee cups.
Studying affordances is hard. They are odd things; they are properties of the world (e.g. the spatial extent of an object) as measured by the action capabilities of the organism (e.g. hand span) during the performance of a particular task (e.g. prehension). This means that you can only identify their exact composition by running experiments involving the objects, an organism and a task. An outstanding example of how to do this is Mon-Williams & Bingham (2011). They had people reaching and grasping objects that varied in size along various dimensions. They then systematically investigated the effects of these size variations on the spatial structure of the prehension movement (structure such as the maximum grip aperture).
They identified that the relevant action capability was the opposition axis. Imagine a line between your thumb and forefinger. This line has a length and an orientation. People are navigating their hands by setting those two parameters, and they alter how they grasp objects as the relationship between the opposition axis and the maximum object extent changes. The MOE is the longest possible distance across the object. If you tilt your hand as you reach for an object, you can increase the effective width your hand has to span. This, not width, is what the perception-action system is measuring with respect to the opposition axis.
This set up holds for a certain range of prehension tasks (one handed grasping of objects within a size range that can be spanned by the thumb and forefinger). This holds, therefore, for grasping something like a coffee cup, and the MOE measured with respect to the opposition axis is the affordance for this object to a person. This will therefore not hold for a picture of a coffee cup. Pictures are thin, almost 2D objects and do not fall within the relevant range. So using pictures of cups to study the affordances of cups will simply not work.
So what do pictures of objects tell us about what those objects afford? This is a very complex question with no answer as yet, because no one to my knowledge has looked. Gibson (1979) struggled with this question; he acknowledged that there is something important here, took an initial swing at an analysis and left it there. If anyone knows of anyone who has picked this problem up properly, I would love to know. It's an important question and would help provide an ecological way into accounting for what's going on in, say, the sentence-picture verification task. We discuss this briefly in our recent Frontier paper - there is as yet no account of what information a picture of an object might contain about that object's affordances and no one relying on picture of objects has ever done the necessary work. It would be exciting to see, because it would impact many lines of research (I'm looking at you too, fMRI).
Summary
Pictures of objects do not afford the same things as the objects themselves. Two things that are not the same thing are, it turns out, not the same thing, and if you want to study affordances it's time to front up to this indisputable fact of the matter.
Mon-Williams, M. & Bingham, G.P. (2011). Discovering affordances that determine the spatial structure of reach-to-grasp movements. Experimental Brain Research, 211(1), 145-160. Pecher, D., de Klerk, R., Klever, L., Post, S., van Reenen, J., & Vonk, M. (2013). The role of affordances for working memory for objects Journal of Cognitive Psychology, 25 (1), 107-118 DOI: 10.1080/20445911.2012.750324 Download
Snow, J., Pettypiece, C., McAdam, T., McLean, A., Stroman, P., Goodale, M., & Culham, J. (2011). Bringing the real world into the fMRI scanner: Repetition effects for pictures versus real objects. Scientific Reports, 1, 130.
Bit of a sidenote. I presented and discussed that Snow paper in our neuro-seminar, with a bunch of people who know how to do fMRI etc. They were not impressed. Not by the paper (the question is quite interesting though). I summarize that experience, with some editorializing here
ReplyDeletehttp://asehelene.wordpress.com/2012/09/23/on-not-noticing-major-problems-with-a-study-and-what-that-may-say-about-the-current-incentive-structure-in-science/
OK, here is the link with html
ReplyDeleteIncentive structure
Important factor, often overlooked, in the Sally-Anne tests of false belief. Responses will differ depending on whether the scenario is depicted in writing, verbally, pictorially, in the form of a model or acted out. Responses in children quite likely to depend on visual, auditory, verbal processing ability etc.
ReplyDeleteJohn Kennedy at Toronto has been working in picture perception. He got his PhD with Gibson in 1971. I think most of his work isn't directly relevant to your question here, its about perspective, haptic pictures and the visually impaired, but his earliest stuff might be relevant.
ReplyDeleteKennedy, J.M. (1970). Information in pictures. Conference on Structural Learning, University of Pennsylvania, Philadelphia.
Kennedy, J.M. (1970). Pictorial perception. Conference on Ecological Optics, Cornell.
Kennedy, J.M. (1974). A psychology of picture perception. San Francisco: Jossey Bass.
Kennedy, J.M. (1983). Meaning based theory of depiction. Art Education, 36, 12 14.
Kennedy, J.M. (1987). A Gibsonian picture theory. Invited address. Symposium on ecological theory of perception and representation, Bonn, October 9 10.
Kennedy, J.M. (1995). Where is the information in a picture or a symbol? Invited address, Division 10, American Psychological Association, August 11-15, New York.
Beyond all that, I think the question of what can be afforded by a picture, or on a computer display, or in a virtual environment remain fertile questions for perceptionists.
I just found 'Psychology of Picture Perception' online!
DeleteYou can find a number of papers by Gibson that are relevant to this topic here: http://gombrich.co.uk/gombrichgibson-dispute/
ReplyDeleteI find his paper "The Information Available in Pictures, Leonardo, Vol. 4, 1971, pp.27-35" particularly informative; In my opinion it is much more clearly written than the chapter on picture perception in his 1979 book, although I think he's trying to convey a similar message in both writings.
Here are two quotes from the 1971 paper that, I think, summarise Gibson's view well (sorry, I don't have page numbers):
"A picture is a surface so treated that a delimited optic array to a point of observation is made available that contains the same kind of information that is found in the ambient optic arrays of an ordinary environment.”
"This definition covers both the photograph and the caricature. It admits that a photographic color transparency can provide an eye with almost the same brightness and color contrasts that the cone of light intercepted by the camera provided. The relations of luminous intensity and spectral composition of the stimulus energies in the two arrays are in sufficient correspondence to make the low-order stimulus information very nearly the same. But the definition is broad enough also to admit the case of a caricature, where the contrasts of luminous energy are quite different and even the forms are different but where the high-order information to specify a particular person is common to both arrays. In short the optic array from a picture and the optic array from a world can provide the same information without providing the same stimulation."
The first quote and the last sentence of the second quote suggest that affordance perception, at least in some cases (such as what tools afford in terms of their function, not their, say, graspability), can in principle be studied using pictures (like Pecher et al did).
In a recent paper in the special issue of Avant on ecological psychology to which you (and I, together with D. Dotov and L. Nie) also contributed a paper, Costall writes about "affordances in general" and the "canonical affordances" that are connected primarily to artifacts: http://avant.edu.pl/wp-content/uploads/AC-Canonical-affordances-in-context.pdf
I think that paper may be relevant to this discussion as well.
Cheers,
Matthieu
The first quote and the last sentence of the second quote suggest that affordance perception, at least in some cases (such as what tools afford in terms of their function, not their, say, graspability), can in principle be studied using pictures (like Pecher et al did).
DeleteYes, although as I note here, I think that Gibson overestimates the amount of actual information that can be captured by a static image. More recent research has focused on motion based information and this seems to be pretty common.
That said, he does highlight that picture perception is just an interesting example of perception, and so the same rules apply. This is a good starting point, I think.
Great that you've continued writing on picture perception in the blog. I think it's high time indeed that more ecological psychologists started thinking (again) about this topic. And thanks for pointing out the book by John Kennedy, I've added it to my pile..
DeleteI'm genuinely fascinated, and it fills an interesting gap in the research on information, which is fundamentally what Sabrina and I are grappling with right now.
DeleteWow - great set of links, people!! It's great that this stuff is around and about time it resurfaced; thanks a lot for all this, I'll add it to my pile!
ReplyDeleteAndrew, Great post! (And great replies)
ReplyDeleteThis is, of course, one of the Big Questions that ecological psychologists have been mostly avoiding these past 50 years. The big fear (I suspect) is that picture perception will require Something Different.
As you point out, there are not affordances, and yet some people seem to experience the presence of affordances. Gibson tried to argue that there was some information present, but if you take the moving-through-the-world-to-detect-affordances principle seriously, there is not sufficient information to account for the experienced affordances. That is, whatever information is re-presented (in the proper use of the term) when I look at a photograph of a coffee cup, it is not sufficient to tell me whether the original coffee cup was pick-up-able.
What can we allow in this type of situation while staying true to first principles? Is there a way to save Direct Perception in the case of literal, physical re-presentation? Is it sufficient that “mental” never enters into the discussion, or is any re-presentation too risky to allow? Etc.
but if you take the moving-through-the-world-to-detect-affordances principle seriously, there is not sufficient information to account for the experienced affordances. That is, whatever information is re-presented (in the proper use of the term) when I look at a photograph of a coffee cup, it is not sufficient to tell me whether the original coffee cup was pick-up-able.
DeleteThis is the empirical question, I think. People can make judgments about the affordances of objects in pictures; they can't use the information underpinning that judgment to perform an action, though, because there isn't really an object there with those affordances. So pictures are clearly providing some information about the object.
And it is about the object. If I take a picture of a coffee cup, the static image structure that is preserved in the image is the static image structure created by that cup and present from the camera's point of observation.
Now, this information doesn't transform it would as you move around a a real cup; it transforms as it would when you move around a static image of a cup. That's how you know it's a picture and not a cup.
I need to read all the links listed here; I'm very intrigued by this question and it could be a really interesting project.
And it is about the object. If I take a picture of a coffee cup, the static image structure that is preserved in the image is the static image structure created by that cup and present from the camera's point of observation.
DeleteWell... yeah... but a single eye, not moving, in a not moving world can't unambiguously know the world. That is what takes us down Descartes's rabbit hole to begin with. The whole point of Gibson 1961, which formally starts the ecological approach, is to overcome this classic philosophical corundum.
You need the transformations to disambiguate, because an infinite array of worlds can create a static projection to a single point.
Of course. But as evidence by pictures, that static image structure ain't nothing. The question is, what is it? :)
DeleteI think it was Gibson that made the point that when observers in perception experiments look at the images in our displays, they are essentially playing a game, knowing that, when we ask them to judge size, distance, an affordance, etc, that those "entities" are not real but simulated, and as part of the "game" they agree to play along and give judgments as if they were real. Or at least maybe I heard that from Bingham or another...
ReplyDeleteThis recent paper from robotics may be useful to you:
ReplyDeletehttp://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=5778950&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5778950
This comment has been removed by the author.
ReplyDelete