The Information for Progressive Occlusion

Gibson's ecological psychology is weird, if you are coming from a more traditional information-processing background. The two approaches make radically different assumptions about the nature of the world to be perceived; they have radically different ontologies. This means that there is little if any useful overlap in the way they do things, and communicating across the gap is very hard. I have a recent paper - preprint here - where I go into detail about the two ontologies as I defend ecological psychology from interface theory. It's essentially Turvey et al, 1981, but that's a bear of a read if you aren't already ecologically minded. Do mine first :)

Anyway, concrete examples help. My go-to is the outfielder problem but people are tired of that one. My other favourite is progressive occlusion (Gibson, Kaplan, Reynolds & Wheeler, 1969; Kaplan, 1969). Gibson worked this example up himself in great detail and so it stands as a nice concrete example to illustrate some elements of the ecological ontology. Given the recent total solar eclipse, it seems like the right time to blog it!

This post will review occlusion, talk about how it works and work with some demos. These are all linked from here; there is Matlab/Psychtoolbox code to run a demo, a video of that running and a Powerpoint with some slides. I'll refer to these throughout - occlusion is a dynamic process and so you need to see it moving for it to make sense.

Evidence for the Kinematic Specification of Dynamics

Gibson’s most detailed analysis of the KSD problem came from work on the perception of dynamic occlusion (viewing one surface become progressively hidden behind another as they move; Gibson, Kaplan, Reynolds & Wheeler, 1969; Kaplan, 1969). As one surface goes behind another, the sensations coming from the rear surface stop hitting the retina; they disappear. However, was is perceived is the progressive occlusion of a persisting surface; it is not disappearing, it is going out of view. Gibson and his students identified the kinematic pattern of transformation of the optic array that was specific to occlusion and distinguished it from the pattern specific to a surface actually going out of existence. In the former case, optical texture from the rear surface is progressively deleted over time from the optic array at an edge as it goes in behind the closer surface, and that texture progressively accretes as it comes back into view. In the latter case, there are a variety of transformations depending on how the surface is disappearing (melting vs being eaten, etc). Each event creates a specific optical pattern, but these patterns are not identical to the underlying dynamics. Observers, however, readily and easily perceive and report the underlying dynamics, not the optical patterns. Additional evidence that people are perceiving the dynamics comes from work in multiple object tracking (Scholl & Pylyshyn, 1999). People can track multiple moving targets over time, and can continue to do so even if the objects move in and out of view, but only if they do so in an occlusion event. If the objects go out of view by imploding, tracking goes to chance. In the occlusion case, the visual attention system continues to perceive a persisting object and can often pick it back up when it returns to view. In the imploding case, this system perceives that the object has ceased to exist, and it no longer tracks it. 

The Nature of Ecological Perceptual Information

The central issue in the perception of the world is how the perceptual system comes into contact with that world. In vision, the assumption for centuries has been that the experienced world must be recovered somehow from the 2D image of the world projected onto the retina by the lens of the eye. Scientists therefore hunted for patterns on the retina that preserved critical structure from the world by copying some part of that critical structure. 

James J Gibson was, for a long time, no exception to this hunt. His early empirical work (grounded in the theory he laid out in The Perception of the Visual World; Gibson, 1950) created and manipulated retinal images that, for example, contained gradients of optical texture that matched gradients of physical texture created as surfaces receded in depth, or changed their shape or orientation relative to a point of observation. But time and again, Gibson found that perceptual experience was not any straight-forward function of retinal stimulation (i.e. sensations). People did not ‘see’ what was on the retina (Reed, 1988). The most powerful demonstration of this fact is Gibson’s analysis of dynamic occlusion (Gibson, Kaplan, Reynolds & Wheeler, 1969; Kaplan, 1969) to which we will return below.
Gibson’s later career was defined by the search for an explanation of how perception could be possible if it wasn’t based on sensations and retinal images. Gibson’s solution was his theory of the ecological information available for visual perception published in The Ecological Approach to Visual Perception (Gibson, 1979). 

What follows is a description of ecological information with reference to some of Gibson’s work and the extensive research literature that has taken place since Gibson’s death in 1979. The focus will be on the information in light for vision, because that has been the focus of the research. However, the principles hold for all the various energy media our perceptual systems interact with, and we will review this briefer evidence as well.

Specification and Some of Its Consequences (A Purple Peril)

Perception is how we maintain psychological contact with functionally relevant objects and events in our environments. Explaining how we do this means describing that environment in appropriate terms and investigating what information might possibly exist for that environment, given that description. The ecological hypothesis is that the correct level is dynamics, and that describing the environment this way allows there to be information that can specify those dynamics. This information can support the kind of behaviour we need to exhibit. 

This Purple Peril describes what is meant by specification, and what that implies for how information comes to mean something to an organism. There is more detail in the various links, so check those for information too.

Task Dynamics And The Information They Create

Over the next weeks I want to turn my attention to a detailed account of the process by which you go about studying affordances (formalised as task dynamics) and the perception of affordances (via the kinematic consequences of those task dynamics) using throwing for maximum distances and for accuracy as the task. This post will introduce the basic research programme. Future posts will work through papers from my colleagues Qin Zhu & Geoff Bingham in order (I've done a couple already), as well as work from the animal literature because I want to find ways to use the analyses we're developing to answer questions about throwing and weight perception there.

These posts will do a few things. First, it's important to be as clear as possible about what affordances are, how we might possibly perceive them and how we can do the relevant science within the ecological approach to answer those two questions. Sabrina is developing ways to apply these methodological principles to the study of language, and we have both been working on the issue of information and how it comes to have meaning for us. Being clear about how this all unfolds in the perception-action literature is vital, because this is the foundation for what comes next. Second, I'm working on some throwing data right now and I need to work through the key papers in detail anyway. Third, I'm going to be developing an undergraduate perception-action class for 2014, and this will help me develop course material by laying out the form of the analysis and getting feedback on how well it's coming across. One of my goals is to look at all my collated and edited notes and realise I've accidentally written a text book :)

I'm going to talk about throwing because it's utterly fascinating. It's a complex task but it's one centred around a core dynamic (that of projectile motion) that physics has a pretty good handle on. This is letting us run detailed simulations of the task to identify the affordance structure of the task and see how throwers are operating with respect to those. Throwing entails perception of object and target affordances and the coordination of multiple body segments into precisely timed actions controlled by that perception. It also connects to all kinds of things in our evolutionary history (including, possibly, the origins of spoken language in the form we know) and our psychology (including the size-weight illusion and issues of the psychologist's fallacy). It's close to being that grail of psychology, something only humans do (other animals throw but rarely if ever for the kinds of distances and accuracy we can manage with ease). And most of all, it is endlessly interesting. The deeper I get into this, the cooler it gets. 

Non-Specifying Variables in the Perception of Collisions (Specification III)

Part of Withagen's critique of specification and whether it's necessary to underpin direct perception is a brief review of some empirical literature that shows people using non-specifying variables. I want to spend a few posts reviewing these, because all good potentially sensible ideas need data to confirm whether they're right or not.

First up, the perception of relative mass after a collision. Events in the world are dynamic, that is, they involve motion caused by a pattern of underlying forces. Perceptual systems want access to the underlying dynamics of events, because this is the level at which the event is defined (Wilson & Bingham, 2001). However, perceptual systems can only detect kinematics, that is, motion - this is the perceptual bottleneck  (Bingham, 1988 and this note on dynamics and kinematics). We can only perceive the underlying dynamics of an event, according to the ecological approach, if we can detect motion that is specific to that dynamic. Runeson coined the phrase kinematic specification of dynamics (Runeson & Frykholm, 1983) and investigated whether there were such kinematic patterns and whether we can detect them. Working with Claire Michaels and David Jacobs, he has also investigated the use of non-specifying variables.