General Ecological Information Does Not Support the Perception of Anything

One common critique of the ecological approach is how can we use perception to explain behaviour that is organised with respect to things in the world that aren't currently in our area? How do we plan for future activities, or how do we know that the closed fridge has beer? 

A recent attempt to get ecological about this comes from Reitveld & Kiverstein (2014) who propose a relational account of affordances that enables them to talk about opportunities for more complex behaviours. This account has developed into the Skilled Intentionality Framework (e.g. Bruineberg & Rietveld, 2014), where skill is an 'optimal grip' on a field of task-relevant, relational affordances. 

I have always had one primary problem with this programme of work - I don't believe that they can show how these affordances create information and thus can be perceived. I discuss this here and here, and there's comments and replies for Rietveld and Kiverstein there too. You can indeed carve the world up into their kind of entities, but if they don't create information then they cannot be perceived and they are irrelevant to behaviour. 

I was therefore excited to see a new paper from the group called 'General ecological information supports engagement with affordances for ‘higher’ cognition' (Bruineberg, Chemero & Rietveld, 2018; hence BC&R). There is a lot of excellent work in here; but their proposal for a general ecological information is, in fact, neither ecological nor information. It is a good way of talking ecologically about conventional constraints on behaviour, but it doesn't make those perceivable and so the main thesis of the paper fails. 

The Information for Progressive Occlusion

Gibson's ecological psychology is weird, if you are coming from a more traditional information-processing background. The two approaches make radically different assumptions about the nature of the world to be perceived; they have radically different ontologies. This means that there is little if any useful overlap in the way they do things, and communicating across the gap is very hard. I have a recent paper - preprint here - where I go into detail about the two ontologies as I defend ecological psychology from interface theory. It's essentially Turvey et al, 1981, but that's a bear of a read if you aren't already ecologically minded. Do mine first :)

Anyway, concrete examples help. My go-to is the outfielder problem but people are tired of that one. My other favourite is progressive occlusion (Gibson, Kaplan, Reynolds & Wheeler, 1969; Kaplan, 1969). Gibson worked this example up himself in great detail and so it stands as a nice concrete example to illustrate some elements of the ecological ontology. Given the recent total solar eclipse, it seems like the right time to blog it!

This post will review occlusion, talk about how it works and work with some demos. These are all linked from here; there is Matlab/Psychtoolbox code to run a demo, a video of that running and a Powerpoint with some slides. I'll refer to these throughout - occlusion is a dynamic process and so you need to see it moving for it to make sense.

Ecological Information Is a Perceptual Mapping That Tracks Evolutionary Fitness

Interface theory in cartoon form. Thanks to Louise
Barrett for reminding me this exists :)

In my last post I was thinking out loud about some ecological lines of attack on interface theory (Hoffman et al, 2015). The first line of attack (Hoffman et al mischaracterise Gibson) fell over eventually; they don't quite go at it right, but using ecological information does fit their definition of a naive realist perceptual strategy ( 'a perceptual strategy for which X [perceptual experience] ⊂ W [the world] and P [the perceptual mapping] is an isomorphism on this subset that preserves all structures on W'; pg 1483). The second line of attack (everything they say about veridicality vs fitness applies only to inferential, constructivist theories of perception and Gibson's not playing that game) is true but not that interesting or convincing to anyone with established views.

Thanks to chats on Twitter (thanks Greg!) and applying the basic move of 'those aren't working but IT is still weird, what's next?', my new line of attack relates to a result from their simulations.

Specification: What It Is, and Why We Need It (Specification I)

The first thing I need to do in a discussion of specification is explain what it is and why it's important to ecological psychology. I've tried to maintain a clear logical progression in this post, building towards the need for specification. In my next post, I'll take a first swing at explaining what specification gives us, namely a reason why information means one thing and not another.

The issue of specification comes from Gibson's (1966, 1979) analysis of visual perception, so that's where I'll start too. Most descriptions of visual perception begin with the anatomy of the eye; people note that the eye resembles a camera, and that the lens seems to focus a messy, upside down image onto the retina. The retina then pixelates that image into neural activity, and this pixelated structure then shows up in primary visual cortex (this is topographic mapping). If vision does indeed begin this way, then a huge amount of work seems to be required to take this impoverished stimulus and use it as the basis for the rich, 3D visual world we experience.

Gibson's ecological theory begins with a re-evaluation of the stimulus for vision. The first three chapters of the 1979 book are about the world and what it contains, while chapter 4 is about how this world can interact with light to produce information. Only once he lays out the information available to the organism does he begin to talk about the act of perception itself; this re-evaluation of the 'job description' for a visual system is one of his most important contributions to psychology. Gibson's reanalysis leads him to conclude that action relevant properties of the world (specifically, affordances) can be specified in the optic array, and this concept underpins the directness of his theory of perception.

The issue of specification is assumed to be critical for the success of a direct theory of perception. The traditional views propose a 'many-to-one' mapping; a given pattern of stimulation on the retina is ambiguous because it could be caused by many possible states in the world. Specification is the hypothesis that there is a 'one-to-one' mapping - a given pattern in the optic array comes from one and only one state of the world. This can happen, according to Turvey, Shaw, Reed & Mace (1981) if (and only if) the creation of information about the world is a lawful process. If the projection of world into optics is underwritten by a law and thus one-to-one, then detecting the optical pattern is equivalent to detecting the property of the world: detecting the information is perceiving the world, with no additional processing work required. Perception can be direct.

A theory of direct perception will require several elements: there must be invariant structure within the endless flow across the retina that relates 1:1 to some property of the world. To be invariant, this structure must be relational, and therefore higher order. If perception is to be direct, these higher-order invariants must be detectable as a piece, and not built out of their elements in some post-perceptual process. Only if you have all this do you have the possibility of a one-to-one mapping between the world and vision, i.e. the possibility of specification.This post lays out what this all means, and  how these pieces come together in ecological psychology.

Prospective Control I: The Outfielder Problem

A couple of posts ago I raised the distinction between prediction and prospective control. I was trying to make the point that, if you are coupled to the right information, you don't need to be mentally simulating what's happening so you can run this simulation ahead and predict what's coming up. Prediction of this sort is invoked by representational cognitive scientists to cope with things like delays in the nervous system (e.g. Changizi's 'perceiving-the-present' framework). It's a risky business (if you make a mistake, you are suddenly controlling your behaviour according to an incorrect guess), and the better solution is prospective control. This is when you couple your behaviour to information in the world that doesn't tell you about the future, but that, if you use it for a while, will get you where you want to go.

People requested some more on this topic, and so here we go. People also requested something other than the outfielder problem, but I am going to start with this problem because it is still the best example, and I'll get into some more as we go to demonstrate it's not just baseball where this works.

F*cking affordances - how do they work?

Over on Bounds of Cognition, Ken has been doggedly pursuing what he thinks is a critical problem with the concept of affordances as described by Gibson (1979) and expanded on by Turvey, Shaw, Reed & Mace (1981; hence TSRM). I feel the need to spend some time consolidating my responses and some ideas in one place; every time I try to lay out why the problem is ill-posed or a potential route out, Ken just says I'm clouding or avoiding the issue or throwing out red herrings.

I actually think Ken has identified one very useful critique of the affordance concept: the lack of care with which we attach '-able' to words. But Ken isn't just making a methodological point; he thinks he's shown that affordances cannot structure light in a way that can specify the affordance, and that's the argument that needs to be tackled.

Saddle up: this is going to take some time, and I'm not going to solve everything. But to get a little ahead of myself, the answer to the problem is that it's complicated.

There's No Prospective Information About Friction, or, Why I Fell Over on the Ice

In which I justify why I, a healthy perceiver-actor, slipped and fell on a clearly visible icy patch, breaking my wrist for the second time, using SCIENCE.

It's been a cold, icy winter here this year, and 6 weeks ago I slipped on a patch of ice and fell entirely on my (previously broken) wrist. The ensuing physics did enough damage that I needed surgery to set the wrist with two pins, and I am only today out of the cast. These kinds of falls and injuries are very common; half of all falls  in the US are caused by insufficient friction, and the types of injuries (broken wrists and collarbones, etc) suggest reactive responses to the slip - people using their arms to try and regain a sudden, unexpected loss of balance. 

The two papers I'm going to talk about are from the lab of my favourite developmental psychologist, Karen Adolph, who has done some excellent affordance work using the transition from crawling to walking as a way to studying the changing perception-action performance of children. This research, however, asked about whether a perceiver can detect information about upcoming friction conditions and use this information for prospective control. The answer seems to be no, because there isn't any information. Given that action requires information, the absence of information might explain the often catastrophic failures of action we see on ice and other low-friction surfaces.

At least, that will be my story.

Gibson vs Physics: Gibson Wins, at the Ecological Scale

One of the interesting questions that popped out of our discussions with Ken Aizawa about Runeson and the Ames Room is this: did Gibson and his followers banish physics and geometry from his psychology? And if so, is Runeson breaking this prohibition by talking geometrically about the Ames Room? And how can you banish physics anyway - what the hell?

I think I've convinced Ken that there is no such prohibition in the comments at the posts linked to above (I think). But it's a topic of fairly central importance to the ecological approach, so I wanted to summarise some of these ideas and examples here.

Poverty of Stimulus and Ecological Laws

A question about a term I’ve been using is a nice segue into an important moment in the history of the ecological approach.

Poverty of stimulus is a term that came from Chomsky, but the intuition has been underpinning theories of perception for as long as there have been theories of perception. The term describes a problem, in which the information required to achieve something is not present in the environment. In language, the argument runs

1. Certain patterns of correct language use can only be learned with exposure to negative evidence (i.e. evidence about what counts as incorrect)
2. Children learning languages only encounter positive evidence (i.e. evidence about what counts as correct)
3. Children do acquire the patterns in (1).