The Information for Progressive Occlusion

Gibson's ecological psychology is weird, if you are coming from a more traditional information-processing background. The two approaches make radically different assumptions about the nature of the world to be perceived; they have radically different ontologies. This means that there is little if any useful overlap in the way they do things, and communicating across the gap is very hard. I have a recent paper - preprint here - where I go into detail about the two ontologies as I defend ecological psychology from interface theory. It's essentially Turvey et al, 1981, but that's a bear of a read if you aren't already ecologically minded. Do mine first :)

Anyway, concrete examples help. My go-to is the outfielder problem but people are tired of that one. My other favourite is progressive occlusion (Gibson, Kaplan, Reynolds & Wheeler, 1969; Kaplan, 1969). Gibson worked this example up himself in great detail and so it stands as a nice concrete example to illustrate some elements of the ecological ontology. Given the recent total solar eclipse, it seems like the right time to blog it!

This post will review occlusion, talk about how it works and work with some demos. These are all linked from here; there is Matlab/Psychtoolbox code to run a demo, a video of that running and a Powerpoint with some slides. I'll refer to these throughout - occlusion is a dynamic process and so you need to see it moving for it to make sense.

Ecological Information Is a Perceptual Mapping That Tracks Evolutionary Fitness

Interface theory in cartoon form. Thanks to Louise
Barrett for reminding me this exists :)

In my last post I was thinking out loud about some ecological lines of attack on interface theory (Hoffman et al, 2015). The first line of attack (Hoffman et al mischaracterise Gibson) fell over eventually; they don't quite go at it right, but using ecological information does fit their definition of a naive realist perceptual strategy ( 'a perceptual strategy for which X [perceptual experience] ⊂ W [the world] and P [the perceptual mapping] is an isomorphism on this subset that preserves all structures on W'; pg 1483). The second line of attack (everything they say about veridicality vs fitness applies only to inferential, constructivist theories of perception and Gibson's not playing that game) is true but not that interesting or convincing to anyone with established views.

Thanks to chats on Twitter (thanks Greg!) and applying the basic move of 'those aren't working but IT is still weird, what's next?', my new line of attack relates to a result from their simulations.

Does Interface Theory Have Consequences for the Ecological Approach?

I've been working on a commentary about interface theory (Hoffman, Singh & Prakash, 2015) which I have previously blogged about here. I'm still interested because it is, in part, a fairly direct shot at the ecological approach and I'm always keen to break those if I can. My piece stalled out, though, so I thought I'd spend some time here thinking out loud about the argument that stalled and another critique that came up as I re-read the paper.

To unbury the lede I just finished writing: the primary thing Hoffman et al get wrong about Gibson is that they think he wanted his theory to produce veridical perceptions, not simply adaptive ones. Gibson actually wanted adaptive perceptions, but found a way in which they were also veridical. This emphasis matters; Gibson does not stand or fall on issues of veridicality. In addition, every one of Hoffman et al's big swings apply only to inferential, constructivist theories of perception; Gibson is immune on these grounds as well. All Hoffman et al have done is redraw the terrain inferential theories have to traverse, and it will be interesting to see if anyone takes the bait. But the major argument simply remains, is perception inferential or ecological, and may the best data set win.

On "The poverty of embodied cognition" (Goldinger et al, in press)

A new paper in Psychonomic Bulletin and Review (Goldinger, Papesh, Barnhart, Hansen & Hout, 2015) has taken a swing at the field of embodied cognition, claiming that it is vague, trivial and unable to add anything scientific to the investigation of cognition.

...our goal is to zoom out from specific empirical debates, asking instead what EC offers to cognitive science in general. To preview, we argue that EC is theoretically vacuous with respect to nearly all cognitive phenomena. EC proponents selectively focus on a subset of domains that work, while ignoring nearly all the bedrock findings that define cognitive science. We also argue that the principles of EC are often (1) co-opted from other sources, such as evolution; (2) vague, such that model building is not feasible; (3) trivially true, offering little new insight; and, occasionally, (4) nonsensical. 

My basic take is a) I actually agree with a lot of the criticisms in the context of the kinds of 'embodied' cognition we critique for similar reasons, but b) there is nothing new to any of these critiques, none of them are compulsory failings of the field and nothing about them makes embodiment an intrinsically empty notion. 

The ecological approach to sporting performance

Last week I took part in a Google Hangout with Mark Upton and Al Smith, who run the sports coaching blog 'My Fastest Mile' and who are generally all about getting sports coaching and training to be more ecological and dynamical. The recorded hangout is here - I assume it's good, I can't watch myself on video without cringing :) I've fleshed out some of these ideas below; please comment below or on Twitter if there's anything you want more on.

We talked perception, affordances and a little about what makes the ecological approach different from more cognitive approaches. There's a lot of detail sitting under the discussion; see, well, the rest of the blog to get a sense of where I'm coming from! I'm not a sports scientist, and so my particular research programme isn't specifically about applying the ecological approach to sports. However, I do study some sports related activity (specifically throwing; see my latest paper, blogged here). 

The main thing that gets applied to sports from the ecological approach is the concept of affordances (which is the topic of that paper). Affordances are cool, obviously, but the thing that makes an approach 'ecological' is a focus on information; how things like affordances are perceived. I wanted to briefly sketch how I see ecological psychology feeding into sports, which I think it's a great idea for everyone involved.

A disclaimer: I am also not a coach. I'm just a scientist. I like to let people know that I know this, because I've found that whenever I interact with practictioners of any kind (coaches, but also occupational therapists, physios etc) there's a real resistance to listening to people like me. I mean, what do we know? We aren't in the trenches, working with the athletes or patients, and our wonderful ideas might simply not apply to the messy real world. I've also chatted to people who felt worried I would be judging their messy actual practice, the one they've put together over years of experiencing the actual needs of the people they help. 

I understand this concern entirely. Let me say, for whatever it's worth, that I am not trying to waltz in, figure out what you're doing wrong and save you with my wonderful theory. I always see my role as just 'the scientist in the conversation'. I'm going to listen to what you actually want to know, and I'm going to see if there's anything I know from what I do that can feed into that process. It will either work or it won't, and that's ok. I get to think about ways to make my science work in an interesting applied high stakes context; I hope you might get some insights into how learning works that help you make sense of what you see in the people you work with. 

The Interface Theory of Perception - The View from Ecological Psychology

Psychonomic Bulletin & Review has released, with much fanfare and a hashtag, an article called 'The Interface Theory of Perception' (Hoffman, Singh & Prakash, 2015). From the website description:

In a nutshell, interface theory postulates that our perception operates like a species-specific desktop: We perceive the world in representations that do not represent the “truth” about the world as it actually is, but that are useful “icons” which represent fitness-relevant information about the world. To illustrate, imagine a world in which red and green berries are nutritious but blue and yellow berries make you sick. Will your perceptual system differentiate red from green and blue from yellow? According to interface theory, the answer is no—the organism will have evolved to differentiate between only two colors, namely gred and byellue.

The meat of the paper is a series of evolutionary simulations that pit various perceptual strategies against one another. These strategies vary in how veridical they are, and the key result is that interface strategies, in which perception codes things in a way that bears no resemblance to the world, wins every time. We do not perceive the world as it really is. 

This seems to go against people like Gibson, who argue that perception is of a real world and real properties of things, like affordances. These simulations seem to show that 'realist' perceptual strategies are evolutionarily unsustainable. 

The devil, as always, is in the details, and having read the paper I am now pretty sure that Gibson is quite safe, and that information offers a path out of the weirdness Hoffman conjures.

(Re)Introducing "The Purple Perils"

'Note for a tentative redefinition of behaviour', 1975. 

James J Gibson ran an afternoon seminar for many years at Cornell. This seminar was widely attended by a variety of students, professors and visiting scholars, and Gibson used them as a place to try out new ideas, new ways of describing those ideas and generally hammer his theory as hard as he could to see where the weak and strong points were.

Before many of these seminars, he would write and distribute a note, often fairly detailed, detailing the topic of that week's meeting. These were copied as he typed by 'ditto' sheets which transferred the text onto multiple sheets of paper. The ink on these sheets was purple, and the notes became known as the Purple Perils. (There is a nearly complete archive of these online hosted by Bill Mace.) 

These Perils were never the final word on anything. They reflected Gibson's current thinking and were always up for debate. That debate often found it's way into the next week's seminar and Peril, so these were always works in progress.

I've just finished reading Ed Reed's Gibson biography (there's a couple of taster chapters here) and I've been inspired by his description of the kind of scientist Gibson was; the Perils and his seminar were good examples of the kind of rigour and openness he embraced in his science. So, in this spirit, we are going to start posting some shorter, more focused posts on specific topics and ideas from the ecological embodied cognitive science we are developing. They will reflect our current thinking on the topic, but not necessarily our final thinking, and they will not aim to solve everything, just move it forward. They will typically include some 'facts of the matter' as seen from the ecological view, and some analysis to reach some conclusions and hypotheses based on those facts. The goal is to stimulate debate and discussion and come away with a better, clearer theory. 

We would like to invite you all to come get into it with us; ask questions, challenge us, agree with us (we like this too!), talk to each other in the comments and argue/agree with each other. Keep it friendly, keep it a little focused on the topic and try to be specific, detailed and clear in your arguments. If specific parts of topics become sticking points they will likely show up in future Perils for a more focused discussion; otherwise they will be what we're trying to clarify for ourselves. If there are particular topics that are bugging you about radical embodied cognitive science, post about it here and they may show up as future Perils. "What about language then, huh? Huh?" is not something we can work with, though, sorry - be specific :)

We cannot promise to have time to do one every week (Gibson didn't have to teach anything else!) but we will post them as frequently as we have things to post. Feel free to steal the idea and host something similar on your own blogs; we will link to them here. Gibson valued a good argument above all other things in science, and I entirely agree, so please take any opportunity you see to make this a dialogue people can all take part in. 

Are we Infomation Processers? (A brief note)

I thought about it a lot and I kept thinking: OK, he’s right, I guess, the information is in the light, it has to be there, because where else are you going to get it. It has got to be there and if it’s there, there’s a sense in which you don’t have to process it at least not in the way that I used to say “process.” But if he’s right, what am I going to do about cognitive psychology? How can I reconcile cognitive psychology, as I knew it, with this theory of Jimmy Gibson’s?

Ulric Neisser, from Szolkolszky, 2013

Are we information processors? No. At least, not if Gibson was right.

Information is clearly important, though, because we talk about it a LOT according to this Wordle.

For James Gibson, information is external to the observer. Information is structure in energy arrays (e.g. the optic array for light) that is specific to the object or event in the world that caused the structure. This structures becomes information when we use it to coordinate and control our behaviour.

This information is not transmitted. At any given possible point of observation, there is a uniquely structured optic array that an observer can interact with by going to (or more likely through) that point of observation. That structure is there as soon as the lights have come on and the light is done filling up the space.

This information is also not processed, because it does not 'get into the system'. The nervous system doesn't take information onboard, it resonates to that information; it's dynamical behaviour is altered by detecting that information. 

What about all the steps that have to happen once the information is detected? Doesn't the information have to be transformed into a behaviour? No. Behaviour simply is the activity of the kind of embodied system that we are in the presence of that particular information. We can alter the kind of embodied system we are via learning, but all the way through learning, the behaviour you exhibit at any given moment simply is that activity.

Therefore, in the radical embodied, ecological approach, it makes no sense to say that cognition involves information processing. This is due to 'information' in REC referring to Gibson-information, not Shannon-information. Shannon-information is not something that exists. It is an abstract description of how to reduce uncertainty between a sender and a receiver. It's an amazing idea; it's the heart of the digital revolution we live in and it's a powerful analysis tool (read James Gleick's great book, The Information for the history). It's just not describing what biological organisms are interacting with. 

Is this merely a semantic issue? Are we cheating by just defining away the problem? No. It's about precision in terms. Information means something very specific in the REC framework, that meaning is not the same as it is in the information processing framework and this difference has consequences. Following up on those consequences is what radical embodied cognitive (neuro)science is up to. We will either be right or wrong, and the data will tell us which, but only if we stop making this basic confusion.

Thanks to Greg Hickok (Twitter, blog) for arguing with me a lot on Twitter about this which helped me clarify a few things, and to everyone else who got into it too. You guys are a big help with your obstinacy and your refusal to take what I say at face value :) I think this post might be the first of a series of 'Brief Notes' where I just try to lay out one thing as clearly as I can. In true Gibson style, I reserve the right to keep critiquing and modifying the details as new evidence comes in!

References
Szokolszky, A. (2013). Interview with Ulric Neisser. Ecological Psychology, 25, 182–199. Download

Visual illusions and direct perception

A while back I reviewed a bunch of papers by Rob Withagen who is currently arguing that while perception is not typically based in specifying variables, it can and should still be ecological in nature. While we are also developing an account of information that goes beyond specification I still have some reservations about the details of Rob's work. That said, I do think there is a lot of overlap and I'm still interested in figuring out what that is. 

Rob's latest paper (de Wit, van der Kamp & Withagen, 2015) is about visual illusions, and how to talk about them ecologically. There is a lot of very useful stuff in here, not least the review of the many things Gibson said about illusions. After this review, the paper tries to put all of this into Rob's and Tony Chemero's evolutionary framework and uses this to formalise and tidy up what Gibson was up to. I'm adding this paper to the set I've covered from Withagen as I continue to think through these issues.

The Information Available in Pictures

I've become fascinated with the problem of pictures and how they relate to the things they are pictures of. One reason is the regular use of pictures of objects to study how the affordances of those objects might ground cognition; this, I think, is a major problem. 

A more positive reason is that, like language, pictures contain information about something they themselves are not (see Sabrina's information taxonomy). I have a hunch that an ecological study of picture perception might help guide an ecological study of language, because the former can take more direct advantage of the work already done about how we perceive meaning in events via ecological laws but then act as a bridge, a point along the way to the conventional world of language meaning.

Finally, the topic seems to be woefully understudied in the ecological approach. There is some, however. In the comments section on my rant about using pictures to study affordances, I was pointed to the work of John Kennedy (a Gibson student, now emeritus at the University of Toronto). I have downloaded his 1974 book, 'A Psychology of Picture Perception' and am working my way through it. Matthieu de Wit then linked me to an archive of a discussion, in papers, between Gibson and Ernst Gombrich about picture perception. I thought I'd start with Gibson (1971), The Information Available in Pictures, to begin to sketch out what we know and what we don't.

The current question at hand is, can pictures provide the same information about the things they depict?

Our first paper based on the blog is out in time for Christmas!

Last year we were invited to contribute to a special issue of Avant, an interdisciplinary open access journal that publishes in English and Polish. The issue is on affordances, and we were asked to contribute a piece detailing the contribution ecological psychology is making to modern cognitive science. Our paper is here; the complete issue is open access and available here (as separate files, a single issue pdf, and an ongoing Polish translation).

We had a lot of fun with this paper. The core of the paper comes from our posts about how psychology lacks a central theory and how Gibson serves as a good model for why theory matters, as well as being a key part of any more complete theory. We take a quick swipe at Daryl Bem and JPSP and then use the contrast between dynamic pattern vs. perception-action approaches in explaining coordinated rhythmic movement. In particular, we highlight Geoff's model as an exemplar of what psychologists should be up to - not just models, but theoretically motivated models.

We'd like to thank Tony Chemero for mentioning us to the people at Avant, and to the Avant team who do interesting work and gave us space to write what we hope will be a fun and useful paper. 

By the by, Avant are hosting a very interesting looking conference next year:

THINKING WITH HANDS, EYES AND THINGS. The 1st edition of the International Avant-Conference “Trends in Interdisciplinary Studies”, 8-10 November 2013, Torun, Poland

Evolution vs.Specification (Specification V)

One of Gibson's key contributions was to reveal that it was possible for the optic array to specify a meaningful property of the world. Gibson insisted that specification existed between the world and optics (each property produced one unambiguous pattern, and thus the mapping is 1:1). Specification, said Gibson, meant direct perception was possible, because picking up that one variable meant perceiving the one property that caused it. 

Turvey, Shaw, Reed & Mace (1981) formalised this idea by describing how ecological laws governed which properties of the world could be specified and identifying that these laws allowed affordances into this set. Turvey et al (hence TSM, because Reed changed his mind later on) then insisted that, in order for perception to be direct, specification also had to exist between the optics and the perceiver; an organism should only use one variable per property, and thus the mapping from world to perceiver is 1:1:1. This is a very high bar, and was put in place to defend ecological psychology from the Establishment attack (Fodor & Pylyshyn, 1981).

Withagen & Chemero (2009) think that the 1:1:1 account is incompatible with evolutionary thinking, and they aren't hot on the 1:1 account either. Specifically, they think that any given species will show individual variation in it's members ability to use information, and that in many cases species will end up using sub-optimal solutions (two important elements of evolutionary thinking). The1:1:1 bar, they say, is implausibly high and a naturalised theory of perception (one that is compatible with evolution) will instead predict the common use of non-specifying information. They also claim that this does not stop perception from being direct, so long as you allow 'directness' to live along a continuum.

I think there are some important issues here, but I think this paper's presentation is problematic. It contains no analysis of any particular information or task, and instead is full of sentences such as 'it seems more plausible to us that' and 'it is possible that'. This comes off as the kind of woolly evolutionary thinking psychology is rightly scolded for. Gibson and TSM spent a lot of time trying to make us pay less attention to what might be and more to what is.My concerns are mostly along these lines, and once I get them off my chest I want to turn in future posts to some ideas for a research programme to pursue this all in more detail.

Theory, and Why It's Time Psychology Got One

Psychology has a problem. We have no core theory to guide our research; no analogue to the theories of evolution or relativity. When particle physicists recently found that some neutrinos had apparently travelled faster than light, it never actually occurred to them that this is what had happened. On the basis of the extraordinarily well supported theory of relativity, everyone went 'huh, that's weird - I wonder what we did wrong?', and proceeded to use that theory to generate hypotheses they could then test. It would take a lot of fast neutrinos to disprove relativity.

Psychology, though, when faced with an empirical result that violates the laws of physics, can't find any principled reason to reject the result and instead spends a lot of time squabbling about whether Bem's result might possibly be true because 'quantum'. Worse, when people do replicate the experiment and fail to support the original result, they can't get their 'null result' published. It's a bit embarrassing, really.

One of the problems of having no core theory is that you can't simply rule things out as options. Psychologists almost all consider this a strength: we can pick and choose from a variety of mechanisms which enables us to cope with our messy and erratic subject matter. Can't imagine how perception can explain a result? Just hypothesise a mental representation to fill the gap. After all, no single theory is going to account for the opportunistic and idiosyncratic behaviour of people, so why limit ourselves? We tried that with behaviourism, and it got us nowhere. Let's stay flexible.

Review: Louise Barrett's "Beyond the Brain"

There are a lot of books about embodied cognition. Like psychology itself, the idea that cognition might not all be in the head has fractured off into about 5 distinct flavours, most of them quite annoying and pointless.There are also quite a few books about the thesis of the 'extended mind' (Clark & Chalmers, 1998) ; the idea that cognition quite literally extends into our bodies and environments, in terms of the tools we use and the objects we interact with. It's getting quite hard to find a book on these topics that isn't a) simply rehashing old ground or b) trying to come up with it's own distinct flavour of embodied, extended cognition that fixes some problem of everyone else's. I find this sort of fracturing and endless bickering quite distracting and troublesome, for reasons best expressed by xkcd. I also agree with Tony Chemero, that much of this work is still, sometimes deep down, representational. I think this is because they fail to embrace a sufficiently useful theory of information, such as that proposed by Gibson. So I often find myself reading these books finding nothing new and far too much that's depressing.

Louise Barrett's book, "Beyond the Brain: How Body and Environment Shape Animal and Human Minds" is, refreshingly, not like this at all. Barrett is a psychologist who studies animal cognition and behaviour, and her book does a lot of things very, very well. I'm not planning on a chapter by chapter book club on this, although I may at some point; Eric Charles has posted a few thoughts on the book as well, here, here, here, and here. To cut to the chase: if you are new to the area of embodied cognition, read this book. If you're familiar with the literature but want a clear, well-structured presentation of many of the key ideas, then read this book. If you're bored with the same old examples and want some new, perhaps more convincing examples of embodied cognition in action, read this book. And if you have heard some of the arguments but still think behaviour really comes from the computational activity of our complex brains, then, for the love of science, read this book.

Perception, Action & Dynamical Systems

Over Easter I visited the Center of Functionally Integrative Neuroscience at Aarhus University in Denmark, courtesy of the Interacting Minds group. I gave a talk, got the tour, and met some of the faculty and students - some interesting opportunities for future collaborations, I hope - thanks for the hospitality!

I wanted to lay out the basics of the talk I gave. I took the opportunity to present some ideas that have been developing as I work on this blog, reading Chemero and working on coordination experiments. There is a core of people in Aarhus interested in things ecological, as well as dynamical systems, so it was a good audience to try these ideas out and they seemed to go over well. This is also the sketch of a paper Sabrina and I are going to work on over the summer.

The take home message of the talk was simple - dynamical systems is the right kind of mindset for cognitive science, but it is not a theory of behaviour. Dynamics merely provides the right kind of modelling tools - the form of the model must be based on hypotheses about the specific kind of dynamical systems we are or else they are merely an exercise in data-fitting. Ecological psychology is the right theory, and the Bingham model of coordinated rhythmic movement is currently the only example of a genuinely perception-action dynamical systems model. My thoughts here are largely from my response to Chapter 4 of Chemero (on 'the dynamical stance') and Chapter 5, his initial attempt to use dynamics to serve as a guide to discovery which I think fails and which Chemero then replaces with ecological psychology. The description of Bingham's model comes from here.

Gibson vs Physics: Gibson Wins, at the Ecological Scale

One of the interesting questions that popped out of our discussions with Ken Aizawa about Runeson and the Ames Room is this: did Gibson and his followers banish physics and geometry from his psychology? And if so, is Runeson breaking this prohibition by talking geometrically about the Ames Room? And how can you banish physics anyway - what the hell?

I think I've convinced Ken that there is no such prohibition in the comments at the posts linked to above (I think). But it's a topic of fairly central importance to the ecological approach, so I wanted to summarise some of these ideas and examples here.

Runeson, the Ames Room and the Irrelevance of Equivalent Configurations

Recently we’ve been discussing several topics with Ken Aizawa at his blog, The Bounds of Cognition; there are several posts brewing from the two of us on this, but some travelling (plus, for me, a broken wrist) has slowed things a little.

I want to spend some time talking about some recent discussions on aperture vision and the Ames Room. This post will go over two papers on the Ames Room; the next few will discuss some of the more interesting points about the ecological approach that came out of the discussion with Ken. This is a really rich topic, and there’s a lot of ground. First, though, we need the set up.

Reading Group - Heft (2001) on Gibson

Part II of Heft moves on from James and Holt and into Gibson's ecological approach. The point of Part I, as I've discussed, was to establish the roots of Gibson in the radical empiricism of James, conveyed via the molar behaviourism of Holt. Part II is a detailed discussion of the ecological approach, specifically that laid out in Gibson (1979). I'm therefore going to leave the details of the exposition to the other reading group posts, and focus on the higher level connections and points Heft makes. I'm also going to handle Chapter 3 in two posts: this one will be a brief summary of some key points, and later this week I'll reveal how I've been mistaken about affordances for quite a while.

There is no poverty of stimulus

In my last post I outlined the basic state of affairs in the study of perception up to the modern day. Cognitive science assumes a poverty of stimulus that must be overcome with internal mental representations. The content of these representations is an empirical question, and the drive in psychology ever since the ‘cognitive revolution’ of the 1960s has been to uncover the contents and format of these representations such that they can do the job apparently required.

A little history, by way of introduction

I’m a Gibsonian. I study perception and action from an ecological perspective, which is based in James Gibson’s theories of perception (and Nikolai Bernstein’s theories of motor control). This perspective is, in many ways, in direct conflict with the dominant cognitive paradigm in psychology, and frankly a lot of people simply think it’s ridiculous. This is partly our own fault: the ecological camp is small, a bit insular and prone to picking odd fights. But I think we are right, even if sometimes I don’t think we’re studying it or talking about it right. My goal for this blog is to work some of these thoughts out (amongst other things) so I can actually turn these thoughts into experiments and papers.

The main driver for me doing this right now is an internal argument that is brewing amongst ecological types that I think is a) flawed and b) a waste of time. I’ve been thinking and talking about this conflict for a little while now, but I need to find ways to go after the flaws empirically: I am a scientist, and in spite of all the evidence I really do believe psychology can be a science. The argument is actually yet another round of the only argument that ever happens about perception: what constitutes information for a perceiving organism? The modern Gibsonian approach makes specific claims about this, so before we get into the recent stuff we need some context. That means we need a little history.

A Little History

The fundamental question in perception is, what is the information for perception? What is the form of the proximal stimulus (the thing that actually causally interacts with an organism’s sensory apparatus)? The answer to this question has almost always been that the proximal stimuli are sensations, meaningless physical events (photons for vision, compression waves for hearing, etc). Sensations are meaningless because they do not fully specify the world that caused them; they are ambiguous, because a given physical event gives rise to many patterns of sensations (a small red ball flying towards you produces a different optical pattern than, say, a large blue ball, although they are both the same type of event).

The basic ‘sensation based theory of perception’ runs as follows:

a. Something happens in the world (the distal stimulus). This is what an observer eventually needs to respond to, and so needs to detect it somehow.

b. This event causes a pattern of change in the light, although this pattern only correlates with the event in the world; local conditions, the observer’s perspective, etc, alters many of the details.

c. An observer detects this pattern (the proximal stimulus) which is related to the event in the world but not uniquely.

d. They must then infer what event in the world lead to this pattern (i.e. resolve the ambiguity).

e. The observer then responds appropriately to the event.

Modern cognitive psychology has what is essentially just the latest version of this hypothesis, in which it fills in some of the details. (a) and (b) remain the same: these are taken to be the facts of the matter (and it’s the correlation aspect in (b) that Gibson will object to):

c. The proximal stimulus (for vision) is the ‘image’ projected to the back of the eye, onto the retina. Like a camera, the eye focuses rays of light (photons) to a point (the fovea) and forms an image. The image isn’t really a picture (not in modern theories, although people still talk about the retinal image); it’s a pattern of light distributed across the retina. There are internal correlations between the ‘pixels’ such that neighbouring pixels tend to resemble each other, unless there is an edge, in which case they differ immensely (telling you there’s an edge there). However, this image is ambiguous with respect to what caused it; there is a one-to-many mapping, in which one event could lead to multiple patterns (every visual illusion is an example of this).

d. The job of vision is to resolve this ambiguity, via inference. Essentially, the solution to this problem is that the visual system must make an educated guess about which of the ‘many’ possible events caused the ‘one’ proximal stimulus. The education of the guess comes from mental representations, the workhorse of cognitive psychology. A representation is a pattern of neural activity/connections that ‘stands in for’ (i.e. represents) the missing information. The representation may contain information about past exposure to this pattern and what the event turned out to be (learning); it may contain information about how certain types of correlations tend to indicate one thing or another. Visual perception is the cognitive process of detecting the proximal stimulus, then selecting and applying the correct representations from your repertoire to resolve the ambiguities.

Cognitive theories about visual perception therefore

1. assume a poverty of stimulus, i.e. an ambiguous proximal stimulus that must be enriched with internally stored information, and

2. are about uncovering the contents of the representations and how the correct one(s) are selected and used.

This is the state of things today, and is the essential form of the argument that James Gibson rejected with his theories. Gibson’s move was simple: he considered (b) from above, and rejected it. If the proximal stimulus only correlates with events in the world, he said, then perception is doomed to failure – how do we ever build correct representations? How could we possibly select the right ones? Gibson’s solution was to rethink information, and propose that events in the world can, in fact, lead to changes in patterns of light that uniquely specify the event that caused them, rather than merely correlate with. He also argued that it couldn’t be any other way, if perception was to ever work the way we experience it working every day. The next step is to consider some details of Gibson’s alternative.