Lecture 24: Perceiving "How to Get About Among Things" (Turvey, 2019, Lectures on Perception)

This chapter focuses on the perception side of the perception-action system we will end up with for behaviour. It covers the objects of perception (affordances), the means of perception (law-based information, informationL), and the notion of prospective control (vs. anticipating or predicting the future). 

The chapter title is 'how to get about among things'. This stands in contrast to how things are normally framed, in terms of distance or space perception. Perception is for action, it is about enabling safe, skilled activity in a cluttered world. Because of this, the objects of perception (what perception is about) must be action related - affordances. For Turvey, affordances are properties of the environment (not relations between environment and organism) and they are perceived if and only if there is informationL about that affordance present and detected. 

Turvey then introduces the idea that perception and action are duals that define a duality (rather than the beginning and end of a linear chain of causation). Note this is a duality, and not a dualism - the two things are not different in kind, but are parts of a greater whole that are intimately and lawfully connected to one another. This implies a relationship between what information specifies, and the energy required to move with respect to what it specifies. The main idea here though is that it is not perception then action, it is only perception-action. 

Turvey then spends time on the optical basis for getting about among things; ecological optics. Ecological optics is not physical optics (which is about things like wavelengths), nor is it physiological optics (which is about the anatomy and physiology of the eye). Ecological optics is about what is available in the light prior to any interaction with an organism, and therefore what is available to stimulate the retina, etc. The key is optic flow: structured light distributions incident to illuminated surfaces. 

The optical structure available at a single point is specific to the stationary surface layout that gave rise to that structure (this is why the Ames Room works, for example). Optic flow is the general case; optical structure in which the transformations are specific to the surface layout and the relative motions of the observer and the environment. Turvey works through the example of tau in some detail, to illustrate how movement generates an optical flow field and how that field is structured specific to the motion. He then briefly reviews more complicated work on other geometrical features of optical arrays (transformations such as divergence, curl, and deformation; Koenderink, 1986). These are examples of the kinds of maths required, and the target of that maths (the structure of the optic array). Turvey also nods to Stoffregan's notion of a global array (although I will note that this hasn't really gone anywhere yet and it's not clear what the medium of this array is). 

Finally, Turvey reviews how optical information can support prospective (future oriented) control. In general there are two approaches to dealing with the future. We have encountered the idea of prediction/anticipation, and the various problems with that. In short, it is a form of induction and it suffers from all the problems of that. Instead, informationL will allow prospective control, via specification of what will occur, if present conditions continue (Bootsma calls this 'the current future'). Turvey of course looks to the outfielder problem as the key example here. 

(I will note here that this topic shows up in the modern literature in the form of information-based control (Bill Warren) vs affordance-based control (Brett Fajen). Turvey nods to Fajen and acknowledges that the information used for control should be about affordances; but the debate is not engaged with here.)

Finally, as a prelude to the next chapter, Turvey notes again the idea that all this information is dual to/reciprocal with the action side of the equation. 

The Ecological Approach to Virtual Reality

As virtual reality (VR) gear gets better, cheaper, and easier to use, there is renewed interest in trying to figure out how best to make a virtual environment feel real.  The typical for framing for VR is in the name: it's about creating the illusion of reality, a virtual world. Programmers even talk this way; they describe creating virtual (pretend) environments, objects, spaces, etc. From this point of view, VR is an attempt to create a stable illusory experience by faking a world that doesn't really exist. 

Of course, VR programmers don't make worlds; they make information. This makes folding VR into the ecological approach a natural move, and I propose that ecologically, VR development is actually an attempt to design an optic array containing information that can support certain behaviours. It's less virtual reality, and more virtual information. This is important because the nature of the information we are using explains the form of the behaviour we are controlling. Your goal as a developer is therefore not to create tricks and illusions, but to provide information capable of supporting the behaviours you want to be possible,

As a first step towards an ecological understanding of VR, I will first follow the path Gibson laid down taking the science of perception away from illusions and towards information. I'll then think about some of the implications of taking an ecological approach for VR design. Virtual reality needs our theory of perception to become the best it can possibly be, and I hope that this post serves as an entry point for designers to become aware of what we have to offer them.

Framing the Debate

In 2014 we published a book chapter with Eric Charles in which we argued that the most important thing psychology and neuroscience needed from people like us was a new language in which to talk about the problems we are trying to solve. Our Ecological Representations paper is part of this, and we have a much larger paper in development laying out the more complete set of conceptual tools needed to do ecological psychology across a wider range of problems.

One reason why this is important is a simple fact; we are asking psychology to change and it is up to us to clearly articulate what we want it to change into, or else nothing can happen. A related reason is that without a clear framework, we can't reformulate the questions in a useful way and we're left stuck because we can't explain something like 'theory of mind' because the actual solution is that ToM doesn't exist or need explaining. Ecological neuroscience, for example, will look very different to cognitive neuroscience.

A final reason is that the language in which psychology frames it's understanding of behaviour drives popular understanding of behaviour too. I recently came across my favourite example of this in a tweet by Alice Dreger;

My dominant eye is very blurry so my left eye has come online (it rarely does) and now I'm seeing 3D. 3D is SO WEIRD. How do you stand it?

— Alice Dreger (@AliceDreger) July 19, 2016

Dreger, for some reason, spends most of her life only using her right eye, even though her left is perfectly functional. She blogged about it here. Every now and again, something makes her left eye kick in and she suddenly has stereo vision.

What caught my eye here is her description of her experience is grounded in the myth that you need two eyes in order to perceive in 3D (I bug my students about this in class every year too). The myth is based in the standard image-based analysis of vision which I'll lay out below; but the point I want to make here is that people still describe their experience of monocular vision as 'not being able to see 3D/depth' even though this is inarguably, demonstrably not what is happening in their visual experience. It's like blind echolocators talking about how the sound creates 'an image in their minds'; this is just not the case, but this is the language psychology has provided them for talking about the perceived experience of spatial layout. What fascinates me is that it's trivial to demonstrate that monocular vision allows for 3D perception, but everyone lets the framing override their own experience. This, to me, is a big part of why our work right now is important - we will never make progress until we can reframe the debate.