Lecture 23: Ecological Optics Primer (Turvey, 2019, Lectures on Perception)

The 'barrage of conceptual details that was Lecture 22' has provided us with the notion of affordances, and these are to be the objects of both perception and action. This means the need to be perceived, and this brings us to the second part of the ecological ontology - information. This chapter will focus specifically on optical information, identify the key characteristics of an optic array, and walk through some ways of analysing perception ecologically by dealing with some problems supposedly raised by illusions. 

Are Illusions Even a Thing?

Traditional vision science is very excited about illusions. These are cases when perception seems to break down; there is a mismatch between what is out there and what we experience, and traditional approaches consider these breakdowns as clues to how vision has to work, given what it is working with. 

Ecological psychologists don’t like illusions. Typically, they occur when information is either made ambiguous or faked, and in general we think these are the wrong situations to study perception in. We sometimes engage with the literature on these effects, but usually to show how the trick is the result of not thinking ecologically. 

Rogers (2022) has taken this basic analysis but gone one interesting step further. He’s argued that the notion of ‘visual illusion’ is simply not a clear category; it’s not a useful way to describe any of the effects people study. He argues that there simply is no sufficient definition of what an illusion is that works, and that what we call illusions are just either tricks (as above) or inevitable consequences of how the visual system works. 

I am broadly on board with this additional step, and it’s made me think hard about what illusions are and how best to respond when people use them against direct perception. 

The Ecological Approach to Virtual Reality

As virtual reality (VR) gear gets better, cheaper, and easier to use, there is renewed interest in trying to figure out how best to make a virtual environment feel real.  The typical for framing for VR is in the name: it's about creating the illusion of reality, a virtual world. Programmers even talk this way; they describe creating virtual (pretend) environments, objects, spaces, etc. From this point of view, VR is an attempt to create a stable illusory experience by faking a world that doesn't really exist. 

Of course, VR programmers don't make worlds; they make information. This makes folding VR into the ecological approach a natural move, and I propose that ecologically, VR development is actually an attempt to design an optic array containing information that can support certain behaviours. It's less virtual reality, and more virtual information. This is important because the nature of the information we are using explains the form of the behaviour we are controlling. Your goal as a developer is therefore not to create tricks and illusions, but to provide information capable of supporting the behaviours you want to be possible,

As a first step towards an ecological understanding of VR, I will first follow the path Gibson laid down taking the science of perception away from illusions and towards information. I'll then think about some of the implications of taking an ecological approach for VR design. Virtual reality needs our theory of perception to become the best it can possibly be, and I hope that this post serves as an entry point for designers to become aware of what we have to offer them.

Visual illusions and direct perception

A while back I reviewed a bunch of papers by Rob Withagen who is currently arguing that while perception is not typically based in specifying variables, it can and should still be ecological in nature. While we are also developing an account of information that goes beyond specification I still have some reservations about the details of Rob's work. That said, I do think there is a lot of overlap and I'm still interested in figuring out what that is. 

Rob's latest paper (de Wit, van der Kamp & Withagen, 2015) is about visual illusions, and how to talk about them ecologically. There is a lot of very useful stuff in here, not least the review of the many things Gibson said about illusions. After this review, the paper tries to put all of this into Rob's and Tony Chemero's evolutionary framework and uses this to formalise and tidy up what Gibson was up to. I'm adding this paper to the set I've covered from Withagen as I continue to think through these issues.

Breaking the (ecological) law: why illusory sounds don't make for safer cars

Electric cars are great for the environment, but they come with a problem - they're too quiet, and this makes them dangerous to pedestrians used to the loud noises of the internal combustion engine. One idea is to add noise to the cars, and Mark Changizi recently wondered whether perceptual psychologists could help design a better sound, using an illusion. I don't think it would work, and the reason is a nice example of why it's important to understand the relationship between the world and perceptual information about the world.

Events in the world create information. When a car moves towards you, for example, there are sounds coming from the engine, and the way these sounds change over time is information about the heading of the car. If it's getting louder over time, it's coming towards you; but if the pitch is decreasing at the same time, then the car is going to pass you, not hit you. Increasing volume and constant pitch, however, specifies a collision and the particular rate of change of these variables tells you about the time-to-contact. These patterns of change over time in the acoustic array are related to the way the car is moving via the laws of physics and are therefore informative about the details of the car's motion. Importantly, the relationship between these patterns is also governed by the laws of physics; increasing volume and constant pitch only go together when the car is really heading towards you, for example.

Every perceptual system has limits, though, and those limits are called thresholds. When a stimulus is outside a threshold (e.g. too quiet, or too high pitched) the perceptual system won't respond to it.  The problem with electric cars is that they are quiet at low speeds, and so the variation in pitch you need to detect a collision is small and possibly below threshold. You could make the cars louder, but this goes against one of the selling points of the cars - reduced noise pollution. Mark suggested creating an artificial sound, an illusion, in which the range of the variation in pitch created by the slowly moving car is amplified without just making the car louder. This is in principle possible, but it's a problem because it's breaking the law, and you end up with less information about the car than when you started.

The Size-Weight Illusion is Functional, and It's About Throwing

My colleagues, Geoff Bingham and Qin Zhu, have recently published some fascinating data which has emerged from their work on the uniquely human skill, long-distance throwing. This is a novel and rich perception-action task which Bingham and Zhu (and recently, me) have been investigating for some time, with many interesting results. I'll get onto blogging about this project once I've caught up with the coordination studies and have had some time to get my head around the data I'm helping generate.

I wanted to blog about this new paper, though, because it's an exciting result which deserves all the attention it gets. The result is about the size-weight illusion, one of the most robust illusions around. As I've talked about before, illusions are a concern to ecological psychologists only in that they suggest the task has been incorrectly characterised. This paper presents data that suggests the size-weight illusion is actually functional, and that it reflects the readiness of the human perception-action system to throw objects long distances.

This paper has seen some activity in the popular press already (e.g. here and here): Geoff's hoping for the NYT Science section too! 

UPDATE: Geoff being interview on NPR

Runeson, the Ames Room and the Irrelevance of Equivalent Configurations

Recently we’ve been discussing several topics with Ken Aizawa at his blog, The Bounds of Cognition; there are several posts brewing from the two of us on this, but some travelling (plus, for me, a broken wrist) has slowed things a little.

I want to spend some time talking about some recent discussions on aperture vision and the Ames Room. This post will go over two papers on the Ames Room; the next few will discuss some of the more interesting points about the ecological approach that came out of the discussion with Ken. This is a really rich topic, and there’s a lot of ground. First, though, we need the set up.

Visual Illusions Again

VSS is the Vision Science's Society conference that's held every year in Naples, FL. I couldn't make it this year, but they have a contest for the best new visual illusion that's usually a pretty good show; illusions are getting cleverer.

This is this year's winner:



There's actually visual information that something is wrong that's available before they rotate the display - 5 internet points to anyone who points it out!

A Note on Holt on Visual Illusions (Heft, 2001)

This is a visual illusion. When you look at it, there appears to be a great deal of motion as things rotate in various directions. But there's no actual motion; this isn't an animated display and you can verify this by fixating the centre or touching your computer screen - no motion. I just fooled your visual system.

But Andrew, you say, you've been talking all this time about how great perception is! It's direct, functional, reflects experience, and lawful: if perception is all these things, how can it be fooled so easily by some pixels? Direct realism surely cannot handle this: perception is flawed, and readily shown not to correspond to the way things really are. Clearly this James Gibson is a mad man and we can go back to 'knowing' that perception is clearly mediated by pretty good but fallible representations.