What Does it Mean for Perception to be 'Direct' vs 'Indirect'?

The ecological approach is a theory of direct perception. Put simply, direct perception proposes that our perceptual experience of the world is not mediated by anything that sits between the world and that experience. Making this viable is a big challenge, however, and the idea seems preposterous on the face of it to many people. 

In this post, I want to lay out the basic idea of direct vs indirect perception, and then explain how each approach addresses the problem of how we perceive the world. There will be some big words, but I will aim here to place them in a context that supports them usefully, and focus mainly on the straight-forward ideas in play. In a future post, I will ask the question 'what do we know about how plausible direct and indirect theories of perception are?'

Muscle Homology in Coordinated Rhythmic Movements

One of my main experimental tasks is coordinated rhythmic movement. This is a simple lab task in which I ask people to produce rhythmic movements (typically with a joystick) and coordinate those at some mean relative phase. Not all coordinations are equally easy; without training, people can typically only reliably produce 0° (in-phase) and 180° (anti-phase) movements. People can learn other coordinations, however; I typically train the maximally difficult 90° (although my PhD student has just completed a study training people at 60°; more on that awesome data shortly). I use coordination to study the perceptual control of action and learning.

My work is all designed to test and extend Bingham's mechanistic model of coordination dynamics. This model explicitly identifies all the actual components of the perception-action system producing the behaviour, and models them. In particular, it models the perceptual information we use to perceive relative phase; the relative direction of motion. This is an important contributor to coordination stability and this model is a real step up in terms of how we do business in psychology.

There is another factor that affects coordination stability, however, and the model currently only addresses this implicitly. That factor is muscle homology, and it's been repeatedly shown to be an important factor. For a long time, I have avoided worrying about it, because I have had no mechanistic way to talk about it. I think I have the beginnings of a way now, though, and this post is the first of several as I develop my first draft of that analysis.

The Shrunken Finger Illusion

Ed Yong has a great write-up of an interesting little study in Current Biology (Ekroll, Sayim, Vander Hallen & Wagemans, 2016) that caught my eye. The study reports an illusion (the 'shrunken finger illusion') that shows how amodal volume completion can make you feel like your finger has shrunk, and everyone is very excited about how this shows our experience of the hidden back-sides of objects is "real".

In this post, I'll review the results, do a little ecological finger wagging about the breathless write-up (Ekroll's, not Ed's) and think about some studies the ecological reframing of the effect might motivate. Briefly, I think this effect is definitely real, and that we really do genuinely perceive hidden objects under certain circumstances. Of course, this has nothing to do with amodal mental representations of what we think is there and everything to do with the information the system is interacting with, but you know that of course because this is always the answer!

The Interface Theory of Perception - The View from Ecological Psychology

Psychonomic Bulletin & Review has released, with much fanfare and a hashtag, an article called 'The Interface Theory of Perception' (Hoffman, Singh & Prakash, 2015). From the website description:

In a nutshell, interface theory postulates that our perception operates like a species-specific desktop: We perceive the world in representations that do not represent the “truth” about the world as it actually is, but that are useful “icons” which represent fitness-relevant information about the world. To illustrate, imagine a world in which red and green berries are nutritious but blue and yellow berries make you sick. Will your perceptual system differentiate red from green and blue from yellow? According to interface theory, the answer is no—the organism will have evolved to differentiate between only two colors, namely gred and byellue.

The meat of the paper is a series of evolutionary simulations that pit various perceptual strategies against one another. These strategies vary in how veridical they are, and the key result is that interface strategies, in which perception codes things in a way that bears no resemblance to the world, wins every time. We do not perceive the world as it really is. 

This seems to go against people like Gibson, who argue that perception is of a real world and real properties of things, like affordances. These simulations seem to show that 'realist' perceptual strategies are evolutionarily unsustainable. 

The devil, as always, is in the details, and having read the paper I am now pretty sure that Gibson is quite safe, and that information offers a path out of the weirdness Hoffman conjures.

The Perturbation Experiment as a Way to Study Perception

When you study perception, your goal is to control the flow of information going into the system so that you can measure the resulting behaviour and evaluate how that information is being used. There are two ways to do this, one (sometimes) used by me, one used by, well, everyone else. In this post I'm going to compare and contrast the methods and describe why the perturbation method is what we should all be doing.

The standard method is to present experimentally isolated cues and test whether people can detect those cues. The perturbation experiment presents a 'full cue' environment but selectively interferes with the link between a single variable and the property it might be information about. These two different methods lead to very different ways of thinking and talking about perceptual abilities. 

A taxonomy of information

Over the past several months I've been thinking about how perception falls within a hierarchy of types of information use. This was spurred by my ideas about an ecological approach to language, in which perceptual information and linguistic information are distinguished on the basis of the relationship between event structure and meaning. As part of this work, I defined perception as the apprehension of structure in an energy array where 1) the structure is specific to an event or property in the world, 2) where the meaning of the structure (for that organism in that task) is about that event or property (i.e., a dog's bark is about the event of a barking dog), and 3) where the meaning of the structure must be learned (or, more correctly, where an organism must learn how to coordinate action with respect to this structure). I arrived at this definition because it seemed to capture the ecological approach to perception and because it makes it obvious how perceptual information and linguistic information differ (also because I am crazy-obsessive about definitions).

Breaking the (ecological) law: why illusory sounds don't make for safer cars

Electric cars are great for the environment, but they come with a problem - they're too quiet, and this makes them dangerous to pedestrians used to the loud noises of the internal combustion engine. One idea is to add noise to the cars, and Mark Changizi recently wondered whether perceptual psychologists could help design a better sound, using an illusion. I don't think it would work, and the reason is a nice example of why it's important to understand the relationship between the world and perceptual information about the world.

Events in the world create information. When a car moves towards you, for example, there are sounds coming from the engine, and the way these sounds change over time is information about the heading of the car. If it's getting louder over time, it's coming towards you; but if the pitch is decreasing at the same time, then the car is going to pass you, not hit you. Increasing volume and constant pitch, however, specifies a collision and the particular rate of change of these variables tells you about the time-to-contact. These patterns of change over time in the acoustic array are related to the way the car is moving via the laws of physics and are therefore informative about the details of the car's motion. Importantly, the relationship between these patterns is also governed by the laws of physics; increasing volume and constant pitch only go together when the car is really heading towards you, for example.

Every perceptual system has limits, though, and those limits are called thresholds. When a stimulus is outside a threshold (e.g. too quiet, or too high pitched) the perceptual system won't respond to it.  The problem with electric cars is that they are quiet at low speeds, and so the variation in pitch you need to detect a collision is small and possibly below threshold. You could make the cars louder, but this goes against one of the selling points of the cars - reduced noise pollution. Mark suggested creating an artificial sound, an illusion, in which the range of the variation in pitch created by the slowly moving car is amplified without just making the car louder. This is in principle possible, but it's a problem because it's breaking the law, and you end up with less information about the car than when you started.

Language: A task analysis (kind of)

In the last post, I discussed the similarities and differences between language and other types of information. From the first person perspective, spoken language is just another type of auditory event. The main distinction between the word "dog" and the sound of a dog barking is that the auditory event of barking is about the thing that caused the sound - a barking dog - while the auditory event of the word "dog" is not about the thing that caused the sound - a human speaker. The word "dog" is (usually) about an animal that is related to the auditory event by convention. Thus, the sound of a barking dog conveys auditory information and the sound of the word "dog" conveys linguistic information.

In this post I want to lay out classes of tasks in which linguistic information is useful. As a starting point, I will identify situations where language appears to fill a gap, although at this point these are no more than general descriptions. In any specific task analyses that might eventually follow, the basic strategy will be to begin by asking what perceptual resources exist to carry out the task. If perceptual resources are unable to explain task performance and if linguistic resources are available, then these will be considered for their potential contribution. It might be helpful to think of linguistic and perceptual information as occupying different niches in a task space. Perceptual information helps me to walk and catch a fly ball and linguistic information helps me do the types of things described below.