Does Interface Theory Have Consequences for the Ecological Approach?

I've been working on a commentary about interface theory (Hoffman, Singh & Prakash, 2015) which I have previously blogged about here. I'm still interested because it is, in part, a fairly direct shot at the ecological approach and I'm always keen to break those if I can. My piece stalled out, though, so I thought I'd spend some time here thinking out loud about the argument that stalled and another critique that came up as I re-read the paper.

To unbury the lede I just finished writing: the primary thing Hoffman et al get wrong about Gibson is that they think he wanted his theory to produce veridical perceptions, not simply adaptive ones. Gibson actually wanted adaptive perceptions, but found a way in which they were also veridical. This emphasis matters; Gibson does not stand or fall on issues of veridicality. In addition, every one of Hoffman et al's big swings apply only to inferential, constructivist theories of perception; Gibson is immune on these grounds as well. All Hoffman et al have done is redraw the terrain inferential theories have to traverse, and it will be interesting to see if anyone takes the bait. But the major argument simply remains, is perception inferential or ecological, and may the best data set win.

Attack 1: Hoffman et al mischaracterise the ecological approach

I've tried developing the idea that Hoffman et al have mischaracterised the ecological approach as overly direct; our perceptual experience of the (dynamical) world is mediated by our detection of (kinematic) information, and it occurred to me that this kind of makes information an interface. I'm not quite sure this works, however.

Hoffman classifies Gibson's approach as an example of a naive realist' strategy, which is 'a perceptual strategy for which X [perceptual experience] ⊂ W [the world] and P [the perceptual mapping] is an isomorphism on this subset that preserves all structures on W' (pg 1483). He talks about W in terms of affordances, which isn't quite correct, but this classification holds up if perceptual experience is a veridical mapping of information as the subset of the world, which is what the ecological approach actually claims. For example, the behavioural structure of coordinated rhythmic movement maps onto the structure of the information variable relative direction, not the world variable relative phase, which is evidence that perceptual experience as measured by the action is isomorphic to the information subset of the task. So Gibson is indeed a naive realist, by this definition.

That's not to say Hoffman et al understand Gibson. They just rule him out because they claim 'evidence for information processing is now overwhelming', but there's quite a lot of evidence for perception via information too, so that doesn't help. He also cites Fodor & Pylyshyn (1981) when they say Gibson fails because he can't handle illusions; Turvey, Shaw Reed and Mace (1981) have plenty to say here, of course (Section 8, 'Misperception Misconstrued'). So their glib dismissals don't really add much and don't need more attention.

Attack 2: The results of interface theory only apply to inferential theories of perception

The second objection I have to interface theory is that it seems to only kick in if perceptual is an inferential process, and if we have the need and ability to create any inferential mapping that works out. If perceptual learning is about building an internal, representational system that maps sensing the world into perceptual experience, then it makes sense to test out various mapping strategies to see which works best. This would serve as an invaluable constraint on our representational theories of perception. Of course, the ecological approach explicitly rejects this idea and proposes the alternative theory that perceptual learning is about attuning to specifying information already present in the world. So this might be the end of it; the ecological approach simply denies that the interface analysis applies to it and moves on.

Is There Space and Time, Though?
There's a twist, though. The ecological analysis hinges on the hypothesis that information can specify the world because that information is related lawfully (via the ecological laws; Turvey, Shaw, Reed & Mace, 1981). I can use information to perceive the world because when I try to 'track back' from information to the world it works, and it works because the two are lawfully (non-accidentally) connected. Hoffman et al have an analysis that goes to the heart of this and seems to suggest it's impossible.

The Problem: Structure in perceptual experience tells you nothing about the structure of the world
Part of interface theory is the Invention of Symmetry Theorem  (see Figure 1) and it's corollary, the Invention of Space-Time Theorem.

Figure 1. The Invention of Symmetry Theorem, formally expressed in a way I don't fully understand

Translated into English,

Russell claims that if a feature of our perceptions is invariant under these group actions, then it can be taken as veridical.

This claim is false. The following theorem shows that the world itself may not share any of the symmetries that the observer observes. The world need not have the structure the observer perceives, no matter how complex that structure is and no matter how predictably and systematically that structure transforms as the observer acts.

...

An observer’s perceptual experiences can have a rich structure, e.g., a 3D structure that is locally Euclidean, and that transforms predictably and systematically as the observer acts, but this entails absolutely nothing about the structure of the objective world.

pg 1498

Apparently, this theorem proves that just because my perceptual experience has a particular structure, that doesn't entail that any part of the world does. If true, this kills the ecological approach, which depends on the lawful relation between information and the world to work.

The Defence: It still only applies to interfaces
I do not believe for an instant that the interface theory has uncovered anything incredible about the nature of the universe. This is not a theory of physics (sorry, Greg). So this theorem cannot possibly be saying 'it is impossible to track back from perception to the world'. Interface theory is a theory of perception, specifically an inferential, constructivist, representational theory of perception. This theorem must be saying that "it is impossible to track back from an interface to the world"; your interface can be richly structured, but this doesn't say anything about the nature of the world because you build interfaces to be functional, not veridical.

Luckily, information is not an interface. It is not constructed by us; its form is not optional; and so long as there is a world that can be measured by a perceptual system at an ecological scale (which I think I'm proving by typing just now) we can work to understand the mechanism of that measurement and expect the relevant laws to apply.

So What Do Hoffman et al Get Wrong About Gibson?

Interface theory claims to have examined all possible perceptual strategies. However, it has only examined all possible ways to construct an interface. They never tested the perceptual strategy 'attune to information created by the operation of ecological-scale laws of physics'. 

When discussing illusions, Hoffman et al make a useful point:

The basic mistake of the textbook theory is its claim that selection shapes perceptions to be true. This forces illusions to be departures from truth. The correct claim is that selection shapes perceptions to guide adaptive behavior.

pg 1499

They think Gibson claims the former. Actually, he claims the latter, and then identifies a perceptual. mechanism that supports adaptive behaviour. Adaptive behaviour is behaviour that complements the task demands. More formally, the dynamics of behaviour need to be coupled to the dynamics of the task so that the way behaviour evolves over space and time is shaped by the way the task evolves. Task dynamics create specifying information, and the dynamics of behaviour track the behaviour of that information. 

Evolution selects for perceptions that can guide adaptive behaviour. One way to do that is to construct an interface according to the rules of this paper. The other way is to attune to information that can support adaptive behaviour. The central question remains - is perception inferential, or ecological, and interface theory has not changed that. 

Summary

My article has hit a wall when Attack 1 deflated, so this post was me doing some more thinking about just what the hell it is that's wrong with the interface theory, from the ecological point of view. I think I got it right the first time - interface theory only applies to inferential theories of perception, so as long as we are fighting those off, the game remains afoot. I'm glad to have taken the time addressing the 'interface theory says there's not space and time' argument, though; comments on that welcome.