The medium for direct perception (Notes on Van Dijk & Kiverstein, 2020)

The ecological approach has hit a point in its history where it has become interested in expanding its scope, to go beyond the real time coordination and control of action. There are many challenges from non-ecological cognitive science about how to tackle representation-hungry problems, and how to conceptualise things like language, social behaviour, and what the brain is up to. I am all on board with this move - it was important we waited till we were ready, but since Gibson died in 1979, the empirical programme on the basics has matured into a solid foundation and we have a lot of developed or adopted a lot of things that will come in useful. 

However, if we are going to do it, I want us to do it with rigour and care and with reference to all our hard-won successes. My current view is that our best path lies in looking at the ways we are able to use ecological information, and grounding our explanations and hypotheses at this scale. Sabrina first developed this idea in a paper about how to think about what language is (Golonka, 2015). The big take-home from that paper is the analytic distinction between law-based use and convention-based use of information, and the first draft of the consequences of this distinction. We built on this when we started thinking about brains (Golonka & Wilson, 2019), and I'm currently thinking about the next step along this path. 

I'm pretty sure that a big chunk of the work I need to do is explicitly connecting this distinction up to work on the skilled intentionality framework, and the notion of our variable levels of grip on the field of affordances. This work is wrong about affordances (they aren't relations) but other than that, there's a ton of really great work about how intentionality isn't an all-or-nothing thing, and a lot of really useful vocabulary and framing development that I think will be useful for articulating these ideas. I don't like re-inventing wheels, so I'm skilling up on this literature as I develop ideas for a paper. 

This post is about a recent paper (Van Dijk & Kiverstein, 2020) that is explicitly about developing a usage-based notion of information. To unbury the lede, I think this is a robust piece of work with solid internal logic, but I think like all this enactivist style work, it ends up in a place that cannot support a how-actually explanation of behaviour - this particular usage-based theory of information and the things that come with it aren't the framework that will let the ecological approach expand its scope. This is ok, at one level, because I don't think mechanisms are the goal of enactivist analyses. But it's a worry at another level, because I want an ecological theory of direct perception that can actually explain behaviours and this isn't going to cut it. 

Is Direct Perception Plausible? Ecological Information

We're in the home stretch of working through how direct perception is, at least, an option, and how the ecological approach in particular attempts to make it work. We've talked about what direct vs indirect means, the kinds of properties direct perception needs to be out there in order to work, and affordances/effectivities-as-dispositions as the specific properties ecological psychology claims are out there and fit the bill. 

I ended that last post by highlighting ecological psychology had one last thing to do in order to be plausible, and that is to have a way to bring affordances and effectivities together into a kind of contact that allows them to work together. That contact can't be mechanical, or simply physical proximity, because almost all of the things we perceive and act with respect to are not in that kind of contact. The ecological solution is informational contact, and so this post will build on the pieces I've assembled to identify what kind of thing ecological information has to be in order to work. 

Lecture 3: Direct Perceiving, Indirect Perceiving (Turvey, 2019, Lectures on Perception)

In this lecture, Turvey provides a formal definition of what it means to claim a theory of perception is direct vs indirect. A theory of direct perception invokes lawfully specifying information, while all indirect theories invoke at least one mediating physiological or psychological process. Direct theories are allowed to discuss internal states, etc (Gibsonian neuroscience isn't a contradiction in terms), but these states are not allowed to alter information so it is no longer lawfully related to the environment. The big Turvey word we will learn about in this chapter is impredicative entailment - exciting! I'll also briefly point to some implications this chapter has for my recent papers with Sabrina on neuroscience and mechanism.

The Information for Progressive Occlusion

Gibson's ecological psychology is weird, if you are coming from a more traditional information-processing background. The two approaches make radically different assumptions about the nature of the world to be perceived; they have radically different ontologies. This means that there is little if any useful overlap in the way they do things, and communicating across the gap is very hard. I have a recent paper - preprint here - where I go into detail about the two ontologies as I defend ecological psychology from interface theory. It's essentially Turvey et al, 1981, but that's a bear of a read if you aren't already ecologically minded. Do mine first :)

Anyway, concrete examples help. My go-to is the outfielder problem but people are tired of that one. My other favourite is progressive occlusion (Gibson, Kaplan, Reynolds & Wheeler, 1969; Kaplan, 1969). Gibson worked this example up himself in great detail and so it stands as a nice concrete example to illustrate some elements of the ecological ontology. Given the recent total solar eclipse, it seems like the right time to blog it!

This post will review occlusion, talk about how it works and work with some demos. These are all linked from here; there is Matlab/Psychtoolbox code to run a demo, a video of that running and a Powerpoint with some slides. I'll refer to these throughout - occlusion is a dynamic process and so you need to see it moving for it to make sense.

Is the Ecological Approach Radical Enough?

Radical enactivists do not just want to get representations out of their explanations for our mental life. They also want to get rid of the notion of content. Hutto & Myin (2013) is the strong version of this claim. 

Mental states or processes have content if there are specified conditions of satisfaction and if it can be evaluated for things like truth (i.e. does the thing conveying content doing so accurately or not?). Part of the concern is that ecological psychology is committed to content and thus can’t play with the other radical theories. The evidence is that we talk about things like ‘information about affordances’; that ‘about’ implies content. 

van Dijk, Withagen & Bongers (2015) took a swing at defending a content-less ecological psychology. I admire the attempt to get the radical camps on the same page, but at the end of the day I think a) the defence is grounded on the wrong notion of affordances (as relations, instead of dispositions) which means b) I don’t think it works but that c) I don’t think I care. I am as yet unfazed by critiques of content, although I’m happy to hear more on this; frankly Hutto & Myin’s book is a real struggle to read and any clarity people can add, I’ll take. 

From Specification to Convention (A Purple Peril)

I previously laid out how specification works and why it's important to the ecological approach. Read that first, because I build on it a lot here. I also laid out the corollary of specification, that it allows that information to come to be something an organism can actually use to coordinate and control functional behaviour. Here, I think out loud about how convention might be able to do similar work, because of Sabrina's work (here, and published now here; read that paper for the extended detail on this) expanding ecological information to handle tasks such as language where specification is not always an option.

This is in part an attempt for me to get my head around some implications of Sabrina's analysis. My plan here is to develop an analogy to specification. This analogy will detail the work specification does to make information informative, then try to lay out how conventions fills this role. The goal is to see if conventions can support behaviour without needing representational help. The answer will be yes, because all the differences between law based and convention-based information are 'behind the curtain', only visible from the third-person perspective. From the first person perspective of the organism, all it gets is structures in energy arrays it can try to use to organise behaviours. Conventions place no special learning burden on the organism (Golonka, 2015) and this means that convention can support behaviour with any representational enrichment the same way specification can. (The hidden differences do have consequences, however, so I will map that out a bit.)

There are many things I have not attempted to explain and as usual this reflects my current thinking, not necessarily my final thinking. I look forward to hearing what questions this leaves unanswered for the reader as a way to move this discussion forwards.

Specification and Some of Its Consequences (A Purple Peril)

Perception is how we maintain psychological contact with functionally relevant objects and events in our environments. Explaining how we do this means describing that environment in appropriate terms and investigating what information might possibly exist for that environment, given that description. The ecological hypothesis is that the correct level is dynamics, and that describing the environment this way allows there to be information that can specify those dynamics. This information can support the kind of behaviour we need to exhibit. 

This Purple Peril describes what is meant by specification, and what that implies for how information comes to mean something to an organism. There is more detail in the various links, so check those for information too.

What Kind of Thing is an Information Variable? The (Annoying) Case of Tau

The central contribution of the ecological approach is the idea of ecological information. Information variables are higher order relations that remain invariant over time as the elements of the pattern change. These relations are the kind of thing that can specify a dynamical property of the environment and support direct perception of that environment. 

This is all a little abstract. One nice, simple example is the variable tau which specifies time-to-contact (TTC). TTC is an important property of objects approaching you that, if perceived, would support you intercepting or avoiding the object. Tau is one variable that specifies TTC and therefore might get used by organisms to perceive TTC.

This post will use tau as an example of information because it's straight forward and has lots of the relevant key features. However, tau is a pain in the ass because organisms typically don't actually use it - it's too limited in its scope to be the best information. People discussing this fact sometimes says it reveals a weakness in the ecological approach. It doesn't; it just reveals a weakness in tau (and the error ecological psychologists made getting as excited as they did about it as an exemplar). It highlights a lot of useful issues, though, so I thought it was still worth the post.