The medium for direct perception (Notes on Van Dijk & Kiverstein, 2020)

The ecological approach has hit a point in its history where it has become interested in expanding its scope, to go beyond the real time coordination and control of action. There are many challenges from non-ecological cognitive science about how to tackle representation-hungry problems, and how to conceptualise things like language, social behaviour, and what the brain is up to. I am all on board with this move - it was important we waited till we were ready, but since Gibson died in 1979, the empirical programme on the basics has matured into a solid foundation and we have a lot of developed or adopted a lot of things that will come in useful. 

However, if we are going to do it, I want us to do it with rigour and care and with reference to all our hard-won successes. My current view is that our best path lies in looking at the ways we are able to use ecological information, and grounding our explanations and hypotheses at this scale. Sabrina first developed this idea in a paper about how to think about what language is (Golonka, 2015). The big take-home from that paper is the analytic distinction between law-based use and convention-based use of information, and the first draft of the consequences of this distinction. We built on this when we started thinking about brains (Golonka & Wilson, 2019), and I'm currently thinking about the next step along this path. 

I'm pretty sure that a big chunk of the work I need to do is explicitly connecting this distinction up to work on the skilled intentionality framework, and the notion of our variable levels of grip on the field of affordances. This work is wrong about affordances (they aren't relations) but other than that, there's a ton of really great work about how intentionality isn't an all-or-nothing thing, and a lot of really useful vocabulary and framing development that I think will be useful for articulating these ideas. I don't like re-inventing wheels, so I'm skilling up on this literature as I develop ideas for a paper. 

This post is about a recent paper (Van Dijk & Kiverstein, 2020) that is explicitly about developing a usage-based notion of information. To unbury the lede, I think this is a robust piece of work with solid internal logic, but I think like all this enactivist style work, it ends up in a place that cannot support a how-actually explanation of behaviour - this particular usage-based theory of information and the things that come with it aren't the framework that will let the ecological approach expand its scope. This is ok, at one level, because I don't think mechanisms are the goal of enactivist analyses. But it's a worry at another level, because I want an ecological theory of direct perception that can actually explain behaviours and this isn't going to cut it. 

Is Indirect Perception Plausible?

There are two basic ways perception might work to let us experience the world in behaviourally relevant ways. Direct perception is the idea that perception only requires two components; the environment and the organism. Indirect perception is the idea that perception requires at least three components; the environment, the organism, and at least one other component that mediates between the organism and the environment. Over the last few posts, I've been working through the specifics of the ecological approach to making direct perception plausible, because this is a question I often get (usually in the form of 'I don't see how this could work in this case'). Regardless of whether or not it's correct, we can show that we have all the pieces needed to make direct perception work in principle, and the empirical programme is about seeing if it works in practice. What about indirect perception?

I asked this question on Twitter, and one interesting thing I noticed was just how little sense the question seems to make to people these days. Responses fell into roughly two categories: 'I don't see how we can do with it in this case', and 'brains do stuff, so...', neither of which answer the question. Even if some form of indirect perception is required in those cases (which is, of course, still up for grabs) we're still owed an account of how this might work, at least in principle and then later in practice. 

People used to know this. The most recent indirect perception hypothesis is that the key mediator is a mental representation, understood as a computational, information processing system implementing some form of inference that combines sensory data and information stored in memory to create a model of the world that represents the system's best guess about what is out there and how to behave successfully with respect to what's out there. This hypothesis didn't come out of nowhere; the development of the computer and the theory of information that allows them to work turned out to provide the pieces required to create a formal account of representations that stood a chance of living up to the challenge of explaining perception. Cognitive scientists therefore leaned heavily into the details of these pieces as they worked very hard, from the late 1950s on, to make indirect perception implemented this way plausible. 

The exact details of the process have, of course, changed and evolved with empirical data and developments in computational theory. For example, while all the accounts have to do inference that combine sources of information into a best guess, there are a variety of ways of doing inference, some better than others. Probably the best way to do inference is via Bayesian methods, and so most modern theories propose that indirect perception combines sources of information this way so as to be optimal. 

Before these inferential methods can even be brought into play, however, there remain two related and big unanswered challenges that need to be addressed. The first one is the grounding problem; how do representations get the content they need so as to combine sources of information in a way that works? It's all very well describing the inferential process of the fully formed system, but how do you build one in the first place? The second is the 'which representation?' problem; of all the different sources of information the system has to combine, how does it know which information to bring together for a given task? These reflect a circular problem indirect theories create for themselves. If perception is not good enough to be direct, and thus requires representational support, where do those representations come from? In order for a theory of indirect perception to be plausible, these must be addressed (analogous to how in order for a theory of direct perception to be plausible, questions like 'can the physical world present itself in behaviourally relevant ways?' had to be addressed). 

I am not going to address these challenges to indirect theories, because it isn't my job. But they are legitimate questions that people have mostly stopped asking. Debates about the form and content of representations were prominent and explicit right up until the end of the 1990s, and then it all just seemed to stop. Interface theory, for all it's problems, at least got back into the fight and tackled the grounding problem (unsuccessfully, I've argued, but it was a solid swing and at least Hoffman recognised he owed us an account). Mark Bickhard's work is probably the only currently active research programme explicitly working out the details, but I don't know many scientists who even know who he is, and a lot of his work is about mapping out the rules of living up to the challenge, versus actually solving the problem. 

Until these foundational issues are addressed and answered, whether indirect perception is plausible remains unclear, and no matter how sophisticated your inferential machinery is (looking at you, free energy principle) it can't help until you explain how it came to be organised that way in the first place. Even if the ecological theory of direct perception doesn't hold up, representational theories of indirect perception are not viable options if they cannot be shown to be plausible. 

Is Direct Perception Plausible? Ecological Information

We're in the home stretch of working through how direct perception is, at least, an option, and how the ecological approach in particular attempts to make it work. We've talked about what direct vs indirect means, the kinds of properties direct perception needs to be out there in order to work, and affordances/effectivities-as-dispositions as the specific properties ecological psychology claims are out there and fit the bill. 

I ended that last post by highlighting ecological psychology had one last thing to do in order to be plausible, and that is to have a way to bring affordances and effectivities together into a kind of contact that allows them to work together. That contact can't be mechanical, or simply physical proximity, because almost all of the things we perceive and act with respect to are not in that kind of contact. The ecological solution is informational contact, and so this post will build on the pieces I've assembled to identify what kind of thing ecological information has to be in order to work. 

Is Direct Perception Plausible? The Case for Affordances-as-Dispositions

The first post in this series laid out the rules for what makes a theory of perception direct or indirect. In order to avoid having to require organisms to somehow figure out behaviourally relevant properties (indirect perception), direct perception requires that such properties are out there already, ready and able to be detected. Such properties are a bit weird - by definition, they must be properties of objects that include some reference to the organism doing the detecting (specifically, they are impredicative properties). 

The second post laid out some reasons to think that such circularly defined properties are legitimate options for the physical world. Properties like this are required by the mathematics of the quantum scale; and so, while a bit weird, are possible. Organisms don't work with quantum properties, though, so if they only happen at that scale, this doesn't help a theory of direct perception. The key to making impredicative properties work is measurement. At the quantum scale, all properties are uncertain until measurement collapses the uncertainty of the thing measured. At the ecological scale, properties aren't uncertain in this way, but which properties are 'primary' still depends on the measurement device; to a polar planimeter, for example, the higher order property 'area' is simple and lower order properties such as 'length' must be figured out. The reverse is true of a ruler. 

We have a set of pieces now. Impredicativity requires measurement to affect which properties are immediately available to the measurer, this is possible at the ecological scale, and with the right measurement device higher-order properties can be immediately available. This establishes an in-principle case in favour of direct perception as an option. Next, if we can identify the higher-order form of behaviourally relevant properties, we will identify what kind of measurement device is required for these to be directly available. 

The main ecological hypothesis is that higher-order behaviourally relevant properties, which we call affordances, take the form of dispositional properties. This post will walk through how this works and how it fulfils what's required to support direct perception.

(As many of you know, not everyone is happy with the dispositional ontology, and propose instead that affordances are relations. I am not going to lay out the parallel case for how this ontology fulfils the requirements I have been laying out, for the simple reason that I do not think it can do any such thing and I have yet to see any successful attempt to make it work. I talk about this in this paper, currently still living in review limbo.)

Are Affordances Plausible? Updating Some Intuitions

In the previous post, I laid out the basic distinction between direct and indirect theories of perception. The basic issue is how to get to behaviourally relevant properties. Such properties of the world have to be partly about the organism, and not just the world; not just 'mass', which is something about the world, but 'moveability' which is something about the world, relative to the organism. If there are properties like these out in the world, and we can perceive them, then perception can, in principle, be direct. If there aren't properties like these there to be perceived, we would have to invent them, and perception would have to be indirect. The ecological hypothesis has to begin by finding behaviourally relevant properties out in the world, ready to be perceived without having to be invented; we need to find affordances. 

The problem is that this seems to be madness, right off the bat. It seems bizarre to think that things separate from the organism could possibly have properties that have anything to do with the organism. Why would they have such things? Surely things can only have properties that are about themselves and make them what they are? 

This post will talk about some pieces of information that make this at least plausible; the next post will talk about the specific ecological hypothesis about affordances that tries to implement that plausibility. To get to that plausibility, I am going to have to update your intuitions about what is simple, and what sorts of things can be physically real, and along the way introduce some useful vocabulary I will do everything I can to use clearly. 

What Does it Mean for Perception to be 'Direct' vs 'Indirect'?

The ecological approach is a theory of direct perception. Put simply, direct perception proposes that our perceptual experience of the world is not mediated by anything that sits between the world and that experience. Making this viable is a big challenge, however, and the idea seems preposterous on the face of it to many people. 

In this post, I want to lay out the basic idea of direct vs indirect perception, and then explain how each approach addresses the problem of how we perceive the world. There will be some big words, but I will aim here to place them in a context that supports them usefully, and focus mainly on the straight-forward ideas in play. In a future post, I will ask the question 'what do we know about how plausible direct and indirect theories of perception are?'

Introducing...The Southampton Project

I am embarking on a very exciting and potentially game-changingly huge collaboration this year, with Southampton Football Club and a bunch of ecologically minded researchers. The goal is to develop a club-based Learning Lab centred on ecological approaches to skill acquisition and coaching, and to embed this Lab in an ecologically minded club-wide ecosystem. I'm working already with a lot of really good people, and key people in the club are fully on board with this (which is why it's now something rather than nothing!) 

In this post, I want to talk about how it came about, what I've been up to over the past year to get it moving, where we are now and where I want to take this in the near future. In line with my general theory of how to get a new collaboration up and running my initial focus has been on getting all the members involved in doing something we can achieve now, with the resources we have at hand, so that we can be moving and doing the things required to target additional funding and resources. In Phases 2 & 3, I want to open things up a little to begin getting those additional resources in place; if you are interested in being involved, feel free to drop me a line.

Structured Flow on Neural Manifolds (Jirsa et al, 2019)

As I try to develop a vocabulary for an ecological neuroscience, I am looking for two things. First, I'm looking for help from existing methods to help identify real neural parts and processes; so far I've ruled the FEP out for that. Second, I'm looking for an existing dynamical framing to help express whatever ecological psychology figures out about the brain. The jury is still out on whether the FEP is that framing; another option is a development of Kelso's coordination dynamics that invokes structured flow on manifolds. (This paper by Jirsa, McIntosh & Huys was a contribution to the special issue on Ecological Neuroscience). 

As I review the paper, I am going to be trying to figure out if this mathematical framing is going to help. It's not going to be a guide to real neural parts, but it might be the right way to formally describe the real parts we identify by other means. 

Do Markov Blankets Give Us Real Neural Parts?

In my last post, I laid out what I think the rules are for developing a mechanistic model of the neural scale contribution to behaviour. I ended there with a question: what counts as a real neural part? How can we successfully decompose neural activity supporting a given perception-action loop into parts? 

In this post, I want to discuss one potential option: the hypothesis that Markov blankets, a key feature of the free-energy principle approach to neuroscience, can identify and pick out real neural parts. I'll discuss some recent ecological critiques of Markov blankets and some potential answers to the challenges.

Mechanistic Models of the Brain

I'm getting increasingly interested in neuroscience, and how to make it ecological. I also think that the ecological approach is capable of supporting mechanistic explanatory models of behaviour and is the correct scale* at which to ground these models. This means that my current plan is to find a way to add neuroscience as a lower scale part of a model grounded at the scale of the organism-environment system. 

There's a lot going on in that sentence, though, so I want to unpack it a bit to lay out the rules. and the things I currently don't know. 

(*NB I am using scale rather than level throughout because the concept of a level is complicated and currently, I am convinced that scale is a better term. The argument continues, however). 

The Constraints-Based Approach to Teaching in the Classroom

If you read this and think 'hey, this sounds like something they do in [insert teaching method here]', please let me know. I've had some chats about the Montessori method, and there's certainly overlap there. But I'm on the hunt for a literature I can connect to, and any help would be appreciated.

I've been thinking a lot about education lately. I'm home-schooling the kids, I've been chatting a lot with coaches about ecological approaches to their teaching (most publicly here and here), and I'm reading Tim Ingold's Anthropology And/As Education. I'm also wondering why the demonstrated success of the ecological dynamics approach in sports pedagogy has had zero consequences for education more broadly. 

I think a couple of things. I think the reason why ecological dynamics hasn't spilled over is that we live in a dualist world where knowledge and physical skills are two distinct domains (think about how physical education is treated in schools). I also think that because the ecological approach doesn't endorse that dualism, there is simply no reason for classroom education to work completely differently from physical education. And finally, I think this might be really, really important.

I used to teach a module called Foundation Research Methods, and after a while I finally realised that I was teaching it in a constraints-based, ecological dynamics style. (This explains why a lot of my colleagues were genuinely confused by what I was doing at times, I think!). The module developed over the years, and the last year I taught it we solved our attendance problem and the students crushed the exam. 

I want to walk through what I did, and reflect on how it embodied an ecological approach. This is not me saying this is how all classes should be taught. This is just me laying out what a constraints-based approach looked like in the class, what I thought worked, and what I would like to have done next.