Lecture 9: The Space Enigmas II: Kant, the Nature of Geometry, and the Geometry of Nature (Turvey, 2019, Lectures on Perception)

The first space enigma was the fact that vision lives in the two dimensions of Flatland, but produces an experience of three dimensional Spaceland. You can't logic or experience your way from Flatland to Spaceland (as described in the famous book). Berkeley tried to solve this problem by providing a guide, in the form of the Spaceland-dwelling body, but this fell apart and the only remaining suggestion was an unrepayable loan of intelligence from God. 

Another way to consider this problem that leads to another proposal is what Turvey calls 'the outness problem'. This is the annoying fact that sensations on the retina are experienced as things out there, in the world. This makes space a necessary precursor to perceptual experience: however the outness problem is solved, a notion of space is required to drive the search for a solution. Kant is the main person who worked to establish how space might be baked into perception; 'Space, therefore, is not an object of perception...but something very different, namely, a mode of perceiving objects' (Turvey, 2019, pg 124). Spoiler alert: it doesn't work, for interesting reasons that feed into the development of the ecological analysis. 

Trip Report from the Uncontrolled Manifold

I've spent the past few months getting a new paper to the 'complete first draft' stage (you can find a copy here in the meantime; it's still got some work to do though). It's about affordances, using targeted long-distance throwing as the task, and it's my first dip into the world of the uncontrolled manifold. I collected this data over five years ago, and it's been deeply satisfying to actually use it after all this time.

Part of what's taken so long is that I've had to learn the details of the uncontrolled manifold analysis. I blogged some about it here and here but this was the year I finally had the time and data to actually get into the maths. I still really like it as an approach to analysing human movement, but learning the details and trying to figure out how to get affordances into it has raised a lot of interesting questions about how it gets used right now and what this all implies for how we think movement is controlled. I'm raising a bunch of these issues in the paper but I wanted to sketch some out here for comment.

Broadly (and this shouldn't be a surprise to anyone really) I've realised that UCM is only a method, not a theory, and it's therefore not able to serve as a 'guide to discovery' about movement control. However, it's being used as if it can, and to be honest I was quite shocked at how carelessly it's being used in the literature. 

Are Illusions Even a Thing?

Traditional vision science is very excited about illusions. These are cases when perception seems to break down; there is a mismatch between what is out there and what we experience, and traditional approaches consider these breakdowns as clues to how vision has to work, given what it is working with. 

Ecological psychologists don’t like illusions. Typically, they occur when information is either made ambiguous or faked, and in general we think these are the wrong situations to study perception in. We sometimes engage with the literature on these effects, but usually to show how the trick is the result of not thinking ecologically. 

Rogers (2022) has taken this basic analysis but gone one interesting step further. He’s argued that the notion of ‘visual illusion’ is simply not a clear category; it’s not a useful way to describe any of the effects people study. He argues that there simply is no sufficient definition of what an illusion is that works, and that what we call illusions are just either tricks (as above) or inevitable consequences of how the visual system works. 

I am broadly on board with this additional step, and it’s made me think hard about what illusions are and how best to respond when people use them against direct perception. 

Lecture 8: The Space Enigmas I: Berkeley (Turvey, 2019, Lectures on Perception)

One of the big problems that emerges from all the proceeding discussions of perception is how we are able to perceive space. Space has been considered as a mathematical concept (in terms of Euclidean geometry), as a psychological concept (a construction of the mind) but never really as a biological, ecological concept. This first chapter about space perception is focused on one mathematical conception, some of it's implications, and one specific attempt to deal with those implications (Berkeley's New Theory of Vision). 

The medium for direct perception (Notes on Van Dijk & Kiverstein, 2020)

The ecological approach has hit a point in its history where it has become interested in expanding its scope, to go beyond the real time coordination and control of action. There are many challenges from non-ecological cognitive science about how to tackle representation-hungry problems, and how to conceptualise things like language, social behaviour, and what the brain is up to. I am all on board with this move - it was important we waited till we were ready, but since Gibson died in 1979, the empirical programme on the basics has matured into a solid foundation and we have a lot of developed or adopted a lot of things that will come in useful. 

However, if we are going to do it, I want us to do it with rigour and care and with reference to all our hard-won successes. My current view is that our best path lies in looking at the ways we are able to use ecological information, and grounding our explanations and hypotheses at this scale. Sabrina first developed this idea in a paper about how to think about what language is (Golonka, 2015). The big take-home from that paper is the analytic distinction between law-based use and convention-based use of information, and the first draft of the consequences of this distinction. We built on this when we started thinking about brains (Golonka & Wilson, 2019), and I'm currently thinking about the next step along this path. 

I'm pretty sure that a big chunk of the work I need to do is explicitly connecting this distinction up to work on the skilled intentionality framework, and the notion of our variable levels of grip on the field of affordances. This work is wrong about affordances (they aren't relations) but other than that, there's a ton of really great work about how intentionality isn't an all-or-nothing thing, and a lot of really useful vocabulary and framing development that I think will be useful for articulating these ideas. I don't like re-inventing wheels, so I'm skilling up on this literature as I develop ideas for a paper. 

This post is about a recent paper (Van Dijk & Kiverstein, 2020) that is explicitly about developing a usage-based notion of information. To unbury the lede, I think this is a robust piece of work with solid internal logic, but I think like all this enactivist style work, it ends up in a place that cannot support a how-actually explanation of behaviour - this particular usage-based theory of information and the things that come with it aren't the framework that will let the ecological approach expand its scope. This is ok, at one level, because I don't think mechanisms are the goal of enactivist analyses. But it's a worry at another level, because I want an ecological theory of direct perception that can actually explain behaviours and this isn't going to cut it. 

Is Indirect Perception Plausible?

There are two basic ways perception might work to let us experience the world in behaviourally relevant ways. Direct perception is the idea that perception only requires two components; the environment and the organism. Indirect perception is the idea that perception requires at least three components; the environment, the organism, and at least one other component that mediates between the organism and the environment. Over the last few posts, I've been working through the specifics of the ecological approach to making direct perception plausible, because this is a question I often get (usually in the form of 'I don't see how this could work in this case'). Regardless of whether or not it's correct, we can show that we have all the pieces needed to make direct perception work in principle, and the empirical programme is about seeing if it works in practice. What about indirect perception?

I asked this question on Twitter, and one interesting thing I noticed was just how little sense the question seems to make to people these days. Responses fell into roughly two categories: 'I don't see how we can do with it in this case', and 'brains do stuff, so...', neither of which answer the question. Even if some form of indirect perception is required in those cases (which is, of course, still up for grabs) we're still owed an account of how this might work, at least in principle and then later in practice. 

People used to know this. The most recent indirect perception hypothesis is that the key mediator is a mental representation, understood as a computational, information processing system implementing some form of inference that combines sensory data and information stored in memory to create a model of the world that represents the system's best guess about what is out there and how to behave successfully with respect to what's out there. This hypothesis didn't come out of nowhere; the development of the computer and the theory of information that allows them to work turned out to provide the pieces required to create a formal account of representations that stood a chance of living up to the challenge of explaining perception. Cognitive scientists therefore leaned heavily into the details of these pieces as they worked very hard, from the late 1950s on, to make indirect perception implemented this way plausible. 

The exact details of the process have, of course, changed and evolved with empirical data and developments in computational theory. For example, while all the accounts have to do inference that combine sources of information into a best guess, there are a variety of ways of doing inference, some better than others. Probably the best way to do inference is via Bayesian methods, and so most modern theories propose that indirect perception combines sources of information this way so as to be optimal. 

Before these inferential methods can even be brought into play, however, there remain two related and big unanswered challenges that need to be addressed. The first one is the grounding problem; how do representations get the content they need so as to combine sources of information in a way that works? It's all very well describing the inferential process of the fully formed system, but how do you build one in the first place? The second is the 'which representation?' problem; of all the different sources of information the system has to combine, how does it know which information to bring together for a given task? These reflect a circular problem indirect theories create for themselves. If perception is not good enough to be direct, and thus requires representational support, where do those representations come from? In order for a theory of indirect perception to be plausible, these must be addressed (analogous to how in order for a theory of direct perception to be plausible, questions like 'can the physical world present itself in behaviourally relevant ways?' had to be addressed). 

I am not going to address these challenges to indirect theories, because it isn't my job. But they are legitimate questions that people have mostly stopped asking. Debates about the form and content of representations were prominent and explicit right up until the end of the 1990s, and then it all just seemed to stop. Interface theory, for all it's problems, at least got back into the fight and tackled the grounding problem (unsuccessfully, I've argued, but it was a solid swing and at least Hoffman recognised he owed us an account). Mark Bickhard's work is probably the only currently active research programme explicitly working out the details, but I don't know many scientists who even know who he is, and a lot of his work is about mapping out the rules of living up to the challenge, versus actually solving the problem. 

Until these foundational issues are addressed and answered, whether indirect perception is plausible remains unclear, and no matter how sophisticated your inferential machinery is (looking at you, free energy principle) it can't help until you explain how it came to be organised that way in the first place. Even if the ecological theory of direct perception doesn't hold up, representational theories of indirect perception are not viable options if they cannot be shown to be plausible. 

Is Direct Perception Plausible? Ecological Information

We're in the home stretch of working through how direct perception is, at least, an option, and how the ecological approach in particular attempts to make it work. We've talked about what direct vs indirect means, the kinds of properties direct perception needs to be out there in order to work, and affordances/effectivities-as-dispositions as the specific properties ecological psychology claims are out there and fit the bill. 

I ended that last post by highlighting ecological psychology had one last thing to do in order to be plausible, and that is to have a way to bring affordances and effectivities together into a kind of contact that allows them to work together. That contact can't be mechanical, or simply physical proximity, because almost all of the things we perceive and act with respect to are not in that kind of contact. The ecological solution is informational contact, and so this post will build on the pieces I've assembled to identify what kind of thing ecological information has to be in order to work.