Lecture 26: Strong Anticipation and Direct Perception (Turvey, 2019, Lectures on Perception)

In this final chapter, Turvey lays out the basics of the ecological approach to coordinating action with respect to the future. This is actually a key part of behaviour - we reach for things we know are graspable before grasping them, we move to intercept or avoid things before they are anywhere near us, and so on. The question is how do we do this.

The ecological approach cannot, of course, use prediction to solve this problem. Prediction entails representation, making educated guesses about what is coming up based on loans of intelligence we cannot ever pay off. The whole point of this book has been to lay out the argument that a) this approach is doomed to fail but that b) the ecological approach, grounded in laws, is different enough in kind to be a viable option. 

Coordinating with things not in the immediate present is therefore going to rely on lawful informational coupling to dynamical events, with several interesting consequences. 

Lecture 25: The Mechanical Basis for "Getting About Among Things" (Turvey, 2019, Lectures on Perception)

This chapter is about the mechanical consequences of interacting with the environment; what happens when we come into physical contact with things. Importantly, though, we are going to focus on the informational consequences - what information is created by mechanical interactions, what does it enable us to perceive, and what is the medium for this information? Broadly then, the topic of this literature is dynamic/effortful touching. 

Talking about this topic will enable us to make a few key points. First, dynamic touch is a commonplace activity, namely the kind of activity an ecological psychology takes as it's main topics. Second, it engages with the ecological hypothesis that information is specific to the dynamics of what it is about, and not specific to the medium in which it lives. Visual information and haptic information, for example, are the same kind of thing, and are sometimes even the same information variable. The important thing for a theory of perception is the information, and not the details of the medium, nor the anatomy that happens to pick it up. 

Turvey then spends most of the chapter reviewing key empirical findings from the ecologically motivated investigation of dynamic touch. The key perceptual variables are not things like mass or weight, for example, but the moments of inertia; how an object resists changing state. This leads to tasks that involve hefting or wielding (changing the state, essentially), and experimental manipulations that separate out things like mass from mass distribution and the inertia tensor. 

Several things emerge. People can perceptually identify different properties via similar movements, by attending to different properties of the object. This demonstrates that intentionality matters (you can ask people to judge different things and that drives attention to different variables), and that in any task there are multiple variables available, defined across various scales and relations between properties (the task is multi-fractal). These mechanical properties are also the basis of perceiving things about ourselves, as well as what objects we are mechanically interacting with; we perceives things about our limbs this way as well as things our limbs are dynamically touching. There's no in-kind difference between perceiving ourselves and other things, it's all grounded in informationL. Finally he notes the case of weight perception, and how it is not based on the weight of objects, but on their inertia (this connects to Gibson's point, that the variables of perception must be empirically discovered and confirmed, not assumed from one description of the task at hand). This connects to things like the size-weight illusion and reveal it's only an illusion given the assumption of the wrong variables at play. 

Turvey then discusses a fascinating case, a man called Ian Waterman who lost all proprioception and had to effortfully replace that perception of the environment with vision. The details of his ability to do so and the limits on what he could do are a fascinating story for ecological psychologists (some clips from a documentary here, and there's a great book about it too). 

Turvey briefly reviews his hypothesis that the medium for all this mechanical interaction perception is the body considered as a tensegrity structure (Turvey & Fonseca, 2014). The hypothesis is that the way the body deforms under mechanical pressure provides a medium for the same kinds of structure as the optic array does; it has the right kind of features to support implementing informationL. Analogous to the optic array, the key is the right level of description - the optic array begins with optical solid angles, for example, rather than points or rays of light. 

Turvey then notes briefly that hearing is also about the detection of mechanical events, just not necessarily ones in which our body was involved. He points to the work of Gaver (1993a, b) on the kinds of events that can be specified in the acoustic array, and how we can 'hear what a struck object looks like' because of the nature of the informationL involved. 

Lecture 24: Perceiving "How to Get About Among Things" (Turvey, 2019, Lectures on Perception)

This chapter focuses on the perception side of the perception-action system we will end up with for behaviour. It covers the objects of perception (affordances), the means of perception (law-based information, informationL), and the notion of prospective control (vs. anticipating or predicting the future). 

The chapter title is 'how to get about among things'. This stands in contrast to how things are normally framed, in terms of distance or space perception. Perception is for action, it is about enabling safe, skilled activity in a cluttered world. Because of this, the objects of perception (what perception is about) must be action related - affordances. For Turvey, affordances are properties of the environment (not relations between environment and organism) and they are perceived if and only if there is informationL about that affordance present and detected. 

Turvey then introduces the idea that perception and action are duals that define a duality (rather than the beginning and end of a linear chain of causation). Note this is a duality, and not a dualism - the two things are not different in kind, but are parts of a greater whole that are intimately and lawfully connected to one another. This implies a relationship between what information specifies, and the energy required to move with respect to what it specifies. The main idea here though is that it is not perception then action, it is only perception-action. 

Turvey then spends time on the optical basis for getting about among things; ecological optics. Ecological optics is not physical optics (which is about things like wavelengths), nor is it physiological optics (which is about the anatomy and physiology of the eye). Ecological optics is about what is available in the light prior to any interaction with an organism, and therefore what is available to stimulate the retina, etc. The key is optic flow: structured light distributions incident to illuminated surfaces. 

The optical structure available at a single point is specific to the stationary surface layout that gave rise to that structure (this is why the Ames Room works, for example). Optic flow is the general case; optical structure in which the transformations are specific to the surface layout and the relative motions of the observer and the environment. Turvey works through the example of tau in some detail, to illustrate how movement generates an optical flow field and how that field is structured specific to the motion. He then briefly reviews more complicated work on other geometrical features of optical arrays (transformations such as divergence, curl, and deformation; Koenderink, 1986). These are examples of the kinds of maths required, and the target of that maths (the structure of the optic array). Turvey also nods to Stoffregan's notion of a global array (although I will note that this hasn't really gone anywhere yet and it's not clear what the medium of this array is). 

Finally, Turvey reviews how optical information can support prospective (future oriented) control. In general there are two approaches to dealing with the future. We have encountered the idea of prediction/anticipation, and the various problems with that. In short, it is a form of induction and it suffers from all the problems of that. Instead, informationL will allow prospective control, via specification of what will occur, if present conditions continue (Bootsma calls this 'the current future'). Turvey of course looks to the outfielder problem as the key example here. 

(I will note here that this topic shows up in the modern literature in the form of information-based control (Bill Warren) vs affordance-based control (Brett Fajen). Turvey nods to Fajen and acknowledges that the information used for control should be about affordances; but the debate is not engaged with here.)

Finally, as a prelude to the next chapter, Turvey notes again the idea that all this information is dual to/reciprocal with the action side of the equation. 

Lecture 23: Ecological Optics Primer (Turvey, 2019, Lectures on Perception)

The 'barrage of conceptual details that was Lecture 22' has provided us with the notion of affordances, and these are to be the objects of both perception and action. This means the need to be perceived, and this brings us to the second part of the ecological ontology - information. This chapter will focus specifically on optical information, identify the key characteristics of an optic array, and walk through some ways of analysing perception ecologically by dealing with some problems supposedly raised by illusions. 

Lecture 22: Ontology at the Ecological Scale (Turvey, 2019, Lectures on Perception)

After a book length set of preliminaries, we are finally getting to the meat of the positive case the ecological approach has to make. That case of course sits on top of a rejection of the Cartesian ontology that has been driving everything else; this chapter describe the first part of the ecological replacement, specifically affordances. 

Lecture 21: Barriers to Ecological Realism (Turvey, 2019, Lectures on Perception)

Last Lecture Turvey introduced the notion of ecological realism; realism that has a species-dependence to it. Normally a realism is for everyone, but different organisms inhabit different habitats and/or niches, and have different job descriptions. So another preliminary that needs to be dealt with is that there are immediate and long-standing objections to ecological realism that need to be noted and addressed. The two main ones addressed in this Lecture are dualism and the doctrine of physics as complete.

Lecture 20: Ecology: The Science that Reasons Why (Turvey, 2019, Lectures on Perception)

Turvey has finally arrived at the ecological approach, but all the preceding chapters have made it such that there are preliminaries to deal with first. This first Lecture sets up a very broad, all-encompassing motivation for an ecological approach, summarised in the first sentence: "A theory of perception ought to be a theory for all organisms.". Turvey will spend some time identifying that previous work has focused on humans, we are one of a countless number of organisms and not especially representative, and that the why of perception-action should be rooted in the laws of thermodynamics. 

Lecture 19: Reflections on the Physical Symbol System Hypothesis (Turvey, 2019, Lectures on Perception)

The final lecture about the computational-representational perspective is focused on the general claim that intelligent action can arise from a computer, specifically a physical symbol system. This hypothesis underpins the computational-representational perspective; it proposes that an organism can come to 'know about' things via computations over symbols. The last few lectures have, of course, revealed flaws with this approach, but there are a few issues left to address.

Lecture 18: Turing Reductionism, Token Physicalism: The Computational System Assumption (Turvey, 2019, Lectures on Perception)

We have encountered the idea in Lectures 6 and 16 that perception might be formalised as a form of computation, and that this might be the step that enables the Cartesian programme to succeed. This Lecture spends time on work on the underpinnings of this hypothesis, namely work on what kind of thing mathematics and computation are and can do. 

Lecture 17: Pattern Recognition and Representation Bearers (Turvey, 2019, Lectures on Perception)

Last Lecture Turvey reviewed the basics of the computational approach, and highlighted again how it is just the latest iteration of the Cartesian programme. In this Lecture, he explores the specific topic of pattern recognition, which has been a major topic in the computational approach and exemplifies many of the major problems. These problems primarily boil down to systems requiring loans of intelligence to even come close to working.

Lecture 16: The Computational-Representational Perspective: Preliminaries (Turvey, 2019, Lectures on Perception)

The next four Lectures are specifically focused on the modern form of the Cartesian programme - the computational-representational approach. It's important to spend some time here, because this is the ecological approach's current opposition, and because the fact it is just yet another Cartesian programme matters, and is at the heart of most of our objections to it. 

In Lecture 6 we learned about the 3 grades of sense. First, there is reflex. The second is limited awareness of secondary qualities. The third is full mental awareness of what it all means. The proposed solution (Hobbes) for building the third grade out of the other two is the manipulation of symbols to do inference. In modern times, this is implemented as computation, and implemented in a representation.

Lecture 15: Gestaltism III: Experience error, CNS error, Psych-neural Isomorphism, Behavioural Environment (Turvey, 2019, Lectures on Perception)

The two previous lectures reviewed some of the changes happening in related fields to psychology that were showing up in Gestalt psychology, specifically the notion of fields, the understanding that behaviour could emerge from dynamical processes, and the limits on mechanical approaches to living systems. This chapter finally actually engages with Gestalt psychology, and discusses how it was applying these changes.

Lecture 14: Gestaltism II: Fields, Self-Organization, and the Invariance Postulate of Evolution (Turvey, 2019, Lectures on Perception)

In the previous Lecture, Turvey discussed the machine metaphor in which things are just equal to the sum of their parts. In this Lecture, Turvey introduces the Gestalt notion that wholes are different from the sum of their parts. Well, specifically, he discusses the notions of fields and self-organisation, and the kinds of physical systems that these entail; Gestalt psychology is mostly just a launching pad for the broader discussion. 

Lecture 13: Gestaltism I: Atomism, Anatomism, and Mechanical Order (Turvey, 2019, Lectures on Perception)

We are nearing the end of the Foundational Concepts section of the book, and we have arrived at one final historical attempts to explain perception; the Gestalt school. This chapter doesn't actually spend any time on Gestalt, however, but instead lays out the things Gestalt was developed to oppose. Those things are atomism, anatomism, and mechanical order.

Lecture 12: The Space Enigmas IV: On Learning Space Perception (Turvey, 2019, Lectures on Perception)

This Lecture is in roughly two parts. The first brief section walks through a Helmhotzian method for perceiving depth via unconscious inference. This inference process is learned (Helmholtz wanted to be an empiricist) but as usual entail loans of intelligence in the form of some givens not acquired via experience. This then raises a question: what exactly is experience, and what about it is used to be the basis for future inference? It turns out what counts as the relevant parts of experience can be very non-obvious, raising many problems that need more modern, less Cartesian solutions. 

Lecture 11: Doctrines of Sensations and Unconscious Inferences (Turvey, 2019, Lectures on Perception)

This Lecture is about Helmholtz, and his theory of sensations being integrated into perceptual experience via unconscious inference. Everything in here should look very familiar to anyone who has ever taken a Sensation and Perception class, because modern theories of indirect perception are literally versions of this account. (I'll note as well that, while there are embodied and ecological interpretations of the free energy principle (e.g. Bruineberg et al, 2018), one common interpretation is that it is an implementation of Helmholtz's unconscious inference; e.g. Friston and Kiebel 2009; Hohwy, 2013). 

Unconscious inference is a firmly Cartesian programme. It is about using the physiological measurements of things like light, and inferring their underlying causes (i.e. what is happening in the world to be perceived). Unsurprisingly, it is going to run into the same main problem, namely unrepayable loans of intelligence. 

Lecture 10: The Space Enigmas III: Local Signs and Geometrical Empiricism (Turvey, 2019, Lectures on Perception)

The previous two lectures have covered aspects of the problem of space perception. We perceive objects, and these inhabit space - they are at some distance from us and each other, etc. So what is space, and how do we come to know about it? We've tried using touch (a sense that lives in 3D) to provide the necessary clues (this was Berkeley, and the story of Flatland). This doesn't work because touch isn't a perfectly reliable source of space information, it's a perceptual system like vision and faces the same kinds of problems. We've also tried to just make space a necessary feature; not an object of perception, but a mode of perception (this was Kant, and the power of Euclidean geometry). This doesn't work, because there are perfectly coherent non-Euclidean geometries, that all provide different answers when measuring the same space. As soon as there is more than one geometry, choosing one becomes part of the problem and you cannot simply assume it. 

Given this, how might you go about selecting the right geometrical description to use as the basis for your experience of space? Turvey discusses one major attempt to show how this might happen, specifically Helmholtz's account of how to derive a geometry (a notion of space) from non-spatial local signs. 

This Lecture involves quite a bit of mathematical detail, which I am not going to get into here because it's slightly besides the point. The overall goal, however, is to take some sensory raw material that isn't intrinsically about space (because it can't be), and work to turn it into an experience that is about space. This is going to be the start of the move to talk about sensations and perception, the organising principle still at the heart of all non-ecological discussions of how we come to experience the world. More on this in the next lecture. 

Lecture 9: The Space Enigmas II: Kant, the Nature of Geometry, and the Geometry of Nature (Turvey, 2019, Lectures on Perception)

The first space enigma was the fact that vision lives in the two dimensions of Flatland, but produces an experience of three dimensional Spaceland. You can't logic or experience your way from Flatland to Spaceland (as described in the famous book). Berkeley tried to solve this problem by providing a guide, in the form of the Spaceland-dwelling body, but this fell apart and the only remaining suggestion was an unrepayable loan of intelligence from God. 

Another way to consider this problem that leads to another proposal is what Turvey calls 'the outness problem'. This is the annoying fact that sensations on the retina are experienced as things out there, in the world. This makes space a necessary precursor to perceptual experience: however the outness problem is solved, a notion of space is required to drive the search for a solution. Kant is the main person who worked to establish how space might be baked into perception; 'Space, therefore, is not an object of perception...but something very different, namely, a mode of perceiving objects' (Turvey, 2019, pg 124). Spoiler alert: it doesn't work, for interesting reasons that feed into the development of the ecological analysis. 

Lecture 8: The Space Enigmas I: Berkeley (Turvey, 2019, Lectures on Perception)

One of the big problems that emerges from all the proceeding discussions of perception is how we are able to perceive space. Space has been considered as a mathematical concept (in terms of Euclidean geometry), as a psychological concept (a construction of the mind) but never really as a biological, ecological concept. This first chapter about space perception is focused on one mathematical conception, some of it's implications, and one specific attempt to deal with those implications (Berkeley's New Theory of Vision). 

Turvey, Lectures on Perception: The Story So Far

I have now reviewed the first 7 Lectures on Perception from Turvey' textbook (posts indexed here). I feel like I've reached a natural pause point before carrying on with the rest of the lectures, and I wanted to summarise what I thought has been going on so far.

These are lectures on perception, quite generally. While Turvey will explain the ecological approach, that's not what is currently going on. Right now Turvey is laying out the scope of the question 'what kind of material system could be a perceiving system?' and reviewing the typical, non-ecological answers to that question.

The main takeaway at this point is that we have been trying to answer that question using 17th century notions of material systems, specifically Newtonian mechanics. This has placed what turn out to be unacceptable restrictions on the kinds of properties we are allowed to invoke, and theories of perception have had to fill the gaps with unpayable 'loans of intelligence'. 21st century physics, however, has more on offer; complexity science and quantum mechanics teach us many valuable lessons on how to expand our horizons so that perception becomes possible, rather than miraculous.