Lecture 12: The Space Enigmas IV: On Learning Space Perception (Turvey, 2019, Lectures on Perception)

This Lecture is in roughly two parts. The first brief section walks through a Helmhotzian method for perceiving depth via unconscious inference. This inference process is learned (Helmholtz wanted to be an empiricist) but as usual entail loans of intelligence in the form of some givens not acquired via experience. This then raises a question: what exactly is experience, and what about it is used to be the basis for future inference? It turns out what counts as the relevant parts of experience can be very non-obvious, raising many problems that need more modern, less Cartesian solutions. 

On "The poverty of embodied cognition" (Goldinger et al, in press)

A new paper in Psychonomic Bulletin and Review (Goldinger, Papesh, Barnhart, Hansen & Hout, 2015) has taken a swing at the field of embodied cognition, claiming that it is vague, trivial and unable to add anything scientific to the investigation of cognition.

...our goal is to zoom out from specific empirical debates, asking instead what EC offers to cognitive science in general. To preview, we argue that EC is theoretically vacuous with respect to nearly all cognitive phenomena. EC proponents selectively focus on a subset of domains that work, while ignoring nearly all the bedrock findings that define cognitive science. We also argue that the principles of EC are often (1) co-opted from other sources, such as evolution; (2) vague, such that model building is not feasible; (3) trivially true, offering little new insight; and, occasionally, (4) nonsensical. 

My basic take is a) I actually agree with a lot of the criticisms in the context of the kinds of 'embodied' cognition we critique for similar reasons, but b) there is nothing new to any of these critiques, none of them are compulsory failings of the field and nothing about them makes embodiment an intrinsically empty notion. 

Brains Don't Have to be Computers (A Purple Peril)

A common response to the claim that we are not information processors is that this simply cannot be true, because it is self-evidently the case that brains are transforming and processing information - they are performing computations. Greg Hickok throws this ball a lot, and his idea is clear in this quote from his book 'The Myth of Mirror Neurons':

Once you start looking inside the brain you can’t escape the fact that it processes information. You don’t even have to look beyond a single neuron. A neuron receives input signals from thousands of other neurons, some excitatory, some inhibitory, some more vigorous than others. The output of the neuron is not a copy of its inputs. Instead its output reflects a weighted integration of its inputs. It is performing a transformation of the neural signals it receives. Neurons compute. This is information processing and it is happening in every single neuron and in every neural process whether sensory, motor, or “cognitive.”

Hickok, pg 256.

There are two claims here. First, neurons are processing information because their input is not the same as their output; they are transforming the former into the latter. Second, this process is computational; 'neurons compute'.

This is a widely held view; psychologist Gary Marcus even wrote about this in the NYT saying 'Face it, your brain is a computer'. In response, Vaughn Bell at Mindhacks posted about this op-ed and this issue in a nicely balanced piece called 'Computation is a lens'. He sums up the issue nicely by asking 'Is the brain a computer or is computation just a convenient way of describing its function?'. The answer, I propose here, is that computation is a fantastically powerful description of the activity of the brain that may or may not be (and probably isn't) the actual mechanism by which the brain does whatever it does. This is ok, because, contra Hickok,  not every process that sits in between an input and a different output has to be a computational, information processing one. 

Rhythmic constraints on stress timing in English

What kind of embodied constraints affect the production of speech? Can we say anything we like when we like, or are there constraints in play that make some things easier than others? This is the question asked in Cummins & Port (1998) which we recently read in lab meeting (with our PhD student Agnes).

Cummins and Port asked participants to produce sentences over and over and examined when during the cycle a certain stress beat occurred. They set it up so that the beat was timed with a beep to occur throughout the cycle, but showed that people could actually only place the beat in 2 or 3 places in the beat reliably. The big picture result is that speech production is shaped, in part, by the underlying dynamics of production described in terms of the rhythms it is set up to produce.

The nice detail here comes from the theoretical set up and analysis that drives this study. Cummins and Port are directly inspired and guided by work in coordination dynamics. Agnes is interested in this work because she's looking at ways to investigate language and speech using the tools of dynamical systems and embodied cognition - remember, our big pitch is that language is special but not magical and we should be able to study it the way we study, say, rhythmic movement coordination. 

'Embodied Cognition Is Not What You Think It Is' - the paper!

Whoops, we did it again - a paper based on the blog! This time we are in press at Frontiers in Psychology, in a Research Topic on embodied cognition, with a paper we somehow got away with calling 'Embodied Cognition is Not What You Think It Is'. 

This paper draws from a lot of posts on the blog on embodied cognition, perception-action and language. We have used this opportunity to tackle some key issues head on, and we like this paper a lot :) We cover all the important issues and we set up what we think is the way forwards for embodied cognitive science. In addition, it sets up the ground work that we want to build on with our own Research Topic on Radical Embodied Cognitive Neuroscience. We've laid out what we think is the task facing the brain; this is what the brain is engaging with, and so this is what we think neuroscience needs to work with in order to understand what the brain is doing.

It's the kind of paper that will either land with a splash or vanish without trace. We want it to make some serious waves, and we're hoping that we can encourage people to publish free Commentaries on it at Frontiers, to challenge us or pick up our challenges, and, most fun for all, to come work with us to take all this forwards! We want this to be the basis of an empirical research programme and we want you all to work with us on it :) At the very least, feel free to pepper us with questions; this paper is the start of something for us, not the end and we're interested in the response to this paper to frame the next step.

Are babies super? Performance, competence and infant habituation

Are babies really more competent than we give them credit? (No.)

Developmental psychology is filled with studies that claim to show the hidden abilities of babies. The claim is that babies come pre-packaged with all kinds of knowledge and skills that provides them with the foot in the door they need to learn about the world. Babies are limited in their ability to demonstrate this knowledge, however, because of their immature bodies and inability to control these well. In the language of the field, there is hidden competence concealed by problems with performance, and researchers (such as Liz Spelke and Renee Baillargeon) are interested in finding ways to reveal this hidden competence.

At IU we referred to such studies as 'super baby' studies, because they purported to show that infants were remarkably competent and knowledgeable about the world. Besides the rampant dualism of 'mind' being concealed by 'body', these studies are good examples of a common problem (the psychologist's fallacy) in psychological research, one that a rigorous application of embodied cognition helps fix.

Some Ground Rules for a Theory of Psychology

Add psychology to the list

A fairly common response to our theory post was 'here's my theory, which is designed to replace and fix all the others'. However, it's more a symptom of the problem I was discussing than a solution for everyone to have their own entirely separate theory which doesn't talk to any other work in the field (see above). One of my personal goals in science is to not be that guy. I want to see cognitive science become more integrated, not more fragmented. We have also been asked, however, and quite sensibly, what we think the solution to our problem is. The question then is how to propose a theoretical approach for psychology and cognitive science where we don't just reinvent the wheel.

Sabrina and I have been working on this for, well, the entire blog. It has been a place for our "brave attempt to think out loud about theories of psychology until we get some" since day one; we've been identifying problems but, just as importantly, solutions the whole time. The theory post identified the big picture problem we see in psychology; time to lay out some solutions.

Step one is to present a map of the blog, organised thematically to guide new readers to work we've already done here. This should also help map out the gaps in the approach, so we can focus on things to do next; feel free to point us to problems we can't yet address! (And yes, we know about episodic memory and language - we're working on it.) This post is not a comprehensive summary of past work - it's a map for you to use to find what we've done so far.

To summarise: in essence, and some minor details aside, we are advocating for Chemero's (2009) radical embodied cognitive science, with the addition of some elements he was missing (network science & task specific devices). Cognition is embodied, extended and held together by the direct perception of affordances and events; the result is a complex, nonlinear dynamical system that must be analysed as such. The brain is not the sole source of our behaviour, nor is it representing the world; it clearly plays a critical role in this system, though, and we propose that we'll need the tools of network science to describe what it's actually up to (Sporns, 2010). Methodologically, we must carefully characterise the task, the resources available to solve the task (which include brain, body and environment) and the information these resources create which can sustain the formation and control of an embodied solution. This method is Bingham's (1988) task specific device approach (the main piece Chemero was missing, I think).This approach applies to all and any behaviour you want to explain, including the hard stuff like episodic memory and language.

Critically, this approach, while new (and uncommon in insisting on a role for Gibson's ecological approach) isn't just something we invented: all these elements are active parts of modern cognitive science. The only new part is bringing it all under one roof, with the goal of getting on and getting some decent normal science under our belts.

Here's what we've covered so far. If you want more details on any point, click on the links!

Robots, Representation, & Dynamical Systems

When cognitive science tries to explain a given behaviour, it typically looks in one of two places for it's explanation. Some people go looking in the brain for the representation that encodes the solution to the task; these people typically treat the brain as the source of the observed structure in behaviour. Some of us, however, go looking in perception for the necessary access to the properties of the world that enable us to couple our resources to those in the environment. We consider the origin of behaviour to be the dynamical system formed by this coupling; the system provides a set of constraints and behaviour emerges as the constrained system works over time.

It's sometimes a little hard to tease these suggestions apart: after all, they  typically both predict that we succeed at the task at hand. When studying people, the best way to try and separate these two suggestions out is to examine how we succeed. For instance, in catching a fly ball, the brain-based prediction solution says we will run in a direct line to where we think the ball will land; the perception-based coupling solution suggests we will run along curved paths as we attempt to move so as to produce the information required. In this case, data supports the latter hypothesis, but it's not always that easy.

Sometimes, you've just got to start from scratch and build yourself a robot.

Review: Louise Barrett's "Beyond the Brain"

There are a lot of books about embodied cognition. Like psychology itself, the idea that cognition might not all be in the head has fractured off into about 5 distinct flavours, most of them quite annoying and pointless.There are also quite a few books about the thesis of the 'extended mind' (Clark & Chalmers, 1998) ; the idea that cognition quite literally extends into our bodies and environments, in terms of the tools we use and the objects we interact with. It's getting quite hard to find a book on these topics that isn't a) simply rehashing old ground or b) trying to come up with it's own distinct flavour of embodied, extended cognition that fixes some problem of everyone else's. I find this sort of fracturing and endless bickering quite distracting and troublesome, for reasons best expressed by xkcd. I also agree with Tony Chemero, that much of this work is still, sometimes deep down, representational. I think this is because they fail to embrace a sufficiently useful theory of information, such as that proposed by Gibson. So I often find myself reading these books finding nothing new and far too much that's depressing.

Louise Barrett's book, "Beyond the Brain: How Body and Environment Shape Animal and Human Minds" is, refreshingly, not like this at all. Barrett is a psychologist who studies animal cognition and behaviour, and her book does a lot of things very, very well. I'm not planning on a chapter by chapter book club on this, although I may at some point; Eric Charles has posted a few thoughts on the book as well, here, here, here, and here. To cut to the chase: if you are new to the area of embodied cognition, read this book. If you're familiar with the literature but want a clear, well-structured presentation of many of the key ideas, then read this book. If you're bored with the same old examples and want some new, perhaps more convincing examples of embodied cognition in action, read this book. And if you have heard some of the arguments but still think behaviour really comes from the computational activity of our complex brains, then, for the love of science, read this book.

Embodied solutions to neural delays: Information and Network Motifs

One of the bugbears of direct perception is the fact of neural delays. The transmission of signals through the nervous system takes time, and this means that there is a lag between something happening (at, say, the retina) and that event having consequences in cortex, let alone behaviour. In control theory terms, delays in a system can lead to instability in that system's behaviour as you are forced to make corrections that are then incorrect and must themselves be corrected.

It's typically suggested in psychology that these delays are compensated for via computational predictions; the nervous system 'perceives the present' by taking the lagged input and using it as the basis of a guess about what's going on now (e.g. Changizi et al, 2008, plus see this post from a while back). This is a problematic idea: if the perceptual control of actions is based on a guess compensating for a variable time lag, then the stability issue remains, not to mention the consequences for mis-predicting the future. Regardless, it's not really an option for a theory of direct perception, and I want to discuss a couple of options.

Coordination dynamics and relative speed

The Bingham model of coordinated rhythmic movement makes three predictions. First, it predicts that movement stability is a function of perceptual ability, and we confirmed this in two ways (by showing how people can move stably at non-0° with transformed visual feedback (Wilson et al, 2005) and by showing that perceptual learning of 90° led to improved movement stability without practice at the movement task; Wilson et al, 2010). This prediction is also supported by recent work by Kovacs and Shea, who are busy demonstrating that transformed, Lissajous feedback breaks the classic pattern of movement stability in coordination tasks. The second prediction is that relative phase is specified by the relative direction of motion; we confirmed this by selectively perturbing various components of motion and showing selective effects on performance (Wilson & Bingham, 2008). 

The third prediction was that the detection of relative direction was conditioned on the relative speed; the latter was simply a noise term. de Rugy, Oullier & Temprado (2008) tested this prediction by using an amplitude manipulation to alter the relative speeds. Their data did not support the model predictions, and they concluded that the approach taken by the Bingham model was flawed. We recently replicated their experiment (Snapp-Childs, Wilson & Bingham, in press as of Friday; download) and identified numerous critical flaws in their design and analysis which invalidated their criticism.

Task Specific Devices and the Perceptual Bottleneck

I've been wanting to blog this paper, Bingham (1988; download link), for some time, and I've had the excuse to be reading it this week as I develop a grant. There's a lot here, and many of these brief points are worth posts in and of themselves. My goal here was to create a walk through of the paper, and I hope to dive into some of these issues in more detail.

This paper comes from Geoff Bingham, my PhD advisor at IU. And, like most of the good things Geoff has taught me over the years, this paper is a gift that keeps giving as I come to grips with what's in it. What it does is lay out a methodological problem (the massive redundancy and complexity of the human action system), proposes a solution (studying task-specific devices) and firmly embeds the idea that these devices are intrinsically perception-action devices (by discussing the so-called perceptual bottleneck). In effect, it lays out a way to be a productive scientist studying a hugely complex system without shying away from the complexity. This paper blew my fragile little mind when I first read it, and I'm still pulling good ideas from it today.

This paper is what I think the science of perception-action should look like. It's the piece I think Chemero (2009) is missing for his radical embodied cognitive science, and it contains (oddly without a lot of specific references) all the key ideas that have come up on this blog in a single coherent frame work (e.g. Gibson & specification; Turvey et al on the symmetry principle). Frankly, if you want to study perception-action systems from a dynamical systems perspective, this is what you have to acknowledge is the lay of the land and these are the beginnings of the tool kit you'll need.

Perception, Action & Dynamical Systems

Over Easter I visited the Center of Functionally Integrative Neuroscience at Aarhus University in Denmark, courtesy of the Interacting Minds group. I gave a talk, got the tour, and met some of the faculty and students - some interesting opportunities for future collaborations, I hope - thanks for the hospitality!

I wanted to lay out the basics of the talk I gave. I took the opportunity to present some ideas that have been developing as I work on this blog, reading Chemero and working on coordination experiments. There is a core of people in Aarhus interested in things ecological, as well as dynamical systems, so it was a good audience to try these ideas out and they seemed to go over well. This is also the sketch of a paper Sabrina and I are going to work on over the summer.

The take home message of the talk was simple - dynamical systems is the right kind of mindset for cognitive science, but it is not a theory of behaviour. Dynamics merely provides the right kind of modelling tools - the form of the model must be based on hypotheses about the specific kind of dynamical systems we are or else they are merely an exercise in data-fitting. Ecological psychology is the right theory, and the Bingham model of coordinated rhythmic movement is currently the only example of a genuinely perception-action dynamical systems model. My thoughts here are largely from my response to Chapter 4 of Chemero (on 'the dynamical stance') and Chapter 5, his initial attempt to use dynamics to serve as a guide to discovery which I think fails and which Chemero then replaces with ecological psychology. The description of Bingham's model comes from here.

Chemero (2009) Chapter 7: Affordances, etc (Pt 2)

Last time I went over affordances-as-dispositions, and Chemero's first swing at affordances-as-relations. Affordances can't be dispositions, claims Chemero, because

1. Dispositions manifest when the conditions are met; this is compulsory. But I am not currently trying to effect all the affordances in my vicinity, so they can't be dispositions. Relations are functions, and thus support malfunctions.
2. Dispositions require complements - for perception-action, the complement of an affordance is an effectivity. But what exactly is this? Body scale (e.g. leg length)? Actually, it's more likely in terms of ability (per some unpublished experiments Chemero has run); people's judgements of stair climbability are a relation between the riser height and the person's ability to step that high.
3. If affordances are properties that are directly perceived, then when two people perceive the same affordance their minds will overlap: the problem of two minds. Relations solve this problem by making the overall relation which the directly perceived affordance is part of unique to each observer.

This would be all well and good, except that 

1. Affordances and effectivities are complex dispositions, and the conditions for being realised can be a long list. In addition, I can only be one kind of effecting device at a time, so when seated I am literally not capable of complementing the climbing affordances of my stairs at that moment in time.
2. Noting that 'body scale' is an imperfect proxy for an effectivity, and then claiming that this means nothing is an effectivity makes no sense. In addition, 'abilities' are equally approximate. The issue (being careful what you claim is the actual complement of the affordance) is valid but applies equally to dispositions or relations.
3. The solution to the problem of two minds that Heft outlined and Chemero thinks supports his case lies in making the act of perception relational, not the thing perceived. The affordance does not, itself, need to be a relation.

So far, nothing has convinced me that affordances need to be relational. But to round the story out, I want to finish the chapter and address the final tweak Chemero adds: Affordances 2.0.

Chemero (2009), Chapter 5: Guides to Discovery

The dynamical stance laid out by Chemero in the previous chapter has a potential flaw (besides being a bit weak-ass) - it's not clear how it can serve as a guide to discovery. How do you do productive science taking this approach? Chemero is going to make two suggestions, only one of which I think works: first, he's going to suggest dynamical models such as the Haken-Kelso-Bunz (HKB) model can serve to stimulate empirical work even when they are entirely phenomenological. This approach is, I think, entirely incorrect, and this chapter is full of serious problems (only some of which are unique to Chemero). Second, he's going to suggest that Gibsonian ecological psychology can actually solve the problem much more robustly anyway, by serving as an underlying theory of behaviour. This will work better, and I would advocate Bingham's model of coordination as an exemplar of this, more promising route.

But first, the HKB model as guide to discovery (this chapter is largely the material from Chemero, 2000; I intend to turn this post into a paper to rebut that paper and point to the Bingham model as an alternative, so comments are especially welcome on this one). Time to get a little critical, I'm afraid.

Chemero (2009), Chapter 4: The Dynamical Stance

The problem with the theory of representation Chemero settles on in the previous chapter, as we shall see, is that it makes representation unavoidable. If there are representations, then RECS fails to get off the ground (remember, the radical bit is anti-representationalism). In this chapter, Chemero identifies a path past this problem (the 'dynamical stance'), explains why it works, but then concludes with a new problem, that of how to guide future discovery.

Identifying the Visual Information for Relative Phase

Bingham's model predicts that the information for relative phase is the relative direction of movement. The first direct test of this hypothesis was the experiment that followed on from my learning study, in which we systematically perturbed the various candidate information variables to see which affected performance in the perceptual judgement task.

I like this study a lot, if I do say so myself. It's a serious attempt to make a strong test of the model's predictions, and we invested a lot of time in the methodology. This is also that rare paper that benefited from a vigorous review process; the end result is, I think, a clear, careful, and detailed presentation of a critical result for the perception-action approach Geoff and I are developing.

Readers interested in the issue of how you can scientifically study information from an ecological perspective should certainly read the paper (Ken, that's you :); it's my go-to reference for how I believe this has to be done. The main lesson - it's hard to do this properly, but the rewards, in terms of unambiguous data, are clear.

What Does Coordinated Rhythmic Movement Have To Do With Anything?

In which I provide an answer to a question I get asked by everyone, including grant reviewers, students and random people who make the mistake of asking what I do for a living.

I've spent numerous recent posts talking about coordinated rhythmic movement. This is my bread-and-butter experimental task, my go-to example for studying all aspects of perception, action and learning. I'm branching out, now I have my own faculty position, but coordination is where it's at. 

The single most common question I get is "why study this? Surely it's just some fake movement task; I mean finger-wiggling, who ever even does that?" I wouldn't mind so much, but I even get this question from grant reviewers, scientists who should know the answer. Doing science properly is important, but communicating that my methods achieve this matters too, not least because today's funding climate demands it.

So, coordinated rhythmic movement: what the hell?

A Perception/Action Model of Coordination

Other coordination posts are here.

The role of perception in the dynamic leading to the HKB phenomena has been made clear by the work so far. But the exact form of this dynamic had yet to be modelled; the HKB model only consists of two cosine functions superimposed on each other to produce the two attractors at 0° and 180°, and the dynamic pattern hypothesis it embodies made predictions which did not held up empirically. It was now time to take an explicitly perception/action approach to modelling the task, which means it's finally time to turn to (my PhD advisor) Geoff Bingham's model (2001, 2004a, 2004b). This model is a fully perception/action model, and the modelling strategy Bingham lays out is, I think, a masterclass in how to go about building models of this kind.

What else could it be? The case of the centrifugal govenor.

Previously, I’ve dismissed the idea of mental representation because 1) no one knows what a representation is and 2) the arguments for representation tend to be pretty weak. Now, I’d like to spend a bit of time discussing a possible alternative – a dynamical systems approach to cognition. To frame this discussion, I’m going to summarise a very handy philosophy paper by Van Gelder (1995) in which he distinguishes between a computational and dynamical solution to a particular problem (see also Andrew's post on the polar planimeter). Van Gelder has clearly picked a side - that cognition emerges from dynamical systems and that cognitive processes are evolutions in the state-space within these systems. One of the main arguments for computation is that it’s difficult to imagine what else could be going on (see footnotes p. 346 for references for this argument). So, Van Gelder wrote this paper,  not to decisively rule out computation, but to provide an answer to the question “what else could [cognition] be?”