Direct Learning (Jacobs & Michaels, 2007)

The ecological hypothesis is that we perceive properties of the environment and ourselves using information variables that specify those properties. We have to learn to use these variables; we have to learn to detect them, and then we have to learn what dynamical properties they specify.

Learning to detect variables takes time, so our perceptual systems will only be able to become sensitive to variables that persist for long enough. The only variables that are sufficiently stable are those that can remain invariant over a transformation, and the only variables that can do this are higher order relations between simpler properties. We therefore don't learn to use the simpler properties, we learn to use the relations themselves, and these are what we call ecological information variables. (Sabrina discusses this idea in this post, where she explains why these information variables are not hidden in noise and why the noise doesn't have to be actively filtered out.)

Detecting variables is not enough, though. You then have to learn what dynamical property that kinematic variable is specifying. This is best done via action; you try to coordinate and control an action using some variable and then adapt or not as a function of how well that action works out.

While a lot of us ecological people studying learning, there was not, until recently, a more general ecological framework for talking about learning. Jacobs & Michaels (2007) proposed such a framework, and called it direct learning (go listen to this podcast by Rob Gray too). We have just had a fairly intense lab meeting about this paper and this is an attempt to note all the things we figured out as we went. In this post I will summarise the key elements, and then in a follow-up I will evaluate those elements as I try and apply this framework to some recent work I am doing on the perception of coordinated rhythmic movements.

Affordance-based control (Fajen 2005, 2007)

The most commonly studied tasks in the ecological approach involve the perceptual control of actions such as interception and steering. These models all involve perceiving some variable and moving so as to null the discrepancy between a current value and an ideal value. However, none of these approaches involve the perception of affordances; specifically, none of them address how people work to keep the required corrections possible, given their action capabilities. Fajen (2005, 2007) proposes affordance-based control, an ecological research framework that brings these questions to the fore and leads to the discovery of new, affordance based control strategies that account well for the data and solve the problems of simple information-based control models. 

My current sense is that Fajen is absolutely correct in his assessment of the problems and has done sterling work developing an ecological solution. What follows is a brief description of the problems and his solutions; in the future I will blog some thoughts as I work to align my throwing affordance work with this framework.

A Gibsonian analysis of linguistic information

This post is based on a talk I just gave at the Finding Common Ground Conference at the University of Connecticut. Please excuse the Power Pointy nature of some sections! You might need to Ctrl+ to see some of the images clearly.  I have made some changes from the original talk content on the basis of very useful feedback I received from other conference attendees.

What is the place of language in ecological psychology? Is language a type of direct perception? Is language comprehension direct perception? Does language have affordances?

In trying to answer these questions I discovered that some things we think of as being perceptual have a lot in common with the conventionality of language and that some language-related behaviours look a lot like perception (as typically construed). I end up suggesting that we move away from talking about 'perception' and 'language' as different types of entities and instead focus on information / behaviour relations in specific tasks.

Ecological indirect perception

I want to follow on from Sabrina's important posts about information, and why psychology should be about information for the forseeable future. Sabrina's taxonomy includes information beyond what ecological psychologists talk about, information that needs to be investigated to find out exactly what kind of behaviour it can support. (This is what Gibson did for perception, and it paid off in spades.) 

I'm particularly interested in the information in pictures and mirrors; surfaces that present information about being surfaces and about being something else. This post is me thinking out loud about the implications; these are by no means my final thoughts on the matter, it's me taking the taxonomy for a spin and seeing where I end up. Feedback welcome!

The Information Available in Pictures

I've become fascinated with the problem of pictures and how they relate to the things they are pictures of. One reason is the regular use of pictures of objects to study how the affordances of those objects might ground cognition; this, I think, is a major problem. 

A more positive reason is that, like language, pictures contain information about something they themselves are not (see Sabrina's information taxonomy). I have a hunch that an ecological study of picture perception might help guide an ecological study of language, because the former can take more direct advantage of the work already done about how we perceive meaning in events via ecological laws but then act as a bridge, a point along the way to the conventional world of language meaning.

Finally, the topic seems to be woefully understudied in the ecological approach. There is some, however. In the comments section on my rant about using pictures to study affordances, I was pointed to the work of John Kennedy (a Gibson student, now emeritus at the University of Toronto). I have downloaded his 1974 book, 'A Psychology of Picture Perception' and am working my way through it. Matthieu de Wit then linked me to an archive of a discussion, in papers, between Gibson and Ernst Gombrich about picture perception. I thought I'd start with Gibson (1971), The Information Available in Pictures, to begin to sketch out what we know and what we don't.

The current question at hand is, can pictures provide the same information about the things they depict?

Specification: What It Is, and Why We Need It (Specification I)

The first thing I need to do in a discussion of specification is explain what it is and why it's important to ecological psychology. I've tried to maintain a clear logical progression in this post, building towards the need for specification. In my next post, I'll take a first swing at explaining what specification gives us, namely a reason why information means one thing and not another.

The issue of specification comes from Gibson's (1966, 1979) analysis of visual perception, so that's where I'll start too. Most descriptions of visual perception begin with the anatomy of the eye; people note that the eye resembles a camera, and that the lens seems to focus a messy, upside down image onto the retina. The retina then pixelates that image into neural activity, and this pixelated structure then shows up in primary visual cortex (this is topographic mapping). If vision does indeed begin this way, then a huge amount of work seems to be required to take this impoverished stimulus and use it as the basis for the rich, 3D visual world we experience.

Gibson's ecological theory begins with a re-evaluation of the stimulus for vision. The first three chapters of the 1979 book are about the world and what it contains, while chapter 4 is about how this world can interact with light to produce information. Only once he lays out the information available to the organism does he begin to talk about the act of perception itself; this re-evaluation of the 'job description' for a visual system is one of his most important contributions to psychology. Gibson's reanalysis leads him to conclude that action relevant properties of the world (specifically, affordances) can be specified in the optic array, and this concept underpins the directness of his theory of perception.

The issue of specification is assumed to be critical for the success of a direct theory of perception. The traditional views propose a 'many-to-one' mapping; a given pattern of stimulation on the retina is ambiguous because it could be caused by many possible states in the world. Specification is the hypothesis that there is a 'one-to-one' mapping - a given pattern in the optic array comes from one and only one state of the world. This can happen, according to Turvey, Shaw, Reed & Mace (1981) if (and only if) the creation of information about the world is a lawful process. If the projection of world into optics is underwritten by a law and thus one-to-one, then detecting the optical pattern is equivalent to detecting the property of the world: detecting the information is perceiving the world, with no additional processing work required. Perception can be direct.

A theory of direct perception will require several elements: there must be invariant structure within the endless flow across the retina that relates 1:1 to some property of the world. To be invariant, this structure must be relational, and therefore higher order. If perception is to be direct, these higher-order invariants must be detectable as a piece, and not built out of their elements in some post-perceptual process. Only if you have all this do you have the possibility of a one-to-one mapping between the world and vision, i.e. the possibility of specification.This post lays out what this all means, and  how these pieces come together in ecological psychology.

Language: A task analysis (kind of)

In the last post, I discussed the similarities and differences between language and other types of information. From the first person perspective, spoken language is just another type of auditory event. The main distinction between the word "dog" and the sound of a dog barking is that the auditory event of barking is about the thing that caused the sound - a barking dog - while the auditory event of the word "dog" is not about the thing that caused the sound - a human speaker. The word "dog" is (usually) about an animal that is related to the auditory event by convention. Thus, the sound of a barking dog conveys auditory information and the sound of the word "dog" conveys linguistic information.

In this post I want to lay out classes of tasks in which linguistic information is useful. As a starting point, I will identify situations where language appears to fill a gap, although at this point these are no more than general descriptions. In any specific task analyses that might eventually follow, the basic strategy will be to begin by asking what perceptual resources exist to carry out the task. If perceptual resources are unable to explain task performance and if linguistic resources are available, then these will be considered for their potential contribution. It might be helpful to think of linguistic and perceptual information as occupying different niches in a task space. Perceptual information helps me to walk and catch a fly ball and linguistic information helps me do the types of things described below.

Some Ground Rules for a Theory of Psychology

Add psychology to the list

A fairly common response to our theory post was 'here's my theory, which is designed to replace and fix all the others'. However, it's more a symptom of the problem I was discussing than a solution for everyone to have their own entirely separate theory which doesn't talk to any other work in the field (see above). One of my personal goals in science is to not be that guy. I want to see cognitive science become more integrated, not more fragmented. We have also been asked, however, and quite sensibly, what we think the solution to our problem is. The question then is how to propose a theoretical approach for psychology and cognitive science where we don't just reinvent the wheel.

Sabrina and I have been working on this for, well, the entire blog. It has been a place for our "brave attempt to think out loud about theories of psychology until we get some" since day one; we've been identifying problems but, just as importantly, solutions the whole time. The theory post identified the big picture problem we see in psychology; time to lay out some solutions.

Step one is to present a map of the blog, organised thematically to guide new readers to work we've already done here. This should also help map out the gaps in the approach, so we can focus on things to do next; feel free to point us to problems we can't yet address! (And yes, we know about episodic memory and language - we're working on it.) This post is not a comprehensive summary of past work - it's a map for you to use to find what we've done so far.

To summarise: in essence, and some minor details aside, we are advocating for Chemero's (2009) radical embodied cognitive science, with the addition of some elements he was missing (network science & task specific devices). Cognition is embodied, extended and held together by the direct perception of affordances and events; the result is a complex, nonlinear dynamical system that must be analysed as such. The brain is not the sole source of our behaviour, nor is it representing the world; it clearly plays a critical role in this system, though, and we propose that we'll need the tools of network science to describe what it's actually up to (Sporns, 2010). Methodologically, we must carefully characterise the task, the resources available to solve the task (which include brain, body and environment) and the information these resources create which can sustain the formation and control of an embodied solution. This method is Bingham's (1988) task specific device approach (the main piece Chemero was missing, I think).This approach applies to all and any behaviour you want to explain, including the hard stuff like episodic memory and language.

Critically, this approach, while new (and uncommon in insisting on a role for Gibson's ecological approach) isn't just something we invented: all these elements are active parts of modern cognitive science. The only new part is bringing it all under one roof, with the goal of getting on and getting some decent normal science under our belts.

Here's what we've covered so far. If you want more details on any point, click on the links!

Theory, and Why It's Time Psychology Got One

Psychology has a problem. We have no core theory to guide our research; no analogue to the theories of evolution or relativity. When particle physicists recently found that some neutrinos had apparently travelled faster than light, it never actually occurred to them that this is what had happened. On the basis of the extraordinarily well supported theory of relativity, everyone went 'huh, that's weird - I wonder what we did wrong?', and proceeded to use that theory to generate hypotheses they could then test. It would take a lot of fast neutrinos to disprove relativity.

Psychology, though, when faced with an empirical result that violates the laws of physics, can't find any principled reason to reject the result and instead spends a lot of time squabbling about whether Bem's result might possibly be true because 'quantum'. Worse, when people do replicate the experiment and fail to support the original result, they can't get their 'null result' published. It's a bit embarrassing, really.

One of the problems of having no core theory is that you can't simply rule things out as options. Psychologists almost all consider this a strength: we can pick and choose from a variety of mechanisms which enables us to cope with our messy and erratic subject matter. Can't imagine how perception can explain a result? Just hypothesise a mental representation to fill the gap. After all, no single theory is going to account for the opportunistic and idiosyncratic behaviour of people, so why limit ourselves? We tried that with behaviourism, and it got us nowhere. Let's stay flexible.

Review: Louise Barrett's "Beyond the Brain"

There are a lot of books about embodied cognition. Like psychology itself, the idea that cognition might not all be in the head has fractured off into about 5 distinct flavours, most of them quite annoying and pointless.There are also quite a few books about the thesis of the 'extended mind' (Clark & Chalmers, 1998) ; the idea that cognition quite literally extends into our bodies and environments, in terms of the tools we use and the objects we interact with. It's getting quite hard to find a book on these topics that isn't a) simply rehashing old ground or b) trying to come up with it's own distinct flavour of embodied, extended cognition that fixes some problem of everyone else's. I find this sort of fracturing and endless bickering quite distracting and troublesome, for reasons best expressed by xkcd. I also agree with Tony Chemero, that much of this work is still, sometimes deep down, representational. I think this is because they fail to embrace a sufficiently useful theory of information, such as that proposed by Gibson. So I often find myself reading these books finding nothing new and far too much that's depressing.

Louise Barrett's book, "Beyond the Brain: How Body and Environment Shape Animal and Human Minds" is, refreshingly, not like this at all. Barrett is a psychologist who studies animal cognition and behaviour, and her book does a lot of things very, very well. I'm not planning on a chapter by chapter book club on this, although I may at some point; Eric Charles has posted a few thoughts on the book as well, here, here, here, and here. To cut to the chase: if you are new to the area of embodied cognition, read this book. If you're familiar with the literature but want a clear, well-structured presentation of many of the key ideas, then read this book. If you're bored with the same old examples and want some new, perhaps more convincing examples of embodied cognition in action, read this book. And if you have heard some of the arguments but still think behaviour really comes from the computational activity of our complex brains, then, for the love of science, read this book.

Task Specific Devices and the Perceptual Bottleneck

I've been wanting to blog this paper, Bingham (1988; download link), for some time, and I've had the excuse to be reading it this week as I develop a grant. There's a lot here, and many of these brief points are worth posts in and of themselves. My goal here was to create a walk through of the paper, and I hope to dive into some of these issues in more detail.

This paper comes from Geoff Bingham, my PhD advisor at IU. And, like most of the good things Geoff has taught me over the years, this paper is a gift that keeps giving as I come to grips with what's in it. What it does is lay out a methodological problem (the massive redundancy and complexity of the human action system), proposes a solution (studying task-specific devices) and firmly embeds the idea that these devices are intrinsically perception-action devices (by discussing the so-called perceptual bottleneck). In effect, it lays out a way to be a productive scientist studying a hugely complex system without shying away from the complexity. This paper blew my fragile little mind when I first read it, and I'm still pulling good ideas from it today.

This paper is what I think the science of perception-action should look like. It's the piece I think Chemero (2009) is missing for his radical embodied cognitive science, and it contains (oddly without a lot of specific references) all the key ideas that have come up on this blog in a single coherent frame work (e.g. Gibson & specification; Turvey et al on the symmetry principle). Frankly, if you want to study perception-action systems from a dynamical systems perspective, this is what you have to acknowledge is the lay of the land and these are the beginnings of the tool kit you'll need.

Perception, Action & Dynamical Systems

Over Easter I visited the Center of Functionally Integrative Neuroscience at Aarhus University in Denmark, courtesy of the Interacting Minds group. I gave a talk, got the tour, and met some of the faculty and students - some interesting opportunities for future collaborations, I hope - thanks for the hospitality!

I wanted to lay out the basics of the talk I gave. I took the opportunity to present some ideas that have been developing as I work on this blog, reading Chemero and working on coordination experiments. There is a core of people in Aarhus interested in things ecological, as well as dynamical systems, so it was a good audience to try these ideas out and they seemed to go over well. This is also the sketch of a paper Sabrina and I are going to work on over the summer.

The take home message of the talk was simple - dynamical systems is the right kind of mindset for cognitive science, but it is not a theory of behaviour. Dynamics merely provides the right kind of modelling tools - the form of the model must be based on hypotheses about the specific kind of dynamical systems we are or else they are merely an exercise in data-fitting. Ecological psychology is the right theory, and the Bingham model of coordinated rhythmic movement is currently the only example of a genuinely perception-action dynamical systems model. My thoughts here are largely from my response to Chapter 4 of Chemero (on 'the dynamical stance') and Chapter 5, his initial attempt to use dynamics to serve as a guide to discovery which I think fails and which Chemero then replaces with ecological psychology. The description of Bingham's model comes from here.

Chemero (2009) Chapter 7: Affordances, etc (Pt 2)

Last time I went over affordances-as-dispositions, and Chemero's first swing at affordances-as-relations. Affordances can't be dispositions, claims Chemero, because

1. Dispositions manifest when the conditions are met; this is compulsory. But I am not currently trying to effect all the affordances in my vicinity, so they can't be dispositions. Relations are functions, and thus support malfunctions.
2. Dispositions require complements - for perception-action, the complement of an affordance is an effectivity. But what exactly is this? Body scale (e.g. leg length)? Actually, it's more likely in terms of ability (per some unpublished experiments Chemero has run); people's judgements of stair climbability are a relation between the riser height and the person's ability to step that high.
3. If affordances are properties that are directly perceived, then when two people perceive the same affordance their minds will overlap: the problem of two minds. Relations solve this problem by making the overall relation which the directly perceived affordance is part of unique to each observer.

This would be all well and good, except that 

1. Affordances and effectivities are complex dispositions, and the conditions for being realised can be a long list. In addition, I can only be one kind of effecting device at a time, so when seated I am literally not capable of complementing the climbing affordances of my stairs at that moment in time.
2. Noting that 'body scale' is an imperfect proxy for an effectivity, and then claiming that this means nothing is an effectivity makes no sense. In addition, 'abilities' are equally approximate. The issue (being careful what you claim is the actual complement of the affordance) is valid but applies equally to dispositions or relations.
3. The solution to the problem of two minds that Heft outlined and Chemero thinks supports his case lies in making the act of perception relational, not the thing perceived. The affordance does not, itself, need to be a relation.

So far, nothing has convinced me that affordances need to be relational. But to round the story out, I want to finish the chapter and address the final tweak Chemero adds: Affordances 2.0.

Chemero (2009) Chapter 6: Information and Direct Perception

In the previous chapter, Chemero laid out his first idea as to how a radical embodied cognitive science could be a science; he suggested taking a 'dynamical stance' in which researchers use simple dynamical systems models such as the HKB to drive empirical work. In this chapter, he moves towards his more recent suggestion, namely using Gibson's ecological psychology as a theoretical basis from which to make predictions and run experiments. This is a much more robust idea; sciences need theories or else they wander from phenomenon to phenomenon, which I believe to be a real issue in modern psychology and cognitive science.

Chemero has his own flavour of ecological psychology, however. He believes it stays true to Gibson's essential framework but expands it to cope with new and important issues. The two pillars of ecological psychology, information and affordances, remain. Chapter 6 is Chemero's expanded notion of information, while Chapter 7 is his updated theory of affordances as relations (which I've covered briefly before).