Language, thought and the ecological approach; A Purple Peril

As part of a class on cognitive psychology, I give a seminar in which we talk about the research on the relationship between language and thought. In particular, I show this great talk by Lera Boroditsky as a starting point. She talks about the kind of research in this area, and talks about results such as how we linguistically interact with space and time affecting how we physically interact with these things. For example, some languages like English use an egocentric frame of reference when talking about space (e.g. describing things as being to the left or right, where the origin of this space is the speaker). Other languages use a geocentric frame of reference (e.g. describing things as being to the south of you). In order to be able to speak and understand the language, you therefore have to be able to remain oriented in space, and speakers of these kinds of languages have been shown to be capable of impressive feats of dead reckoning previously thought impossible in humans. 

The reason this is all interesting is in the context of how the field is changing how it thinks about language; is it magical, or merely interesting? If the former, language becomes a unique human cognitive capacity that requires specific neural mechanisms that serve language and nothing else. If the latter, language becomes an integrated part of our cognitive systems and we should expect it to show these connections to other capacities. 

The weight of evidence right now I think favours the latter view. In fact, one whole strand of embodied cognition (Shapiro’s ‘conceptualisation’ hypothesis strand) explicitly pursues these connections between language and other capacities, for example Lakoff’s work on metaphors being grounded in action. Language, while still phenomenal in what it can do, is not different in kind to the rest of cognition. 

The field is still very much at the ‘functional model’ stage of developing explanations, however. The research mostly just catalogues linguistic differences and cognitive differences and works to map those onto each other in a fairly metaphorical, word-association kind of way (e.g. politics is talked about in terms of left and right wing so this should connect to physical movements to the left and the right). Our ecological questions has become, what kind of mechanism might allow this kind of cross-talk, and as I’ve been chatting to students I’ve been connecting a few dots for myself. This post sketches the outline of a mechanistic, ecological research programme for attacking the fascinating problem of the relationship between language and thought.

The Affordances of Natural vs Designed Environments (A Purple Peril)

I had a conversation with Eric Brymer (@Ericbrymer) a few weeks ago. Eric is a Reader at Leeds Beckett and is an ecologically minded psychologist interested in the effects of the natural world on mental health. We chatted affordances for a while, and despite it being a very interesting chat, I really wasn’t sure if I had anything to say about the differences between the natural and designed worlds. 

But I have not been able to stop thinking about this topic, and now I think there’s something fairly cool here.

Relational Theories of Affordances are Functional, Not Mechanistic (A Purple Peril)

Everyone and their dog has a theory about how to interpret Gibson's famously vague definition of affordances;

The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill....an affordance is neither an objective property nor a subjective property; or it is both if you like. An affordance cuts across the dichotomy of subjective-objective and helps us to understand its inadequacy. It is equally a fact of the environment and a fact of behavior. It is both physical and psychical, yet neither. An affordance points both ways, to the environment and to the observer.

There are two basic flavours of theories: affordances as dispositional properties of the environment (Turvey, Shaw, Reed & Mace, 1981; Turvey, 1992) and affordances as relational features of the animal-environment system. (A recent paper has just claimed they should be best understood as events, but to be honest I don't really know what's motivating this). The two most recent and popular relational accounts are Chemero's book (Chemero, 2009; see these posts on the relevant chapter) and Rietveld & Kiverstein (2014) in a paper entitled 'A Rich Landscape of Affordances'. Their goal, like most of the relational accounts, is to handle higher-order cognition by scaling up affordances to support it (our move, in contrast, has been to expand the uses of perceptual information; Golonka, 2015; Golonka & Wilson, 2016 preprint).

I am firmly in the 'affordances as dispositional properties' camp (see, for example, the discussion section of my recent throwing paper for an extended analysis). Specifically, they are dynamically defined dispositional properties of objects and events in the context of tasks. The reason is that this is the only way affordances can be the kind of thing that can create information and therefore be perceivable. They have to be 'out there' and made of things that light can bounce off, for example, and relations between organism and environments are not typically such things. In addition, if they do not exist until perceived, we need a story to explain how we come to learn to perceive them, and there is no viable ecological framework that will make this happen (Wilson et al, 2016).

Reading this material with my new mechanism glasses on has given me a new, concise way to identify the problems with these relational accounts:

Affordances-as-relations theories are all functional explanations, and not mechanistic explanations

Information is typically dense and continuous (A Purple Peril)

Optic flow is everywhere, all the time (same with other energy arrays, like the acoustic array). We depend on this fact deeply. When we are cast adrift from information, our behaviour quickly accumulates errors and strays in often disastrous ways. One example is the case of friction, which doesn't exist until two surfaces are in contact and therefore does not create information about itself that is available ahead of that contact. In another example, when cut off from landmarks, people walk in huge circles, getting seriously lost and confused; Souman, Frissen, Sreenivasa & Ernst, 2009). A simple version of this is the game of walking with your eyes closed; you quickly lose all confidence about where you are and what's happening and it's actually very difficult to make yourself walk at normal speed. 

The Perilous proposal is that behaviour emerges in real time, as a function of the current flow of information, and that this flow is typically dense and continuous, not intermittent. I will illustrate this with an example of two designed sets of instructions for navigating though a building, where the dense information set leads to better, more stable behaviour.

Tasks from the First Person Perspective (A Purple Peril)

The great snare of the psychologist is the confusion of his own standpoint with that of the mental fact about which he is making his report. I shall hereafter call this the ‘psychologist’s fallacy’ par excellence.

William James, The Principles of Psychology

This is a video of a baby trying bacon for the first time. The baby gets really really excited, and everyone around him goes 'Ha! Babies love bacon as much as the rest of us, this is great!'. And everyone laughs and cheers.

Except here's the thing. I think what this baby really likes is making his loved ones laugh and cheer. The bacon is fine, but I don't think it's the magical experience his parents assume. They are making the psychologist's fallacy: mistaking what you think is going on for what the person is actually experiencing. (It's our fallacy because our subject matter makes us uniquely susceptible.)

When people come into our labs to take part in experiments, we present them with a situation that we have designed to elicit a specific behaviour from them, and that we manipulate in various ways in order to probe the makeup of that behaviour. We therefore think we know what the person is doing: they are doing the thing we asked them to do. However, this isn't necessarily true, and in order to figure out what our participants did and why, we need to consider how they experienced the experiment. In effect, doing our science right means taking the first person perspective of our participants when we formulate our explanations. 

I take this idea primarily from Louise Barrett's excellent book, Beyond the Brain: How Body & Environment Shape Animal and Human Minds (which I reviewed here). The first couple of chapters spend a lot of time talking about anthropomorphism, and why it's a problem. To be honest, when I read the book I didn't quite know why Louise started with this. But over time, I've realised what an extraordinarily powerful point it is and we now talk about all the time. 

Peril Proposal: The psychologist's fallacy is real, but the ecological approach to understanding task dynamics and the information they create offers a useful framework for avoiding it while we science.

Oh crap. Re-thinking van Gelder (A purple peril)

I have this problem where I like pretty much everything William Bechtel writes except when it pertains to cognitive science. It's annoying because, even when I disagree with him, I think he's worth taking seriously. This was on my mind when I started reading is 1998 paper, "Representations and cognitive explanations: Assessing the dynamicist's challenge in cognitive science."

In this paper, Bechtel critiques van Gelder's construal of the Watt's governor as an exemplar of how cognition could function without representations. I have always liked van Gelder's argument and wrote about it years ago in very favorable terms. I wouldn't say that I don't like it anymore, but Bechtel may have convinced me that one of van Gelder's central claims - that representations don't feature in explanations of the Watt governor - is incorrect. Does this mean that I now think that cognition requires internal representations? No! But, the language Bechtel uses to assess van Gelder's arguments sounded eerily familiar and I realized that the criteria used to label something as a representation are met by...ecological information.

From Specification to Convention (A Purple Peril)

I previously laid out how specification works and why it's important to the ecological approach. Read that first, because I build on it a lot here. I also laid out the corollary of specification, that it allows that information to come to be something an organism can actually use to coordinate and control functional behaviour. Here, I think out loud about how convention might be able to do similar work, because of Sabrina's work (here, and published now here; read that paper for the extended detail on this) expanding ecological information to handle tasks such as language where specification is not always an option.

This is in part an attempt for me to get my head around some implications of Sabrina's analysis. My plan here is to develop an analogy to specification. This analogy will detail the work specification does to make information informative, then try to lay out how conventions fills this role. The goal is to see if conventions can support behaviour without needing representational help. The answer will be yes, because all the differences between law based and convention-based information are 'behind the curtain', only visible from the third-person perspective. From the first person perspective of the organism, all it gets is structures in energy arrays it can try to use to organise behaviours. Conventions place no special learning burden on the organism (Golonka, 2015) and this means that convention can support behaviour with any representational enrichment the same way specification can. (The hidden differences do have consequences, however, so I will map that out a bit.)

There are many things I have not attempted to explain and as usual this reflects my current thinking, not necessarily my final thinking. I look forward to hearing what questions this leaves unanswered for the reader as a way to move this discussion forwards.

Brains Don't Have to be Computers (A Purple Peril)

A common response to the claim that we are not information processors is that this simply cannot be true, because it is self-evidently the case that brains are transforming and processing information - they are performing computations. Greg Hickok throws this ball a lot, and his idea is clear in this quote from his book 'The Myth of Mirror Neurons':

Once you start looking inside the brain you can’t escape the fact that it processes information. You don’t even have to look beyond a single neuron. A neuron receives input signals from thousands of other neurons, some excitatory, some inhibitory, some more vigorous than others. The output of the neuron is not a copy of its inputs. Instead its output reflects a weighted integration of its inputs. It is performing a transformation of the neural signals it receives. Neurons compute. This is information processing and it is happening in every single neuron and in every neural process whether sensory, motor, or “cognitive.”

Hickok, pg 256.

There are two claims here. First, neurons are processing information because their input is not the same as their output; they are transforming the former into the latter. Second, this process is computational; 'neurons compute'.

This is a widely held view; psychologist Gary Marcus even wrote about this in the NYT saying 'Face it, your brain is a computer'. In response, Vaughn Bell at Mindhacks posted about this op-ed and this issue in a nicely balanced piece called 'Computation is a lens'. He sums up the issue nicely by asking 'Is the brain a computer or is computation just a convenient way of describing its function?'. The answer, I propose here, is that computation is a fantastically powerful description of the activity of the brain that may or may not be (and probably isn't) the actual mechanism by which the brain does whatever it does. This is ok, because, contra Hickok,  not every process that sits in between an input and a different output has to be a computational, information processing one. 

Function vs Structure (A Purple Peril)

One of the apparently controversial things that I say is that psychology, as a science, needs to address function before it gets worried about structure; what is the brain trying to do, vs how is it doing it? This Peril lays out the argument in a little more detail. As always, this is my current thinking not my final thinking and I am happy as ever to hear arguments for and against this proposal.

Structure (the details of how a function is implemented) is important, there is no doubt. But I see two related arguments about putting function first, or at least giving it the driver's seat in our science.

Specification and Some of Its Consequences (A Purple Peril)

Perception is how we maintain psychological contact with functionally relevant objects and events in our environments. Explaining how we do this means describing that environment in appropriate terms and investigating what information might possibly exist for that environment, given that description. The ecological hypothesis is that the correct level is dynamics, and that describing the environment this way allows there to be information that can specify those dynamics. This information can support the kind of behaviour we need to exhibit. 

This Purple Peril describes what is meant by specification, and what that implies for how information comes to mean something to an organism. There is more detail in the various links, so check those for information too.

(Re)Introducing "The Purple Perils"

'Note for a tentative redefinition of behaviour', 1975. 

James J Gibson ran an afternoon seminar for many years at Cornell. This seminar was widely attended by a variety of students, professors and visiting scholars, and Gibson used them as a place to try out new ideas, new ways of describing those ideas and generally hammer his theory as hard as he could to see where the weak and strong points were.

Before many of these seminars, he would write and distribute a note, often fairly detailed, detailing the topic of that week's meeting. These were copied as he typed by 'ditto' sheets which transferred the text onto multiple sheets of paper. The ink on these sheets was purple, and the notes became known as the Purple Perils. (There is a nearly complete archive of these online hosted by Bill Mace.) 

These Perils were never the final word on anything. They reflected Gibson's current thinking and were always up for debate. That debate often found it's way into the next week's seminar and Peril, so these were always works in progress.

I've just finished reading Ed Reed's Gibson biography (there's a couple of taster chapters here) and I've been inspired by his description of the kind of scientist Gibson was; the Perils and his seminar were good examples of the kind of rigour and openness he embraced in his science. So, in this spirit, we are going to start posting some shorter, more focused posts on specific topics and ideas from the ecological embodied cognitive science we are developing. They will reflect our current thinking on the topic, but not necessarily our final thinking, and they will not aim to solve everything, just move it forward. They will typically include some 'facts of the matter' as seen from the ecological view, and some analysis to reach some conclusions and hypotheses based on those facts. The goal is to stimulate debate and discussion and come away with a better, clearer theory. 

We would like to invite you all to come get into it with us; ask questions, challenge us, agree with us (we like this too!), talk to each other in the comments and argue/agree with each other. Keep it friendly, keep it a little focused on the topic and try to be specific, detailed and clear in your arguments. If specific parts of topics become sticking points they will likely show up in future Perils for a more focused discussion; otherwise they will be what we're trying to clarify for ourselves. If there are particular topics that are bugging you about radical embodied cognitive science, post about it here and they may show up as future Perils. "What about language then, huh? Huh?" is not something we can work with, though, sorry - be specific :)

We cannot promise to have time to do one every week (Gibson didn't have to teach anything else!) but we will post them as frequently as we have things to post. Feel free to steal the idea and host something similar on your own blogs; we will link to them here. Gibson valued a good argument above all other things in science, and I entirely agree, so please take any opportunity you see to make this a dialogue people can all take part in.