Language, thought and the ecological approach; A Purple Peril

As part of a class on cognitive psychology, I give a seminar in which we talk about the research on the relationship between language and thought. In particular, I show this great talk by Lera Boroditsky as a starting point. She talks about the kind of research in this area, and talks about results such as how we linguistically interact with space and time affecting how we physically interact with these things. For example, some languages like English use an egocentric frame of reference when talking about space (e.g. describing things as being to the left or right, where the origin of this space is the speaker). Other languages use a geocentric frame of reference (e.g. describing things as being to the south of you). In order to be able to speak and understand the language, you therefore have to be able to remain oriented in space, and speakers of these kinds of languages have been shown to be capable of impressive feats of dead reckoning previously thought impossible in humans. 

The reason this is all interesting is in the context of how the field is changing how it thinks about language; is it magical, or merely interesting? If the former, language becomes a unique human cognitive capacity that requires specific neural mechanisms that serve language and nothing else. If the latter, language becomes an integrated part of our cognitive systems and we should expect it to show these connections to other capacities. 

The weight of evidence right now I think favours the latter view. In fact, one whole strand of embodied cognition (Shapiro’s ‘conceptualisation’ hypothesis strand) explicitly pursues these connections between language and other capacities, for example Lakoff’s work on metaphors being grounded in action. Language, while still phenomenal in what it can do, is not different in kind to the rest of cognition. 

The field is still very much at the ‘functional model’ stage of developing explanations, however. The research mostly just catalogues linguistic differences and cognitive differences and works to map those onto each other in a fairly metaphorical, word-association kind of way (e.g. politics is talked about in terms of left and right wing so this should connect to physical movements to the left and the right). Our ecological questions has become, what kind of mechanism might allow this kind of cross-talk, and as I’ve been chatting to students I’ve been connecting a few dots for myself. This post sketches the outline of a mechanistic, ecological research programme for attacking the fascinating problem of the relationship between language and thought.

Thoughts on Ding et al (2015) "Cortical tracking of hierarchical linguistic structures in connected speech"

I happened to be reading Cummins (2000) paper “’How does it work?’ vs. ‘What are the laws?’ Two conceptions of psychological explanation”, when my Twitter feed announced that Chomsky was right and we do have grammar in our heads after all. The Twitter buzz concerned a new Nature Neuroscience paper by Ding and colleagues called “Cortical tracking of hierarchical linguistic structures in connected speech.” You can find it online here. Curious whether I needed to completely overhaul my understanding of language, I tracked down the paper and read it this morning. The method employed is sensible, the results are fairly clear, the analyses seem legit (though I’m not a neuroscientist). So, why am I not worried that everything I thought I knew about language is wrong?

From Specification to Convention (A Purple Peril)

I previously laid out how specification works and why it's important to the ecological approach. Read that first, because I build on it a lot here. I also laid out the corollary of specification, that it allows that information to come to be something an organism can actually use to coordinate and control functional behaviour. Here, I think out loud about how convention might be able to do similar work, because of Sabrina's work (here, and published now here; read that paper for the extended detail on this) expanding ecological information to handle tasks such as language where specification is not always an option.

This is in part an attempt for me to get my head around some implications of Sabrina's analysis. My plan here is to develop an analogy to specification. This analogy will detail the work specification does to make information informative, then try to lay out how conventions fills this role. The goal is to see if conventions can support behaviour without needing representational help. The answer will be yes, because all the differences between law based and convention-based information are 'behind the curtain', only visible from the third-person perspective. From the first person perspective of the organism, all it gets is structures in energy arrays it can try to use to organise behaviours. Conventions place no special learning burden on the organism (Golonka, 2015) and this means that convention can support behaviour with any representational enrichment the same way specification can. (The hidden differences do have consequences, however, so I will map that out a bit.)

There are many things I have not attempted to explain and as usual this reflects my current thinking, not necessarily my final thinking. I look forward to hearing what questions this leaves unanswered for the reader as a way to move this discussion forwards.

A Gibsonian analysis of linguistic information

This post is based on a talk I just gave at the Finding Common Ground Conference at the University of Connecticut. Please excuse the Power Pointy nature of some sections! You might need to Ctrl+ to see some of the images clearly.  I have made some changes from the original talk content on the basis of very useful feedback I received from other conference attendees.

What is the place of language in ecological psychology? Is language a type of direct perception? Is language comprehension direct perception? Does language have affordances?

In trying to answer these questions I discovered that some things we think of as being perceptual have a lot in common with the conventionality of language and that some language-related behaviours look a lot like perception (as typically construed). I end up suggesting that we move away from talking about 'perception' and 'language' as different types of entities and instead focus on information / behaviour relations in specific tasks.

A taxonomy of information

Over the past several months I've been thinking about how perception falls within a hierarchy of types of information use. This was spurred by my ideas about an ecological approach to language, in which perceptual information and linguistic information are distinguished on the basis of the relationship between event structure and meaning. As part of this work, I defined perception as the apprehension of structure in an energy array where 1) the structure is specific to an event or property in the world, 2) where the meaning of the structure (for that organism in that task) is about that event or property (i.e., a dog's bark is about the event of a barking dog), and 3) where the meaning of the structure must be learned (or, more correctly, where an organism must learn how to coordinate action with respect to this structure). I arrived at this definition because it seemed to capture the ecological approach to perception and because it makes it obvious how perceptual information and linguistic information differ (also because I am crazy-obsessive about definitions).

'Embodied Cognition Is Not What You Think It Is' - the paper!

Whoops, we did it again - a paper based on the blog! This time we are in press at Frontiers in Psychology, in a Research Topic on embodied cognition, with a paper we somehow got away with calling 'Embodied Cognition is Not What You Think It Is'. 

This paper draws from a lot of posts on the blog on embodied cognition, perception-action and language. We have used this opportunity to tackle some key issues head on, and we like this paper a lot :) We cover all the important issues and we set up what we think is the way forwards for embodied cognitive science. In addition, it sets up the ground work that we want to build on with our own Research Topic on Radical Embodied Cognitive Neuroscience. We've laid out what we think is the task facing the brain; this is what the brain is engaging with, and so this is what we think neuroscience needs to work with in order to understand what the brain is doing.

It's the kind of paper that will either land with a splash or vanish without trace. We want it to make some serious waves, and we're hoping that we can encourage people to publish free Commentaries on it at Frontiers, to challenge us or pick up our challenges, and, most fun for all, to come work with us to take all this forwards! We want this to be the basis of an empirical research programme and we want you all to work with us on it :) At the very least, feel free to pepper us with questions; this paper is the start of something for us, not the end and we're interested in the response to this paper to frame the next step.

Why does linguistic information mean what it does?

Sabrina has been working on a series of posts on an ecological analysis of language (here, here and here, plus more on the way). Her focus has been on the nature of the information for language, and the similarities and differences this information has with the information for perception. We're working some of this analysis into a paper, and writing that got me thinking about this in a little more detail.

Our main move on language is to reject the assumption that language is a qualitatively different kind of task than perception & action. The goal is to find ways to talk about these behaviours using the same basic analysis tools. Part of that is to draw the analogy to how perceptual information gets its meaning and use that to describe how linguistic information gets its meaning.

What I want to do here is just map this analogy out a little, because I ended up in an interesting place and I want feedback from people who know more than us on this about whether this is just plain crazy. In particular, if you know anything about the relationship between neural dynamics and the dynamics of speech, we think this is going to be relevant!

Language: A task analysis (kind of)

In the last post, I discussed the similarities and differences between language and other types of information. From the first person perspective, spoken language is just another type of auditory event. The main distinction between the word "dog" and the sound of a dog barking is that the auditory event of barking is about the thing that caused the sound - a barking dog - while the auditory event of the word "dog" is not about the thing that caused the sound - a human speaker. The word "dog" is (usually) about an animal that is related to the auditory event by convention. Thus, the sound of a barking dog conveys auditory information and the sound of the word "dog" conveys linguistic information.

In this post I want to lay out classes of tasks in which linguistic information is useful. As a starting point, I will identify situations where language appears to fill a gap, although at this point these are no more than general descriptions. In any specific task analyses that might eventually follow, the basic strategy will be to begin by asking what perceptual resources exist to carry out the task. If perceptual resources are unable to explain task performance and if linguistic resources are available, then these will be considered for their potential contribution. It might be helpful to think of linguistic and perceptual information as occupying different niches in a task space. Perceptual information helps me to walk and catch a fly ball and linguistic information helps me do the types of things described below.

Language isn't magical (but it is special)

One of the most common comments about ecological psychology is that it's hard to imagine how it could apply to things like language. The sense is that language is a completely different kind of beast than perception-action and that it requires a completely different theoretical account (cognitive psychology). Andrew and I disagree. In this post I outline the similarities and differences between language and other types of perceptual information. The main idea is that language is indeed the same type of thing as perception-action, but there are key differences between them in the relationship between the information and what it means. These differences permit language to be flexible according to context, culture, and goals; to be expandable according to changing needs; and to be portable, allowing us to access information about things that are not currently in the environment. These properties make language special, but not magical.

An Ecological Approach to Language

Language is often held up as an example against the possibility of the radical (non-representational) psychology we advocate for. You might be able to explain perception-action without representations, people say, but we can't see how you'll ever be able to explain 'real cognition, like language' without them. It's finally time for us to begin chipping away at this criticism. In the next few posts I'll lay out a first draft of an embodied, ecological analysis of language use.

Psychologists usually assume that catching a fly ball and talking about catching fly balls are two different kinds of thing. I reject this assumption (it is just an assumption) and I am going to treat language use as the same kind of thing as other examples of embodied cognition. Treating language as just another instance of embodied cognition allows me to import the lessons learned from perception-action type tasks and apply these to language tasks. This will lead to very different questions about language use than are typical in the literature. The next post will describe what I mean by this in some detail.

Shared neural resources for throwing and language: a whacky idea for an experiment

For some time now, there has been an hypothesis floating around in evolutionary biology that the human capacity for language emerged, in part, from the development of our ability to throw long distances with high speed and accuracy. There are a few reasons to think this, mostly correlational, inferential kinds of reasons, but they are accumulating.

We were chatting one day about how to test this hypothesis a bit more directly, and we came up with a whacky experiment. We'd like advice from neuroscientists with experience in brain stimulation techniques about whether this sort of thing is feasible. We'd also like to brainstorm the logic of this experiment and see if we can come up with a practical design that stands a chance of finding something. We then need collaborators; I can handle the throwing side (analysis, measurement, etc) but we don't know anything about TMS and would need an expert on board.

There are many other reasons why this might fail, though - I still need to do a detailed lit review on the throwing/language references I have. Our main problem is that we don't know the kind of obvious difficulties in doing TMS in this kind of context. We'd like to assemble a) an experiment and b) a research team to do the experiment if we can get it to make sense, and if it works we will submit the hell out of this to Nature :)

Did language emerge from the neural systems supporting aimed throwing?

Aimed throwing is surprisingly uncommon in the animal kingdom. Humans do it par excellence, and otherwise it only shows up occasionally, even in our closest relatives. Chimpanzees will throw things (often faeces) but unlike humans don't throw things when hunting or trying to get food; when non-human animals throw things, it's usually part of a social encounter.

Throwing is a fascinating task for many reasons; I hope to blog some about the perception-action aspects of this task in the future as I prepare a couple of papers on the topic with my colleagues Qin Zhu and Geoff Bingham (who have previously done some excellent work on throwing to a maximum distance and the size-weight illusion; various papers available here). There are many fascinating questions about the perception of the affordances of throwing and distances to targets which we're starting to tackle empirically.

Biomechanically, throwing an object accurately over any distance requires the precise transmission of force from the large trunk muscles along a kinetic chain formed by the segments of the arm. The large trunk muscles generate forces the arm cannot, and this force is then transmitted by the motion of the arm; each segment weighs progressively less and so the force accelerates each one faster than the last. The end result is a hand moving at high speed. This requires careful timing; if the motion of the segments aren't coordinated carefully you will waste energy moving the limbs in ways that aren't helping the throw.

There has been some speculation for a while now that the neural mechanisms that help support this fine tuned coordination and control for throwing might also be just the kind of resources that could support the development of spoken language. Speech is a complex action that requires exquisite control over the coordination and timing of numerous elements, just like throwing. One hypothesis is that our ancestors began to develop the ability to throw long distances (this being favoured by natural selection processes because it enabled us to hunt and kill huge prey with much less physical risk to ourselves; e.g. Calvin, 1983). Evolution selected for neural resources that supported this activity, and this then opened the door to the possibility of complex spoken language. So do we speak the way we do because we throw the way we do?

How Universal Is The Mind?

If someone asked you to describe the psychological aspects of personhood, what would you say? Chances are, you'd describe things like thought, memory, problem-solving, reasoning, maybe emotion. In other words, you probably list the major headings of a cognitive psychology text-book. In cognitive psychology, we seem to take it for granted that these are, objectively, the primary components of "the mind" (even if you reject a mind/body dualism, you probably accept some notion that there are psychological processes similar to the ones listed above). I've posted previously about whether the distinction between cognitive and non-cognitive even makes sense. But, here, I want to think about the universality of the "mind" concept and its relationship to the modern view of cognition.

In fact, this conception of the mind is heavily influenced by a particular (Western) cultural background. Other cultures assign different characteristics and abilities to the psychological aspects of personhood. Wierzbicka (2005) delves into this problem in detail. She argues that speakers of a particular language make assumptions about what must be universal based on their own ability to imagine doing without a certain concept. Important cross-cultural differences in meaning become lost in translation. For instance, Piaget’s “The moral judgment of the child” was translated to English by substituting the French “juste” with the English “fair.” So, English readers think they are reading about the development of fairness in children, when this was not the author’s intention.

Translation is a deep problem, but it is often ignored in psychology. Generalisations about cognition must be made in some language, but, language is specific to particular cultures. Our choice of language, then, inevitably will bias how we talk about cognition across cultures.

“If we uncritically formulate some hypothetical universals in one particular natural language, for example, English, we run the risk of distorting them by imposing on them the perspective embedded in that particular language; and the same applies to our description of cultural differences” (Wierzbicka, 2005; p. 257).

So, back to the mind and our current view of cognition. Cross-linguistic research shows that, generally speaking, every culture has a folk model of a person consisting of visible and invisible (psychological)  aspects (Wierzbicka, 2005). While there is agreement that the visible part of the person refers to the body, there is considerable variation in how different cultures think about the invisible (psychological) part. In the West, and, specifically, in the English-speaking West, the psychological aspect of personhood is closely related to the concept of "the mind" and the modern view of cognition.But, how universal is this conception? How do speakers of other languages think about the psychological aspect of personhood?

In Korean, the concept "maum" replaces the concept "mind". "Maum" has no English counterpart, but is sometimes translated as "heart". Apparently, "maum" is the "seat of emotions, motivation, and "goodness" in a human being" (Wierzbicka, 2005; p. 271). Intellect and cognitive functions are captured by the Korean "meli" (head). But, "maum" is clearly the counterpart to "mind" in terms of the psychological part of the person. For example, there are tons of Korean books about "maum" and body in the same way that there are English texts on "mind" and body.

The Japanese have yet another concept for the invisible part of the person - "kokoro"."Kokoro" is a "seat of emotion, and also, a source of culturally valued attention to, and empathy with, other people" (Wierzbicka, 2005; p. 272). To illustrate the contrast between "kokoro" and "mind", Wierzbicka gives the following example: A Japanese television programme proclaims, "The 21st century should be the age of kokoro. Let's make a point of meeting with other people" (Hasada, 2000: 110). If an English speaker declared the 21st century to be "the age of the mind" then "meeting with other people" probably would not be a priority - thinking and knowing would be. In contrast to the Korean "maum", "kokoro" is not associated with will and motivation ("hara" meaning belly serves this purpose in Japanese). But, "hara" is not associated with the psychological component of the body, the way "kokoro" is. In other words, "maum" is all about motivation and "kokoro" is all about feelings and "mind" is all about thinking.

Interestingly, Russia, which kind of sits between East and West uses "dusa" as the counterpart to the psychological part of the person. "Dusa" is often translated as "soul", but also sometimes as "heart" or "mind." "Dusa" is associated with feelings, morality, and spirituality. The "dusa" is responsible for the ability to connect with other people. This meaning seems to lie somewhat more with the Eastern conception than with the highly cognitive concept of "mind."

In a larger sense, the fact that there seems to be a universal belief that people consist of visible and invisible aspects explains much of the appeal of cognitive psychology over behaviourism. Cognitive psychology allows us to invoke invisible, internal states as causes of behaviour, which fits nicely with the broad, cultural assumption that the mind causes us to act in certain ways.

To the extent that you agree that the modern conception of "cognition" is strongly related to the Western, English-speaking view of "the mind", it is worth asking what cognitive psychology would look like if it had developed in Japan or Russia. Would text-books have chapter headings on the ability to connect with other people (kokoro) or feelings or morality (dusa) instead of on decision-making and memory? This possibility highlights the potential arbitrariness of how we've carved up the psychological realm - what we take for objective reality is revealed to be shaped by culture and language.



Hasada, Rie (2000). An Exploratory Study of Expression of Emotions in Japanese: Towards a Semantic
Interpretation. Ph.D. thesis, Australian National University..

Wierzbicka, A. (2005). Empirical universals of language as a basis for the study of other human universals and as a tool for exploring cross-cultural differences. Ethos, 33(2), 256-291.