From Specification to Convention (A Purple Peril)

I previously laid out how specification works and why it's important to the ecological approach. Read that first, because I build on it a lot here. I also laid out the corollary of specification, that it allows that information to come to be something an organism can actually use to coordinate and control functional behaviour. Here, I think out loud about how convention might be able to do similar work, because of Sabrina's work (here, and published now here; read that paper for the extended detail on this) expanding ecological information to handle tasks such as language where specification is not always an option.

This is in part an attempt for me to get my head around some implications of Sabrina's analysis. My plan here is to develop an analogy to specification. This analogy will detail the work specification does to make information informative, then try to lay out how conventions fills this role. The goal is to see if conventions can support behaviour without needing representational help. The answer will be yes, because all the differences between law based and convention-based information are 'behind the curtain', only visible from the third-person perspective. From the first person perspective of the organism, all it gets is structures in energy arrays it can try to use to organise behaviours. Conventions place no special learning burden on the organism (Golonka, 2015) and this means that convention can support behaviour with any representational enrichment the same way specification can. (The hidden differences do have consequences, however, so I will map that out a bit.)

There are many things I have not attempted to explain and as usual this reflects my current thinking, not necessarily my final thinking. I look forward to hearing what questions this leaves unanswered for the reader as a way to move this discussion forwards.

When information is not used to perceive the dynamic that created it
The specification set-up works great for the continuous control of actions. You want to organise your behaviour with respect to the dynamics that created the information, because those dynamics are the things that define the task and therefore also define what actions are a suitable, successful complement to that task. Not all patterns in ambient energy arrays are used as the perceptual stand-ins for the local dynamics that created them, however, so the specification analysis, while very important, is incomplete.

Let's take a specific example. In the case of the linguistic behaviour speech, a person moves their articulatory systems in such a way as to create certain kinematic patterns in the acoustic array (and, as a side effect, patterns in the optic array as well, which can support lip reading and also facilitates auditory perception of speech). The relevant dynamics are those of the articulatory systems. The kinematic patterns are indeed specific to those dynamics. But, critically, successful linguistic interactions do not happen if you organise your behaviour with respect to those articulatory dynamics. If you try to behave as if the information "means" or "is about" the articulation dynamics that created it, you will fail to produce functional linguistic behaviour. So interacting with speech as a language act cannot be supported by the Kinematic Specification of Dynamics (KSD).

This is usually where people abandon ship and say 'well the ecological approach is good for perception-action but it can't handle language, I'm out'. We don't frighten this easily, so let's work this through.

To succeed in producing a functional behaviour, you need to use this kinematic pattern as if it were caused by something other than the dynamics of the articulatory system. You need to use it as if it were caused by the structure of the relevant language. This structure is not created by the laws of physics, but it's one that we as a language community have agreed on. That is, it is conventional. How can a nebulous thing like a convention shape what kind of information that structure becomes?

A First Thought, Quickly Rejected
There is no specification between words and the things to which they refer; the word "dog" has no compulsory relationship to the animal in question, as evidenced by the fact that no other language refers to them that way. However, a more ecological framing of that question might be, can there be a specification relation between a language dynamic and the pattern in the acoustic array? That is, is there one pattern per linguistic structural feature? To answer this, we would need to analyse the speech act into the correct structural features, the way we had to analyse tasks at the level of dynamics for specification to work in the continuous control of action. If there can be such a specification relation, then the organism can simply use the pattern to stand in for the language structure because a 1:1 mapping means this works out well (as per specification). 

But this won't work. Even if we constrain the scope of the relationship to a single language, it's not clear what the relevant functional unit would be for specification to work. It certainly isn't phonemes, or words. Language just doesn't seem to exhibit the kind of compulsory relationship between itself and the information created by using it that makes specification an option in perception-action. I include it here just because laying it out in order to rule it out seemed worthwhile.

Figure 1. Wittgenstein knew the deal

Conventions
Given the answer was 'no' to any kind of specification relationship, what is the relationship between the pattern and the language structure? We need to identify this relationship, because it will define what is required to support linguistic behaviour. Chemero (2009) and Golonka (2015) have used the term convention as a contrast to the law based relationship underpinning specification (Turvey, Shaw, Reed and Mace, 1981). A conventional relationship is one that is sustained by agreement: if enough people act as if one thing (e.g. some language information) means another thing (e.g. some language act) then for all intents and purposes, that's what it means, and the fact that there is nothing compulsory about the form of the relationship is neither here nor there (see the linked cartoon). The word "dog" can therefore refer to the animal in question to English speakers even though there's no particular connection between the word and the thing. 

Chemero formalises how conventions do the work required to support structures being informative with situation semantics (see Chemero, chapter 6). My big concern then was that situation semantics only works if you have access to one part of the relation (either the dog or the word "dog", for example) and access to the constraint (the fact that English uses "dog" to refer to dog). Where does the constraint live, though? Chemero had no clear answer and I was left with the concern that the constraint would have to be some kind of supplementary knowledge, hardly the sort of thing I want in a cognitive system. I said at the time "well maybe conventions can be shown to do the same sort of work behind the scenes as laws" although at the time I was not confident. Well, that analysis (among a few other important things) has now happened in Golonka (2015; Sabrina will blog the paper in detail later) so it's time to think about how this plays out.

Here, I just want use the work I did in the specification Purple Peril to lay out an analogy to convention that makes it clear how conventions can work just as ecologically as laws in creating information.

How conventions can promote structures to information
Let's draw out the analogy between specification and convention with respect to how it allows a structure in an energy array to serve as information.

1. Specification is a fact of the (ecological level) laws of physics that govern how dynamics are projected into energy arrays. The ecological level is simply the world of medium sized things that don't travel particularly fast - you know, the world outside your window. 

• Organisms learn to use information in place of dynamics and get away with it because the laws have provided a context (a 1:1 mapping) in which that substitution relationship between them is there and so operating as if it's there works out. 
• Organisms do not need to 'know' about these laws; they don't need to carry these laws around with them to guide inferences based on those laws. We have no choice but to act using only information (the bottleneck problem) and it happens to work because of the laws. Evolution was therefore able to build perceptual systems that use information in place of dynamics because it happens to work out, and no organism needs to know why it works in order for those perceptual systems to do so. 
• Organisms demonstrate that they know what the information is standing in for by using that information to coordinate and control appropriate behaviours. For example, I show that I "know" tau specifies time-to-contact if I use it to control interception and avoidance behaviours that require time-to-contact information to succeed. (Note: yes I know that tau is not very good information for time-to-contact, but it's still a useful example). 

2. Conventional structure does not work this way; its form is not a compulsory function of the local dynamics the way law based structure is. The law based dynamics of a fly ball can only produce one set of information variables and 1:1 mappings are an option, but you can talk about fly balls using multiple different conventional systems, e.g. different languages and the 1:1 relationship goes away. However, some critical parallels remain:

• Organisms can use language information to stand in for what the language is about and get away with it because, in their linguistic environment, enough people act as if the relationship exists that it works well enough. The agreements can break down or change (e.g. English speakers recently agreed that 'literally' now also means 'figuratively') but this generally all holds together long enough to work
• Organisms still do not need to store the rules anywhere. As with perceptual information, your job is to organise your behaviour with respect to the information and this only works if the behaviour you try to organise can be supported by the information in question. The organism doesn't need to know why this works; it only needs to learn that it works, and this suggests we don't have to carry language (or any conventional system) around with us any more than we carry the laws of physics. In both cases, we learn to behave in the presence of information. Studying syntax, semantics etc is to study a language's specific implementation of the convention relation, but it's not studying the things the organism needs to carry around with it in order for it to implement the relation. Situation semantics isn't the deal.
• Organisms show that they know what the linguistic information is standing in for by using that information to select appropriate behaviours. For example, I show that I understand the sentence "Pick up the red cup" by using the information in that utterance to engage in action selection that can be supported by that information (e.g. by picking up the red instead of the blue cup). 

These claims set language up as the same kind of thing as perception-action - learned behaviour exhibited in the presence of information. What kinds of behaviour the information can support varies. Specifying information can support the online control of action because it is about the dynamics that action need to be organised with respect to. Conventional information can support the selection of action because it can distinguish between otherwise equivalent options but it cannot support online control because it isn't about the relevant dynamics. This has consequences but these seem to actually occur; we don't control actions with linguistic information and language behaviour really is much less stable than prehension, etc. So the conventional relationship is not as good as the specification relationship but it is, in principle, good enough for some things and language is one of those things. 

This analysis is just a useful first step to radicalising language. People often say to me "well that idea works for perception-action stuff because of all the information that's around, etc". People are generally ok with this idea. The analysis I've laid out just clarifies that it applies to language too. This is not the complete story; but it creates a foot in the door using something I think people are apparently already happy with, and that's all I'm looking for right now.

Further reading

A Gibsonian analysis of linguistic information
A Taxonomy of Information
Chemero (2009) Chapter 6: Information and Direct Perception: Tony argues that information in general is based in convention and that the right formalism for describing that is the situation semantics of Barwise & Perry.

References
Golonka, S. (2015). Laws and conventions in language related-behaviours. Ecological Psychology, 27(3), 236-250. Download