A selection of problems with representation

As Andrew alluded to in the comments to his last post, I'm cooking up some replies to various questions that have been raised. Being relatively time poor at the moment, I'm going to take these in small bits, rather than attempt a more comprehensive reply. The first thing I want to do is consider a few of the major issues, as I see them, with the notion of representations. As a cognitive psychologist, I spent many years endorsing a representational framework, but over time I have encountered enough problems/limitations with this approach that I consider the idea of representations to be seriously flawed. As a result, I've spent the past couple of years figuring out how to do cognitive psychology without them (but that's another post). What I want to do here are go through some of the things that have convinced me to abandon representations.

1) We don't know what we're talking about when we talk about representations

This criticism goes beyond the fact that there are multiple, competing definitions of representation. It goes beyond the fact that most cognitive psychologists are never asked to seriously wrestle with their own definition of representation, to consider its historical origins, or to consider whether there are alternative approaches to cognition. This criticism is more basic and is summed up nicely by Larry Barsalou: "We have no accounts of how propositional representations arise in the cognitive system, either innately or through experience. We haven't the faintest idea of how biological mechanisms could produce abstract propositions" (Barsalou, 1993, p. 173). Regarding this quote, Linda Smith says "When we cannot imagine how our basic ideas about cognition can possibly be realized, we ought to consider the possibility that they are wrong" (Smith & Jones, 1993, p. 181).


Granted, the field has moved on since the early '90s, but we still don't know much about the relationship between biology and cognition. And, I'm not talking about neuroimaging - interpretations of imaging data are ambiguous enough that they seem to reflect rather than challenge and inform our current understanding. I'm talking about how complex learning and memory are instantiated via networks of cells in a chemical bath.

2) We objectified the problem


Representations were initially invoked as a way to explain why some behavioural responses couldn't be completely predicted on the basis of the stimuli. Based on observations of the environment and our behaviour, it looks like something internal mediates our actions. Andrew has already discussed problems with the poverty of the stimulus argument, so I won't get into that aspect of the issue - there are other examples that are still potentially relevant even with a better understanding of perception/action. For instance, I might perform better on a math test when I am happy rather than sad. Or, I might take longer to respond to a question you ask if I am distracted by other thoughts than if I give you my full attention. Put simply, it is not the case that given a particular external environment I will always respond the same way (there's a good argument to make that objectively characterising an external environment doesn't make any sense because what matters is what goes into the system, not what exists out in the world, but that's a point for another day).

Early cognitive psychologists flagged up this problem and decided to spend some time figuring out what could be happening internally to account for our behaviour. This is a good idea. But, notice that this problem implies nothing at all about the form that internal processes/states/systems/whatever should take. It does not imply that the only way to account for behaviour is via internal discrete, computational representations. And yet, this is the dominant form of representationalism today (you might not know you're a discrete, computational representationalist, but chances are that if you believe in representations that you are one).

So, what do I mean that we've "objectified" the problem? I mean that cognitive psychologists chose to address the problem of internal mediating states with a noun - representation. Nouns are for doing things to. Acting upon. Messing with in some way. If our heads are full of things, of representations, then we must do things to them to get any use from them - you need processes to act on the things. You read a book, file a piece of paper, find a picture of a loved one. By objectifying representations, we treat them like objects to be created, filed, stored, sorted, retrieved, etc. These types of verbs populate much of cognitive psychology. But, creating, filing, storing, sorting, etc are only sensible things to do to discrete objects. If we do not have discrete representational objects in our head then many of the questions asked by modern cognitive psychologists need to be reformulated.

3) Discrete, stable concepts don't explain the most interesting and adaptive aspects of cognition


The traditional cognitive view is that we have object concepts (e.g., "dog") that contain relatively stable information about what we know about things. When these concepts are referred to in various contexts ("The dog chases the cat"; "The big, red dog") we think of basically the same thing. Of course, we're not actually thinking exactly the same thing - the stuff about dogs that comes to mind when you hear "The dog chases the cat" is not the same as the stuff that comes to mind when you hear "The big, red dog".

According to the traditional view, additional processes can modify the information about concepts that we have access to at any given time. Certain information is made "more salient" or is "weighted more heavily" based on goals or tasks. The idea is that we have core concepts, but might think of different aspects of them depending on context. Over time, the content of core concepts can be modified through learning and experience, but this modification is like editing a document - the document (representation) continues to exist, although some content may change over time.

Logically, contextual variability cannot be caused by stable conceptual representations. The variability must be caused by additional processes acting on the representation. As Smith and Jones (1993) note, this breaks cognition into "structure (stability) and process (variability)" making the task of cognitive psychology to figure out what stays the same across "different instances of a single cognitive act" (p. 182). Gelman and Medin (1993) point out that with this partition, the same set of data can be explained in multiple ways - simple representation / complex process, complex representation / simple process. This isn't good. But Smith and Jones (1993) additionally argue that stable concepts can't explain what's "smart" about cognition.

According to the traditional view, cognition is smart because it can represent concepts that reflect abstract and general information, which is most likely to be stable over repeated instances. Smith and Jones argue for a different type of smart cognition that prioritises flexibility and "fit" with a specific situation over stability. In the context of novel word interpretation, they say:

"If there is an abstract represented structure of some kind that sits behind the shape bias [children's tendency to generalise novel object terms according to object shape], it is the least interesting, least intelligent part, of the child's word-learning behavior. All the work that makes novel word interpretations smart is done by those processes that involve the specific objects and specific words at hand. It is these real-time, real-task processes that flexibly adjust attention to find the most likely referent of a specific utterance of some unknown word. Novel word interpretation is not smart because it is stable; novel word interpretation is smart because it is creatively adaptive. And for this kind of intelligence, perception--information about the here-and-now--always matters" (p. 184).  

The nub of their argument is this: If representations are responsible for stability in cognition, but the really useful bits of cognition involve context-specificity and flexibility, then why are representations considered to be so important? This argument isn't a slam dunk against the notion of representations, but it does seriously question the focus of much of cognitive psychology. And, Smith and Jones don't deny that there is stability to cognition, but they do argue against the need for computational representations to account for this stability. Considering the enormous context-sensitivity of all purported cognitive processes, stable, discrete representations don't actually appear to be a very good way of modelling cognition.

So, what else could it be?

4) Representations were invoked to solve a particular problem (see point 2). But other solutions could have been proposed that solve the problem as well and probably better. Unfortunately, these solutions were never really explored and even now are only considered by a small minority of cognitive psychologists.

I've blogged in detail about this issue here, but here's a summary (based on Van Gelder, 1995).

The analogy: A major 18th century engineering problem was reconciling the oscillation of pistons with the rotation of flywheels. Driving a flywheel lets you generate rotative motion, rather than just pumping motion that results directly from piston. In other words, figuring out how to power a flywheel with pistons lets you power a wide range of machines. The trick is getting the flywheel to turn with uniform speed. Flywheels vary in response to the current steam pressure and to the overall engine workload. And, both of these factors, themselves, are variable. A throttle valve allows one to change the pressure of the steam, and therefore control the speed of the flywheel. But, this valve has to be adjusted by just the right amount at just the right time to keep the speed uniform.

The computational solution

One solution to controlling the valve requires something or someone to measure the state of the system at various points in time and adjust the valve by a certain amount in response to those measurements. If the steam pressure is x and the workload is y, then adjust the valve by z. The first characteristic of this type of solution is that it proceeds in stages. The first stage takes measurements. The second stage applies a rule based on those measurements. Because this solution relies on two stages, there is necessarily some time lag between measurement and correction. Depending on the duration of this lag, the correction might be inappropriate for the current state of the system. This type of solution necessitates an executive – someone or something to take account of the state of the system (e.g., “if x”) and then to carry out the appropriate action (e.g., “then y). It also necessitates measuring the difference between things. For instance, the only reason to adjust the valve is if the current speed differs from the speed a second ago.

This is the type of solution implemented in most cognitive models.

The dynamic solution

There is another, radically different way to solve the valve control problem - you can couple the opening of the valve to something that necessarily varies in response to steam pressure and workload in a way that results in constant flywheel speed. By ‘necessarily’ I mean that the physical properties of this thing respond to changes in steam pressure and workload in a particular way. Such a solution responds in one step and does not require measurement. Thus, there is no time lag problem or concomitant source of error. Nor does it require an executive. Hitching the valve control to the flywheel couples the thing you want to control (the valve opening) directly to the thing that embodies the relevant sources of variance (the flywheel speed). This beautiful solution is implemented in the Watts centrifugal governor, an 18th century piece of technology that still works brilliantly.

Consequences for cognition

The computational solution relies on discrete computational symbolic representations. It has to measure and represent flywheel speed, steam pressure, and workload using abstract symbols. Then, it has to apply operations to these symbols in order to calculate how to adjust the valve. This output is a representation that causes the appropriate adjustment to be made by an executive (computer or human worker). The second solution, the one that describes the actual centrifugal governor, is nonrepresentational.  There are no discrete inputs and outputs, no computations performed on inputs and outputs, and no executive in charge.

While cognitive psychologists are often happy to admit that dynamical systems do a good job describing some systems like the centrifugal governor, they are hesitant to admit that dynamics might also characterise complex cognitive behaviour. So, here's a cognitive example:

According to prospect theory we compute the utility of various outcomes of a decision and select the one with the highest utility (Kahneman & Tversky, 1979). This theory clearly depends on discrete representations (i.e., of each option’s utility) and computation (i.e., calculating which option has the largest utility value). But, it is equally possible to describe decision making in terms of state space evolution in a dynamical system. For example, motivational oscillatory theory (MOT; cf. Townsend) describes oscillations resulting from satiation of persisting desires. We approach food when we’re hungry, but not when we’ve just eaten and are temporarily satiated. It’s possible to interpret this behaviour as a decision – when I’m hungry, I decide to eat. But, in Townsend's model there are no discrete states and no algorithmic processes effecting transformation on these states. There is just the evolution of the system over time. See Busmeyer and Townsend (1993) for a fully fleshed out dynamical model of decision making.

Whatever you might think of individual models, it is clear that dynamical systems are a legitimate alternative to the traditional computational approach to modelling cognition. It is also clear that these two approaches lead to very different assumptions about the nature of the underlying cognitive system. The computational approach leads us to think about mental objects (representations) that are manipulated and transformed via cognitive processes. The dynamic approach leads us to think about components in a cognitive system evolving over time and in response to the current context.

Summary

In my experience it's a waste of time trying to convince someone who endorses representational cognition that there are no representations. For one thing, representation is such a broad and ambiguous term that it is unclear what type of evidence could exist that unambiguously rules out representations. Sometimes people just want to keep using the term, so they'll apply it to aspects of dynamical systems (e.g., the location in state space of the MOT model "represents" hunger or satiety). Using representations in this manner doesn't contribute anything to the understanding of the system, and if representations aren't doing any work, then I can't see the point of continuing to invoke them. But this is besides the point. I really wouldn't expect a representationalist to be convinced by anything I've said.

So, what is the point of this post? The point is to demonstrate that representations were originally invoked to solve a particular problem. We currently have no mechanism to explain how representations could be instantiated in a messy biological system, so the representational framework does not have any special biological plausibility going for it. There are also alternative approaches to the problem of modelling mediating states (e.g., dynamical systems) that have several advantages over traditional computational models.

Given these points, I think that cognitive psychology would be vastly improved if cognitive psychologists 1) consider how they came to think that cognition was based on representations. Was it because they were taught from the beginning that cognition was representational? Or was it because they seriously investigated alternative approaches? Did they rule out these alternatives on the basis of data that favoured a representational approach? Or, did they rule them out on the basis of simply not being able to imagine cognition working without representations?

After asking myself these questions, I realised that I had unquestioningly adopted a representational framework because that is what I had been taught and that I had never actually considered alternative approaches. That didn't sit very well with me, and after spending a few years doing my homework I was convinced by the evidence that representations were not the solution to cognition.



Barsalou, L.W. (1993). Challenging assumptions abouy concepts. Cognitive Development, 8, 169-
180.

Busemeyer, J. R. & Townsend, J. T. (1993). Decision field theory: cognitive approach to decision making in an uncertain environment. Psychological Review, 100 (3), 432-459.

Gelman, S.A., & Me,din, D.L. (1993). What's so essential about essentialism? A different perspecfive
on the interaction of perception, language, and conceptual knowledge. Cognitive Development.
8. 157-167.

Kahneman, D. & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263-291.

Smith, L. B. & Jones, S. S. (1993). Cognition without concepts. Cognitive Development, 8, 181-188.

Van Gelder, T. (1995). What might cognition be, if not computation. The Journal of Philosophy, 92 (7), 345-381.