Friday 20 September 2013

Social priming: Of course it only kind of works

Social priming is the field of research about how thinking about or interacting with something (like warm coffee, or old age) can affect later, vaguely related behaviour. (Rolf Zwaan has a useful summary of the theoretical background here and here, and a recipe for how to whip one of these up for yourself here.) It has been a top target for replication efforts in psychology. Although social priming effects in general have been widely demonstrated, many specific results (e.g. priming people to think about old age makes you walk slower; Bargh, Chen & Burrows, 1996) have failed to reliably replicate (even thought the effect sizes for individual studies are often surprisingly large). Most of the attention has been on the work of John Bargh because he basically invented the field (all discussed in this profile from January 2013).  Last year Bargh exploded all over the internet with a bit of a tantrum about these failures on his Psychology Today blog (now deleted, but archived for posterity here and here and discussed in detail by Ed Yong here). This made him something of a punching bag on Twitter, etc and so people are a bit excited that another Bargh social priming result has failed to replicate (oh and hey, here's another, non-Bargh one). Cue panic, gnashing of teeth and reflexive defensive moves by social psychologists (plus coverage of the topic in the NYT).

I'm a bit bemused by it all, really. I am not at all surprised that while social priming works in general, there is wide variation in how well specific social priming tasks work out. Of course priming works - it couldn't not work. But the lack of control over the information contained in social priming experiments guarantees unreliable outcomes for specific examples. Let me see if I can explain what I mean.


Canalisation - Why Priming Basically Works
A useful concept from the developmental biology literature is canalisation. The idea is that systems that have any kind of structure (e.g. genetic or cognitive) are able to use that structure to smooth out variation in outcomes caused by variation in input. This isn't a rigid 'having this gene makes you only able to do one thing'. The idea is more dynamic - the structure acts as a guide to the preferred outcome.

The word canalisation should evoke the formation of a canal by the activity of flowing water. Over time the flow of water creates a preferred path (the canal) and any water that flows along that canal will generally follow the path laid out. Minor variations at the start of the water flow (say, in the exact location of the source of the water) lead to the water taking slightly different paths down the river, but this variation is still structured by the form of the canal so that by the end, the water comes out at the same place. This word is used to describe the kind of robustness you see in complex, nonlinear dynamical biological systems; there is both stability and variability in the overall behaviour of the system as it unfolds over time but (if functioning properly) it's all within the systems's tolerance for error and the net result is still good.

In biological systems, canalisation is like forming a habit, a disposition to act in a certain way. In psychology, most of our dispositions are acquired through experience. Priming is like taking an empty canal formed by the prior water flow of experience and running a little water down it to see what happens. Where the water goes tells you about the form the canal has from its prior history. 

In the social priming literature, therefore, where the 'water' goes should depend on the connections and associations you have in your social knowledge about the world, and it should therefore reveal those connections to the social psychologist. That's what, experimentally, priming is for.


The limits of social priming

People are complex dynamical systems with numerous dispositions and connections lying around based on prior experience. We are also sensitive to all kinds of information in all kinds of formats. Some of that information is social in nature. If you poke this kind of system with a social prime that the system can detect, then that information will indeed flow somewhere, by definition

But where it will flow is underconstrained. Primes are, by definition, merely nudges to a trained system which you then just let run and respond how it likes. Social primes are nudging systems with a lot of room to move; none of our social associations and connections are compulsory that way, say, gravity is. The same people will respond in different ways at different times to the same social prime 'nudge' because at any given moment people are differently 'shaped' systems. This gets worse because social priming studies are typically between-subject designs, and (shock!) different people are even more different from each other than the same people at different times! 

Then there's also the issue of whether the social primes used across replications are, in fact, the same. It is currently impossible to be sure, because there is no strong theory of what the information is for these primes. In more straight forward perceptual priming (see below) if I present the same stimulus twice I know I've presented the same stimulus twice. But the meaning of social information depends not only on what the stimulus is but also who's giving it and their relationship to the person receiving it, not to mention the state that person is in.

In social priming, therefore, replicating the design and the stimuli doesn't actually mean you've run the same study. The people are different and there's just no way to make sure they are all experiencing the same social stimulus, the same information (and remember, information is absolutely where it's at for explaining behaviour). While the system will respond, exactly how and how much will vary over time. 

So frankly I am not surprised that social priming works as a basic effect but that the results of specific attempts to prime produce highly variable results. It just couldn't be any other way, once you'd thought about it for a bit.

A contrast: motor priming
I did a post-doc at the University of Warwick with Friederike Schlaghecken & James Tresilian using masked priming to investigate motor control processes (Wilson, Tresilian & Schlaghecken, 2010, 2011). In this kind of priming, you briefly present a stimulus (say an arrow) that cues the participant about a response option (e.g. press a button with the left vs right hand). You then replace the stimulus with a randomised mask, and then present another stimulus to which the participant actually has to respond. 

Depending on the timing of all this, there are two responses to the priming. There's the positive compatibility effect (PCE) where you are faster to respond if the prime cued the same action. Different timing produces the negative compatibility effect (NCE) where you are actually slower to respond if the prime and the response match. You get a PCE when the prime can be seen or when the response is required right after the mask, and an NCE when the prime is subliminal or there is a longer delay after the mask.

The full details of the NCE are still a little up for grabs, but the basic setup is that it reflects the dynamics of motor cortex as it prepares the response. The prime makes you begin to prepare one movement; the mask interrupts that preparation, making you slam on the brakes. This inhibition then makes it harder to actually do that response if needed. This braking takes time to occur, however, so if you ask a person to respond right after the mask you get in before the brakes hit and you get a PCE. If you give the system just a little more time, however the inhibition has begun and you get the NCE.

So the basic idea is the same as social priming. You give a trained system some incomplete information and then let it just do it's thing. You then poke it at different times to see what's going on, and when you do you find an unfolding pattern of activation and inhibition which relates nicely to the flow of information you are presenting with your stimuli. 

These effects are robust and easy to generate if you do the timing correctly (if you have Matlab and can read instructions you can try it yourself using the toolbox I wrote for Friederike). Why? Because we've exercised a lot of control over the stimuli so we know what is going in, and because the system we're tapping is fairly straightforward and highly canalised so that it responds fairly consistently. This makes priming a useful technique for studying motor control, and suggests that because social priming isn't constrained in this way, priming may simply be too variable a method to find out anything useful.

Some concluding thoughts
The problem with social priming is conceptually similar to the small effects size effect problem (although note that one of the actual problems with social priming is the occasional unrealistically large effect). The problem is that there's just not enough control being exerted on what's going in, and we just don't have enough of a formal description of the system being primed. The solution is the same; more targeted poking of more carefully described systems. I describe two examples at the end of the small effects post; how I broke coordinated rhythmic movement by targeting very specific information variables (Wilson & Bingham, 2008) and how Geoff can turn visual metric shape perception on and off by controlling access to the required large scale (>45°) changes in perspective (e.g. Bingham & Lind, 2008).


Pretty much everything we talk about on the blog these days boils down to this; get as precise as possible (preferably to the point of a decent model; Golonka & Wilson, 2012) about what the hell it is you're doing, and use that to generate testable hypotheses about the kind of poking that should and shouldn't matter. Poking something usually means manipulating information, and so you'd better be prepared to get serious about that too. Until then, social priming (and psychology in general) will just continue working as a general sort of principle, but not work reliably in specific cases. We can probably do better than that.

References
Bargh J.A., Chen M. & Burrows L. (1996). Automaticity of social behavior: Direct effects of trait construct and stereotype activation on action., Journal of Personality and Social Psychology, 71 (2) 230-244. DOI: Download

Bingham, G.P. & Lind, M. (2008). Large continuous perspective transformations are necessary and sufficient for perception of metric shape. Perception & Psychophysics, 70(3), 524-540. Download

Golonka, S., & Wilson, A. D. (2012). Gibson’s ecological approach – a model for the benefits of a theory driven psychology. Avant, 3(2), 40-53. Download

Wilson, A. D., & Bingham, G. P. (2008). Identifying the information for the visual perception of relative phase. Perception & Psychophysics, 70 (3), 465-476. Download 

Wilson, A. D., Tresilian, J. R., & Schlaghecken, F. (2010). Continuous priming effects on discrete response choices. Brain and Cognition, 74, 152-159. Download
 
Wilson, A. D., Tresilian, J. R., & Schlaghecken, F. (2011) The masked priming toolbox: an open source Matlab toolbox for masked priming researchers. Behavior Research Methods, 43, 210-214. Download Link to the toolbox

10 comments:

  1. So two thoughts:

    Firstly, you say social priming isn't surprising. Manifestly, the existence of papers which are are nothing more than reporting the existence of social priming effects testifies that it is surprising to some (many?) researchers. If your account is that priming is prima facie to be expected, this seems as uninformative (but opposite) as the position that the mere fact of this priming deserves reporting. The missing theory work is an account of why and how different bits of cognition are connected to each other (so as to allow priming). Saying it is all information and output is an underspecified level of description. Bargh, at least makes an attempt at this when he says that goals are the locus of action of priming effects.

    Secondly, I don't really understand your point about priming should be inherently variable so we shouldn't be surprised if the effects go away. The point of statistical testing is that we measure and model the variability and make claims based on reliable patterns. There's nothing about between subjects, or test-retest, variability that shouldn't be adequately handled by our experimental/statistical methods. The kind of variability which breaks the reliability of the stats - which you allude to - is the variability in social meaning and social actors (ie if you get different people at different times to whom the primes mean different things). The way you talk about variability in general doesn't seem to be put enough emphasis on different types of variation

    ps I am surprised to hear that a dynamicist like you believes in subliminal primes. We'll have to discuss some time!

    ReplyDelete
    Replies
    1. The missing theory work is an account of why and how different bits of cognition are connected to each other (so as to allow priming).
      Frankly, so is social priming. It's always a fishing trip (let's prime something that might be related and see what sticks).

      Use the masked priming as a model. Masked priming works because it's a structured input into a constrained system with a fairly typical dynamic. We therefore know a lot about why it works the way it does because what happens is stable.

      Saying 'goals' are the locus of something is like saying 'phlogiston' is. It's a made up construct, not a mechanism.

      Manifestly, the existence of papers which are are nothing more than reporting the existence of social priming effects testifies that it is surprising to some (many?) researchers.
      This says more about the researchers not sitting down to do some basic hard work than anything.

      The way you talk about variability in general doesn't seem to be put enough emphasis on different types of variation
      That's fair. My point is more about 'stability' in the dynamics sense than variability in the GLM sense, if that helps.

      ps I am surprised to hear that a dynamicist like you believes in subliminal primes. We'll have to discuss some time!
      Believe in them - I've seen them! :) Well, I've seen it work. Subliminal just means 'at chance on prime identification' as a proxy for 'not consciously aware of the prime'. The information is clearly still getting into the nervous system. consciousness is highly over rated :) Let's chat though - I'm curious!

      Delete
  2. "Until then, social priming (and psychology in general) will just continue working as a general sort of principle, but not work reliably in specific cases".

    This is the kind of statement that makes it difficult to take social psychology seriously as a science. If the supposed evidence for a theory is not sound enough to be replicated then the theory is not supported. It is not okay (or certainly not scientific) to just assert that: " Of course priming works - it couldn't not work." The evidence indicates otherwise.

    ReplyDelete
    Replies
    1. This is kind of my point.

      But a note: there's priming, and then there's social priming. Priming is just the fact that if you provide some information to an embodied nervous system that it can detect, it will start using that information to do something. Social priming is the specific attempt to use social information to see if it does something to reveal how social knowledge is connected. Priming can be true as a general kind of thing that happens, and social priming can be an example that is doomed to be a bit crap, and this isn't a contradiction.

      My point about psychology in general (that we talk about a lot in other posts too) is that psychology in general is more like social priming than masked priming.

      Delete
  3. Sounds like we all violently agree that social priming research is a good example of inadequate psychology! (tom says)

    ReplyDelete
  4. "Manifestly, the existence of papers which are are nothing more than reporting the existence of social priming effects testifies that it is surprising to some (many?) researchers"

    Or, alternatively: these types of studies are easily done using computers, and/or students as participants and therefore are easy ways to get researchers, and their affiliated universities, lots and lots of publications. Who cares about anything else...Athough it is strange that some top tier journals want "new" and "surprising" results, but have no problem with the same recipe (see Zwaan's blog) being done over and over and over and over again. 'Sup with that then?

    You can get a way with that for at least a few decades, who needs a theory anyway when you can build a whole career without any, and journals in the mean time publish lots and lots of highly cited (by a few dozen fellow researchers) papers. Social priming, and maybe social psychology in general, is perhaps well-suited for that kind of stuff.

    ReplyDelete
  5. If "social priming works as a basic effect", as you say, then what *specific* social priming effect can be reliably produced?

    If you're saying the effect "works as a basic effect" but there are zero reliably reproducible results, then what you are saying is exactly like what the ESP people say: "our effects are real but they disappear when you go looking for them, they don't show up when probed by skeptics, etc." That's not science, that's just lame excuses.

    ReplyDelete
    Replies
    1. You need to keep 'priming' and 'social priming' a little distinct. The latter is a not very good example of the basically sound logic of the former.

      I agree that social priming is actually fairly poor science. But the logic of priming (nudging a system by giving it some information) is fundamentally sound.

      Delete
    2. Yes, certainly many cognitive and perceptual priming effects are highly replicable, going back to the Doctor-Nurse priming of Meyer's.

      Delete
  6. Isn't the real problem the outcome of (social) priming? I agree that there are more associations for social categories than more basic semantic categories but in the doctor-nurse example above, we are talking about social categories (and therefore social priming) right ? The doctor-nurse study is about semantic categories where the dependent variable is in the same domain as the presented prime. It gets problematic when (social) priming leads to behavioral outcomes, and this I would call behavioral priming rather than social. That is the real problem I believe, which is also the problem for some embodiment studies, as Rolf Zwaan argues.

    ReplyDelete