In the early 20th century, the world was captivated by a mathematical horse called Clever Hans. He could apparently perform basic arithmetic, keep track of a calendar and tell the time. When his owner, Wilhelm von Osten, asked him a question, Hans would answer by tapping out the correct number with his hoof.
Eventually, it was the psychologist Oskar Pfungst who debunked Hans’ extraordinary abilities. He showed that the horse was actually responding to the expectations of its human interrogators, reading subtle aspects of their posture and expressions to work out when it had tapped enough. The legend of Hans’ intellect was consigned to history. But history, as we know, has a habit of repeating itself.
For the last few decades, psychologists have been using a technique called priming. With subtle hints of words or concepts, they can trigger impressive changes in behaviour. Words of cleanliness can make people behave more morally. Words related to age can slow their bodies. Words of power sharpen our mental abilities. All of these studies have suggested that our behaviour is influenced by subtle things that lie beneath the watch of our conscious awareness.
This view could well be right, but not always in the way that psychologists believe. Stephane Doyen from the Université Libre de Bruxelles has repeated one of the classic experiments in priming and shown that, in this case at least, it’s not the words that create the effect. It’s the experimenters’ expectations.
Back in 1996, John Bargh and his colleagues found that infusing people’s minds with the concept of age could slow their movements (PDF). The volunteers in the study had to create a sentence from scrambled words. When these included a word related to being old, the volunteers walked more slowly when they left the laboratory. They apparently didn’t notice anything untoward about the words, but their behaviour changed nonetheless. It was a classic result, and it turned the paper into one of the most widely cited studies in social psychology.
Two other groups have since replicated the effect, but neither stuck to the original set-up. That’s what Doyen wanted to do, but with three important tweaks. First, in Bargh’s study, a researcher timed the volunteers with a stopwatch. This time, Doyen would use infrared sensors for more accurate readings. Second, Bargh recruited 60 volunteers, but Doyen recruited twice as many. Third, Doyen also recruited four experimenters who carried out the study, but didn’t know what the point of it was.
This time, the priming words had no impact on the volunteers’ walking speed. They left the test room neither more slowly nor more quickly than when they arrived. The famous result hadn’t replicated. Why?
Doyen suspected that Bargh’s research team could have unwittingly told their volunteers how they were meant to behave, just as von Osten unconsciously told Clever Hans when to stamp. Perhaps they themselves moved more slowly if they expected the volunteer to do so. Maybe they spoke more languidly, or shook hands more leisurely.
We know that this sort of thing goes on all the time. It’s why people who run medical trials use “double-blind” designs, where neither experimenters nor patients know who is being given what. That wasn’t the case in Bargh’s study. The experimenter who clocked the stopwatch in the corridor didn’t know which volunteers had been primed, but the experimenter in the test room did. They could have unconsciously amplified the effect of their primes. Maybe they were responsible for creating the very behaviour they expected to see.
To test that idea, Doyen repeated his experiment with 50 fresh volunteers and 10 fresh experimenters. The experimenters always stuck to the same script, but they knew whether each volunteer had been primed or not. Doyen told half of them that people would walk more slowly thanks to the power of priming, but he told the other half to expect faster walks. Again, he measured the volunteers’ speed with infrared sensors, but he also gave the experiments a stopwatch to take some back-up readings.
When Doyen looked at the data from the infrared sensors, he found that the volunteers moved more slowly only when they were tested by experimenters who expected them to move slowly. If Doyen relied on the experimenters’ own stopwatch-based measurements, things were even worse. The ones who anticipated faster walks measured faster walks. The ones that presumed slower walks found those too. Let that sink in: the only way Doyen could repeat Bargh’s results was to deliberately tell the experimenters to expect those results.
It’s a fascinating result, but one that isn’t a deathblow for priming as a method. Note that Doyen isn’t suggesting that Bargh’s team were simply making up their results to fit what they expected. Rather, their expectations affected their behaviour, which then affected the volunteers’ behaviour. The volunteers were still being primed, albeit by the experimenters rather than the word tasks. “Either possibility is a confirmation for the power of priming,” says Tom Stafford from the University of Sheffield.
Bargh himself says, "The basic 'stereotype-priming of behavior' effect has been replicated dozens of times. There are many reasons for a study not to work, and as I had no control over [this] attempt, there's not much I can say."
Joshua Ackerman, a psychologist from MIT, says, “There have been hundreds, if not thousands, of priming studies conducted, many of which use designs that make experimenter bias essentially impossibl. It would be a huge mistake to draw the implication here that [these] studies refute this body of work in any way.”
Doyen’s study doesn’t show a radically new flaw, or one that’s unique to this branch of social psychology. We’ve known for over a century that scientists can very easily bias their own experiments, even in the most carefully controlled cases. “It’s a neat paper that re-emphasises some highly important and widely relevant warnings for everyone who might want to conduct experiments with people,” says Stafford. “Expectations – participants’ and experimenters’ - and inaccurate measurement can combine to give you biased results.” Ackerman adds, “It's a lesson that behavioural researchers are all trained in, but one that bears repeating from time to time.”
“Our results don’t completely rule out the possibility of unconscious priming,” says Doyen, “but they point to the fact that the (generally weak) effects may also be influenced by many other factors that are almost never controlled in such studies.”
The study also serves as a good reminder about how important it is for scientists to try and repeat each others’ results. “The need for independent replications of important results such as those of Bargh cannot be overstated,” says Doyen. “The literature relies far too much on findings that have been produced using different methods dating back to 30 or 40 years ago.”
Reference: Doyen, Klein, Pichon & Cleeremans. 2011. Behavioral Priming: It’s all in the Mind, but Whose Mind? PLoS ONE http://dx.doi.org/10.1371/journal.pone.0029081
Image by Alex Proimos
