It's not been a good month for the theory of ego-depletion - the idea that self-control is a limited resource that can be depleted by overuse. Two weeks ago, researchers reported evidence of bias in the published literature examining the question of whether glucose can reverse ego-depletion. Now, the very existence of the ego-depletion phenomenon has been questioned by an international collaboration of psychologists who conducted a preregistered replication attempt (RRR). The results have just been published in Perspectives on Psychological Science (although they've been circulating for a while.) 23 labs participated in the replication effort, with a total final sample size of 2,141 participants. The project was co-ordinated on the Open Science Framework (OSF) and the paper was written by Martin Hagger and Nikos Chatzisarantis. Methodologically, the replicators did not attempt to reproduce the original version of the ego-depletion task, saying that "the tasks used in the original experiments were deemed too elaborate or complex to be appropriate for a multilab replication." Instead they focussed on the procedures used by a recent paper, Sripada et al. (2014), which were fully computer-administered, and thus more portable. Importantly, Hagger and Chatzisarantis say that "The decision to use these tasks was based on the recommendation of Roy Baumeister"; Baumeister being the man who introduced the concept of ego-depletion back in 1998... but see below. When asked to predict the outcome of the experiment, only one lab predicted a null result. The other 22 predicted seeing at least a small effect, perhaps because there have been dozens of published studies reporting positive evidence of ego-depletion. Contrary to expectations, the replication gave a null result: there was no statistically significant ego-depletion effect, combined across all labs (Cohen's d = 0.04, 95% CI [−0.07, 0.15]). In two out of the 23 labs, a significant ego depletion effect was seen, but in a third lab, a significant reverse effect occurred. Restricting the analysis to English-speaking participants produced a slightly higher effect size, but it was still non-significant (d = 0.14, 95% CI [−0.02, 0.30]).
This looks pretty damning. Hagger and Chatzisarantis comment that these results are much worse than would be expected from the previous literature:
The effects are substantially smaller than the ego-depletion effect size... in Sripada et al.’s (2014) study (d = 0.69) that the present protocol was based on. The present effects are also much smaller than the uncorrected ego-depletion effect sizes reported in Hagger et al.’s (2010a) meta-analysis (d = 0.62)... The results are consistent with a null effect for ego depletion for the current paradigm.
But in a rebuttal piece, ego-depletion pioneers Baumeister and Vohs call the replication project a "Misguided Effort With Elusive Implications". They say that the task used, borrowed from Sripada et al. (2014), may just not be effective at eliciting ego-depletion. Baumeister admits that he himself recommended that task, but he says that he was never especially keen on it:
In retrospect, the decision to use new, mostly untested procedures for a large replication project was foolish. When planning the Registered Replication Report (RRR) on ego depletion, Hagger asked Baumeister for suggestions. Baumeister nominated several procedures that have been used in successful studies of ego depletion for years. But none of Baumeister’s suggestions were allowable due to the RRR restrictions that it must be done with only computerized tasks that were culturally and linguistically neutral. Discussions were stalemated, and we felt pressured to come up with something quickly. We learned of a new study by Sripada et al. (2014) that fit the requirements and passed this along to the RRR team. Since there were no viable other options, that method was chosen... Under the circumstances, we understood our approval to mean “Sure, go ahead” and not “Yes, that’s a definitive test of the phenomenon we’ve been studying all these years.”
Baumeister and Vohs go on to promise to conduct their own "preregistered, multisite replication project next year, using well-tested procedures (ones that actually involve self-regulation)". For now, they conclude that
For two decades, we have conducted studies of ego depletion carefully and honestly, following the field’s best practices, and we find the effect over and over (as have many others in fields as far ranging as finance, health, and sports, both in the lab and large-scale field studies). There is too much evidence to dismiss based on the RRR, which after all is ultimately a single study...
Finally, Sripada and colleagues reply that differences in the experimental procedures might explain why their effect did not replicate
Our study was conducted in an outpatient psychiatry clinic, not a university lab. Second, our participants were drawn from a community sample and were paid $20 for participation, whereas nearly all the RRR studies used undergraduate participant pools and did not pay participants. Third, our paradigm was implemented in the context of a double-blind drug manipulation with Ritalin and a placebo. Participants arrived roughly 90 minutes before the tasks, were presented with an extensive consent detailing effects of Ritalin, ingested an unlabeled capsule, and then waited an hour for the drug (or placebo) effect to take hold. Given the preceding differences, it is likely that our participants were more motivated and invested in study participation than were typical participants in the RRR studies.
In my view, the dispute over the nature of the task is an unfortunate one, and it does complicate the interpretation of these results. Baumeister and Vohs have a point that the Sripada et al. ego-depletion task has not been used as much as other, earlier versions of the paradigm. Then again, it's always possible to quibble about methodogical issues post hoc. If the results had come out positive, I doubt Baumeister and Vohs would be querying the task's validity. Either way, it's great news that Baumeister and Vohs plan to conduct their own RRR. Perhaps that study will provide the definitive test of the theory. Edit: Hagger and Chatzisarantis have published a rebuttal to Baumeister, Vohs, and Sripada here.
Hagger, M., & Chatzisarantis, N. (2016). A Multilab Preregistered Replication of the Ego-Depletion Effect Perspectives on Psychological Science, 11 (4), 546-573 DOI: 10.1177/1745691616652873
