The 'file drawer problem' refers to the fact that in science, many results remain unpublished - especially negative ones. This is a problem because it produces publication bias. Now, a group of Belgian psychology researchers have decided to make a stand. In a bold move against publication bias, they've thrown open their own file drawer. In the new paper, Anthony Lane and colleagues from the Université catholique de Louvain say that they've realized that over the years, "our publication portfolio has become less and less representative of our actual findings". Therefore, they "decided to get these [unpublished] studies out of our drawer and encourage other laboratories to do the same." Lane et al.'s research focus is oxytocin, the much-discussed "love hormone". Their lab has published a number of papers reporting that an intranasal spray of oxytocin alters human behaviour. But they now reveal that they also tried to publish numerous negative findings, yet these null results remain in the file drawer because they weren't accepted for publication.
Is there a file drawer problem in intranasal oxytocin research? If this is the case, it may also be the case in our laboratory. This paper aims to answer that question, document the extent of the problem, and discuss its implications for intranasal oxytocin research. We present eight studies (including 13 dependent variables overall, assessed through 25 different paradigms) that were performed in our lab from 2009 until 2014 on a total of 453 subjects... As we will demonstrate below, the results were too often not those expected. Only four studies (most often a part of them) of the eight were submitted for publication, yielding five articles (2, 8, 27, 34, 35). Of these five articles, only one (27) reports a null-finding. We submitted several studies yielding null-findings to different journals (from general interest in psychology to specialized in biological psychology and in psychoenodcrinology) but they were rejected time and time again.
Neuroskeptic readers may remember Lane et al.'s sole published negative study (27), as I blogged about it last year. The authors go on to present the results of all eight oxytocin studies. A meta-analysis of all of the studies finds that oxytocin has no detectable effect: "The aggregated effect size was not reliably different from zero, Cohen’s d = 0.003 (95% CI: -0.10 - 0.10)". They conclude, in an understated but powerful paragraph:
This large proportion of "unexpected" null-findings raises concerns about the validity of what we know about the influence of intranasal oxytocin on human behaviors and cognition... Our initial enthusiasm on intranasal oxytocin findings has slowly faded away over the years and the studies have turned us from 'believers' into 'skeptics'.
So what? In my view this is a very important paper, and a brave move by the authors. This kind of revelation of what goes on "behind closed drawers" could be an effective remedy for publication bias. I suspect though that prevention is better than cure, and that the best way to keep the file drawers from filling up in the first place will be toreform the scientific process itself. That said, a 'skeptic' might say that Lane et al. are doing too little, too late. After all, their papers reporting positive effects of oxytocin are still out there - and some of them have been highly cited. If Lane et al. no longer have confidence in those papers, should they retract them? I don't think so. If we started expecting scientists to retract papers whenever they changed their minds, I think it would have two effects: slightly more papers would be retracted, and scientists would change their minds a lot less. By publishing these results, Lane et al. have ensured that future meta-analysts will be able to include the full dataset in their calculations. In the long run, this will erase any damage caused by the publication bias. Hat tip: thanks to Bernard Carroll.
Anthony Lane, Olivier Luminet, Gideon Nave and Moïra Mikolajczak (2016). Is there a publication bias in behavioral intranasal oxytocin research on humans? Opening the file drawer of one lab Journal of Neuroendocrinology
