It was the big news that wasn’t. Hyperbolic claims about the possible discovery of alien life, or a second branch of life on Earth, turned out to be nothing more than bacteria that can thrive on arsenic, using it in place of phosphorus in their DNA and other molecules. But after the initial layers of hype were peeled away, even this extraordinary claim started falling under suspicious glances. I’ve already talked about the untrammelled hype that surrounded this paper and I won’t retread that ground again. This is a chronological roundup of the criticism against the science in the paper itself, ending with some personal reflections on my own handling of the story (skip to Friday, December 10th for that bit).
Thursday, December 2^nd:
Felisa Wolfe-Simon published a paper in Science, claiming to have found bacteria in California’s Mono Lake that can grow using arsenic instead of phosphorus. Given that phosphorus is meant to be one of six irreplaceable elements, this would have been a big deal, not least because the bacteria apparently used arsenic to build the backbones of their DNA molecules. In my post, I mentioned some caveats. Wolfe-Simon isolated the arsenic-loving strain, known as GFAJ-1, by growing Mono Lake bacteria in ever-increasing concentrations of arsenic while diluting out the phosphorus. It is possible that the bacteria’s arsenic molecules were an adaptation to the harsh environments within the experiment, rather than Mono Lake itself. More importantly, there were still detectable levels of phosphorus left in the cells at the end of the experiment, although Wolfe-Simon claimed that the bacteria shouldn’t have been able to grow on such small amounts.
Friday, December 3^rd:
Variouscommenters voiced their concerns, and I attached an update to my post with a comment from John Sutherland, a skeptical biochemist in Cambridge. He argued that the arsenic-based DNA wouldn’t have been stable enough in water, and said that the authors need to actually create some arsenic-based DNA to study in detail.
Saturday, December 4^th:
The backlash began in earnest with several scientists criticising the conclusion that the bacteria have actually substituted arsenic for phosphorus in their DNA. Microbiologist Rosie Redfield posted a scathing critique on her blog, pointing out several flaws in heavy technical detail and dismissing the paper as “lots of flim-flam, but very little reliable information”. Among several objections, she noted that the bacteria grown in arsenic were still surrounded by more than enough phosphorus for their needs. The DNA was never purified and instead analysed while embedded within a gel; as a result, the tests could have picked up arsenic stuck around the DNA or loitering within the gel. Alex Bradley, a geochemist and microbiologist, posted his own lengthy analysis, noting that the bacteria’s DNA was immersed in water, which should have quickly broken any fragile arsenic-based compounds. It’s more likely that the DNA had a typical spine of phosphorus. Bradley also says that bacteria in the Sargasso Sea can indeed grow on the very low concentrations of phosphorus found in the GFAJ-1 cultures, contrary to the claims of Wolfe-Simon’s team. The technique that could have resolved the debate - mass spectrometry, which analyses the elements that make up a molecule – wasn’t done. “Unfortunately,” Bradley said, “these exciting claims are very, very shaky.”
Sunday, December 5^th:
The backlash builds. Several posts make mention of Redfield’s and Bradley’s critiques, and David Dobbs carefully considered whether science journalists could have done a better job with the story. “We thought we were getting cupcakes. Some of us wanted cupcakes. Who doesn’t want cupcakes? Now, everybody’s got humble pie in front of them, quite a bit to eat yet, and no dessert on the menu.”
Monday, December 6^th:
The mainstream media started to pick up on the backlash. CBC News (I think) were first out of the stables, with strong quotes from Redfield: "I blog openly…to bring this stuff more into the open where everybody can see it.” Meanwhile, the first signs emerged that NASA weren’t going to engage with the criticisms. Dwayne Brown, their senior public affairs officer, highlighted the fact that the paper was published in one of the “most prestigious scientific journals” and deemed it inappropriate to debate the science using the same media and bloggers who they relied on for press coverage of the science. Wolfe-Simon herself tweeted that “discussion about scientific details MUST be within a scientific venue so that we can come back to the public with a unified understanding.”
Tuesday, December 7^th:
It got worse. Carl Zimmer, writing in Slate, called up a dozen scientists, the vast majority of whom said that Wolfe-Simon hasn’t made her case. One of them frankly stated, "This paper should not have been published.” NASA’s intransigence continued. In a frankly astonishing quote to Ivan Oransky, Brown asserts, “NASA DID NOT HYPE anything – others did. Credible media organizations have not questioned NASA about any text. Bloggers and social media have.” Wolfe-Simon herself tells Zimmer that media debates "do not represent the proper way to engage in a scientific discourse and we will not respond in this manner.” It didn’t go down well. Jonathan Eisen says that “they carried out science by press release and press conference” and “are now hypocritical if they say that the only response should be in the scientific literature." David Dobbs calls the attitude “a return to pre-Enlightenment thinking”, and rightly noted that “Rosie Redfield is a peer, and her blog is peer review”. Chris Rowan agreed, saying that what happens after publication is what he considers to be “real peer review”. Rowan said, “The pre-publication stuff is just a quality filter, a check that the paper is not obviously wrong – and an imperfect filter at that. The real test is what happens in the months and years after publication.”Grant Jacobs and others post similar thoughts, while Nature and the Columbia Journalism Review both cover the fracas. Nonetheless, later in the day, NASA arranged for a quirky lecture about the findings. After some bizarre goofing-off, Oremland addressed a few of the criticisms. He said that lack of money prevented them from doing mass spectrometry experiments. And contrary to Bradley’s point about Sargasso Sea bacteria, he reiterated the claims in the paper that GFAJ-1 couldn’t have grown on the “smidgen” of phosphorus it contained. He finished by encouraging other groups to try and replicate their experiments, even offering to send them the bacteria. Meanwhile, Eisen urged critics to “make this an open discussion of science and science reporting and not a venue to spout derogatory comments about the people involved.”
Wednesday, December 8^th:
Wolfe-Simon posted a public statement on her website. While she welcomed the “lively debate”, she didn’t address any criticisms. Promising an FAQ about the paper later, she invited others to “read the paper and submit any responses to Science for review so that we can officially respond”. She’ll get her wish - Redfield has already formalised her critique in a letter to Science. Wolfe-Simon’s viewpoint garnered some support - Jack Gilbert at the University of Chicago said that impatient though he is, peer-reviewed journals are the proper forum for criticism. Others were not so kind. At the Guardian, Martin Robbins says that “at almost every stage of this story the actors involved were collapsing under the weight of their own slavish obedience to a fundamentally broken... well... 'system'” And Ivan Oransky noted that NASA failed to follow its own code of conduct when announcing the study. Carl Zimmer, ever the journalistic innovator, posted the full text of all his Slate interviews on his blog. The responses are illuminating. Many of the points had been made but notably, Hazel Barton (another microbiologist) was one of the few figures to publicly criticise Rosie Redfield’s review. She said, “It’s important that the reviews of individuals who are not experts in the field do not have as much weight as that of the original reviewers, no matter how public the review. I felt that her review had an equal number of flaws in it.” She praised Alex Bradley’s critique instead, as did his former boss Roger Summons. The list of mainstream outlets to cover the controversy grew to include Wired, New Scientist, the Daily Mail, the Atlantic, and many more. Much of it went over well-trodden ground, although Carmen Drahl added something new to the mix with more detail about the chemical analyses used in the study. Meanwhile, Science made the article freely available to the public for the following two weeks.
Thursday, December 9^th:
It seemed that most of the criticisms had been voiced but some scientists spoke out in support of Wolfe-Simon’s reaction. Dr Isis said, “If question remains about the voracity of these authors findings, then the only thing that is going to answer that doubt is data. Data cannot be generated by blog discussion... Talking about digging a ditch never got it dug.” Doctor Zen wondered if people only want Wolfe-Simon to engage with her critics to see a public smackdown: “I almost get the sense that they think that debating science bloggers would be like wrestling with a pig: they’ll get dirty and the pig would enjoy it.”
Friday, December 10^th:
And that brings us up to now. Looking back, it is astonishing how quickly these events unfolded and the sheer number of bloggers and media outlets that became involved in the criticism. This is indeed a brave new world, and one in which we are all the infamous Third Reviewer. Others have discussed these issues in greater depth and the links above should provide a good cross-section of opinion. For my part, I wanted to think about my own handling of the story, especially because I’ve been criticisedonTwitter for dropping the ball on it. I don’t actually disagree. This is what I tweeted on Sunday:
To clarify that, I tried to quell the hype around the study as best I could. I had the paper and I think that what I wrote was a fair representation of it. But, of course, that’s not necessarily enough. I’ve argued before that journalists should not be merely messengers – we should make the best possible efforts to cut through what’s being said in an attempt to uncover what’s actually true. Arguably, that didn’t happen although to clarify, I am not saying that the paper is rubbish or untrue. Despite the criticisms, I want to see the authors respond in a thorough way or to see another lab attempt replicate the experiments before jumping to conclusions. However, the sheer amount of negative comment indicates that I could have been more critical of the paper in my piece. Others have been supportive in suggesting that this was more egg on the face of the peer reviewers and indeed, several practicing scientists took the findings on face value, speculating about everything from the implications for chemotherapy to whether the bacteria have special viruses. The counter-argument, which I have no good retort to, is that peer review is no guarantee of quality, and that writers should be able to see through the fog of whatever topic they write about. There is no easy answer to this. On Twitter, my response was that we should expect people to make reasonable efforts to uncover truth and be skeptical, while appreciating that people can and will make mistakes. So for me, it comes down to this: did I do enough? I was certainly cautious. I said that “there is room for doubt” and I brought up the fact that the arsenic-loving bacteria still contain measurable levels of phosphorus. But I didn’t run the paper past other sources for comment, which I typically do it for stories that contain extraordinary claims. There was certainly plenty of time to do so here and while there were various reasons that I didn’t, the bottom line is that I could have done more. That doesn’t always help, of course, but it was an important missed step. A lesson for next time. To conclude, my aim in this is not to flagellate myself in a dervish of self-indulgence. However, I do believe that it you’re going to try to hold your profession to a higher standard, you have to be honest and open when you’ve made mistakes yourself. I also think that if you cover a story that turns out to be a bit dodgy, you have a certain responsibility in covering the follow-up. Hence, this post. Image by Michael Gabler
If the citation link isn't working, read why here
tweetmeme_source = 'DiscoverMag'; tweetmeme_service = 'bit.ly';
