As you know if you've seen my previous posts on arsenic-based life, that claim has been met with a lot of skepticism. The skeptics have posted detailed critiques on blogs in addition to submitting letters to Science questioning the results.
But Wolfe-Simon and her colleagues refuse to respond to their critics. They want the debate to play out in the pages of peer-reviewed journals. Funny. They were more than willing to share their claims with a broad public audience, but they aren't wiling to defend their claims against critics in the same venue. They want the critiques to be peer reviewed. Here's part of what the editors of Nature say in response.
Formal peer review does give criticized authors time to think critically and carefully, and it is a good way to filter out rubbish. But in this case, much of the criticism was already coming from the researchers' peers. And it should be remembered that peer review as conducted by journals is itself full of differing opinions, and is not the only way to crystallize truth from such disputes. In this instance, a prompt and explicitly provisional response from the authors would have been a better approach, particularly given the way they encouraged the original attention.Blogging is no substitute for peer review. But peer review doesn't stop when a paper is published. If they're any good, papers are argued about in graduate seminars, in hallways, and at conferences. None of those discussions are peer reviewed, though they may lead to experiments or observations that are peer reviewed. By holding a press conference to announce a new finding, scientists invite a broader audience to hear what they have to say. They also invite a broader audience to hear what their critics have to say.
In the end, the scientific truth will prevail, as it usually does. In the meantime, researchers must accept some harsh truths about the speed and spread of digital criticism.
It's worth noting that when Rosie Redfield posted her original blog entry on 4 December, she "wrote [it] mainly to clarify my own thinking. [She] didn't expect anyone other than a few researchers to ever read it." Rosie wasn't trying to take the debate public, but her detailed critique was available and quickly found by those, like Carl Zimmer, who write for a broad audience and were looking for a careful evaluation of the original paper.