"If it disagrees with experiment, it's wrong. That's all there is to it."



Hat tip: jashapiro (@jashapiro)

If GFAJ-1 doesn't ring a bell, maybe the #arseniclife hashtag will. And no, GFAJ-1 hasn't arrived here. It's arrived at the University of British Columbia, Rosie Redfield's lab in particular.

Maybe now things are starting to click. Let's see. Rosie Redfield, she was among the first to criticize the paper by Felisa Wolfe-Simon and colleagues¹ claiming discovery of a bacterium that can substitute arsenic for phosphorous in biomolecules, including DNA. GFAJ-1 is the strain of bacteria claimed to have this ability. Now Rosie has cells of GFAJ-1 growing in her lab and can run her own experiments.

Leave aside whether or not Felisa Wolfe-Simon and her co-authors are shown to be wrong, as seems likely given my reading of the critiques,² the #arseniclife episode illustrates something important about how science works, something that intelligent design proponents ignore. I pointed out last December that

Wolfe-Simon and her colleagues will give other scientists samples of the bacterium that is claimed to have these extraordinary properties and those scientists will have a chance to verify Wolfe-Simon's claims. If Wolfe-Simon's claims turn out to be right, textbooks will be rewritten, and all of us will have to think differently about the chemistry of life. If her claims turn out to be wrong, only a few specialists will remember that the paper was ever published.

Now those samples are in the hands of Wolfe-Simon's critics. Experiments, data, and analysis will now vindicate either Wolfe-Simon or her critics

Compare that lengthy, tortuous process of debate and review among experts to the "debate" that intelligent design creationists want to have in high school textbooks. Which process do you think is most likely to help kids understand their world?

Related articles

• Starting to work with GFAJ-1! (rrresearch.blogspot.com)
• From the shadows to the spotlight to the dustbin - the rise and fall of GFAJ-1 (scientificamerican.com)
• Critics weigh in on arsenic life (nature.com)

Last December, Science published a paper by Felisa Wolfe-Simon and colleagues claiming to have found A bacterium that can grow by using arsenic instead of phosphorous -- that was the title of their paper. As you probably know, a firestorm erupted when Rosie Redfield posted a detailed critique on her blog concluding that

Basically, [the paper] doesn't present ANY convincing evidence that arsenic has been incorporated into DNA (or any other biological molecule)

There was such an uproar that the paper, which was published online in Science Express, has yet to appear in print. Links to a lot of information about the paper and critiques of it appeared on Twitter with the hashtag #arseniclife.

On Friday, eight technical comments on the original Science Express article and a response to those comments were published -- again in Science Express. The final version of the Wolfe-Simon paper will appear, along with the technical comments, in the 3 June issue of Science. But as Carl Zimmer says in Slate,

For those of us who have been tracking #arseniclife since last Thanksgiving, however, today comes as an anticlimax. There's not much in the letters to Science that we haven't read before. In the past, scientists might have kept their thoughts to themselves, waiting for journals to decide when and how they could debate the merits of a study. But this time, they started talking right away, airing their criticisms on the Internet. In fact, the true significance of the aliens-that-weren't will be how it helped change the way scientists do science. (emphasis added)

James Holland Jones applies lessons from the arsenic-based life controversy to anthropology. His conclusions?

1. [W]e need to have scientific theories that are sufficiently robust that they can generate testable predictions that transcend the particularities of time and place. Results generated in one population/place/time can then be challenged by testing in other populations/places/times.
2. [W]e need to be scrupulous in our documentation of our results and the methods we employ to generate these results.
3. [W]e need to be willing to share our data.

If those sound like reasonable, unobjectionable conclusions, they are. You may be surprised to learn that there is controversy within anthropology about whether these precepts should be followed, because there's debate about whether anthropology should be a science.¹

Related articles

• Anthropology: Engaged Social Science in a Changing World (blogs.berkeley.edu)
• Anthropology Group Tries to Soothe Tempers After Dropping the Word 'Science' (nytimes.com)
• anthropology and the problem of science (orgtheory.wordpress.com)
• Anthropology and Science (nodivide.wordpress.com)
• Anthropology: More a Hobby Than a 'Science' [Science] (gawker.com)
• Anthropology, Science, and the AAA Long-Range Plan: What Really Happened (3quarksdaily.com)
• "Is Anthropology Science?" and related posts (telicthoughts.com)

Shortly after the report of arsenic-based life appeared, I pointed out that several scientists were skeptical of the claim. I used the debate as an example of how science works and compared it to the "debate" about intelligent design:

Compare that lengthy, tortuous process of debate and review among experts to the "debate" that intelligent design creationists want to have in high school textbooks. Which process do you think is most likely to help kids understand their world?

Much the same could be said about climate science. In fact, the folks at RealClimate have a post showing how the debate about arsenic-based life demonstrates three things:

1. Major funding agencies willingly back studies challenging scientific consensus.
2. Most everyone would be thrilled to overturn the consensus.
3. Scientists offer opinions based on their scientific knowledge and a critical interpretation of data.

Here's the money paragraph:

This is the key lesson to take from this incident, and it applies to all scientific disciplines: peer-review continues after publication. Challenges to consensus are seriously entertained - and are accepted when supported by rigorous data. Poorly substantiated studies may inspire further study, but will be scientifically criticized without concern for funding opportunities. Scientists are not "afraid to lose their grant money".

But you should go read the whole thing.

Related articles

• 'Weird life' reveals science at work (cosmiclog.msnbc.msn.com)

When Felisa Wolfe-Simon and her colleagues announced that they had found a bacterium that could replace phosphorous with arsenic, they published a paper in Science. But NASA also scheduled a press conference to announce the findings:

As you know if you've seen my previous posts on arsenic-based life, that claim has been met with a lot of skepticism. The skeptics have posted detailed critiques on blogs in addition to submitting letters to Science questioning the results.

But Wolfe-Simon and her colleagues refuse to respond to their critics. They want the debate to play out in the pages of peer-reviewed journals. Funny. They were more than willing to share their claims with a broad public audience, but they aren't wiling to defend their claims against critics in the same venue. They want the critiques to be peer reviewed. Here's part of what the editors of Nature say in response.

Formal peer review does give criticized authors time to think critically and carefully, and it is a good way to filter out rubbish. But in this case, much of the criticism was already coming from the researchers' peers. And it should be remembered that peer review as conducted by journals is itself full of differing opinions, and is not the only way to crystallize truth from such disputes. In this instance, a prompt and explicitly provisional response from the authors would have been a better approach, particularly given the way they encouraged the original attention.

...

In the end, the scientific truth will prevail, as it usually does. In the meantime, researchers must accept some harsh truths about the speed and spread of digital criticism.

Blogging is no substitute for peer review. But peer review doesn't stop when a paper is published. If they're any good, papers are argued about in graduate seminars, in hallways, and at conferences. None of those discussions are peer reviewed, though they may lead to experiments or observations that are peer reviewed. By holding a press conference to announce a new finding, scientists invite a broader audience to hear what they have to say. They also invite a broader audience to hear what their critics have to say.

Related articles

• More on Arsenic Bugs - Nature responds to the blogosphere (scienceblogs.com)
• Arsenic bacteria - a post-mortem, a review, and some navel-gazing | Not Exactly Rocket Science (blogs.discovermagazine.com)
• Author of Controversial Arsenic Paper Speaks (news.sciencemag.org)
• DNA, Phosphorus, and Arsenic (sandwalk.blogspot.com)
• Post-publication peer review in public: poison or progress? (cenblog.org)
• Poisoned Debate Encircles a Microbe Study's Result (nytimes.com)
• Arsenic about face (guardian.co.uk)

If you read this blog, you probably read about the report in Science describing a claim that scientists had isolated a bacterium from Mono Lake in California that

substitutes arsenic for phosphorus to sustain its growth. Our data show evidence for arsenate in macromolecules that normally contain phosphate, most notably nucleic acids and proteins. (from Wolfe-Simon et al.; reference below; click through at the bottom)

In an accompanying news article, Elizabeth Pennisi quotes two scientists commenting on the report:

"This is a very impressive and exciting discovery," says Barry Rosen, a biochemist at Florida International University in Miami. "The implication of this work is that life can be quite different from what we know," agrees [Clara] Chan, [a geomicrobiologist at the University of Delaware.

To understand which the discovery would be so impressive, if it turns out to be right, just remember that the backbone of DNA is composed of bonds between phosphorous atoms. The claim that Wolfe-Simon and her collaborators make is that phosphorous can be replaced with arsenic in DNA. In fact, if that claim turns out to be right, it won't be merely "impressive" or "exciting". It will be truly astounding.

So why did I qualify what I wrote? Why did I write "if it turns out to be right" instead of simply writing "to understand why the discovery is so impressive..."? Click through for more of the story.

If you've done any computer programming, there's a good chance that you've heard of Edsger Dijkstra. In 1968, he published one of the most famous papers in computer science, "GOTO statement considered harmful."¹ I was also reminded earlier today that he wrote short memos almost every day. The memos have been put on line at the University of Texas, where he was on the faculty until 2000. One of those memos has three golden rules for scientific research that we'd all do well to remember.

• Raise your quality standards as high as you can live with, avoid wasting your time on routine problems, and always try to work as closely as possible at the boundary of your abilities. Do this, because it is the only way of discovering how that boundary should be moved forward.
• We all like our work to be socially relevant and scientifically sound. If we can find a topic satisfying both desires, we are lucky; if the two targets are in conflict with each other, let the requirement of scientific soundness prevail.
• Never tackle a problem of which you can be pretty sure that (now or in the near future) it will be tackled by others who are, in relation to that problem, at least as competent and well-equipped as you.

The first and the third of these rules are particularly sound advice. But at the risk of seeming presumptuous, I'd like to suggest a modification to the second.

• We all like our work to be socially relevant and scientifically sound. Never sacrifice the requirement of scientific soundness for social relevance and interpret "social relevance" broadly (not just in terms of immediate utility), but always seek problems that are both socially relevant and scientifically sound.
  

Related articles by Zemanta

• The Three Golden Rules for Successful Scientific Research (stat.columbia.edu)
• An Interview With Edsger W. Dijkstra (cacm.acm.org)

I learned from this week's Nature that the attorney general of Virginia has taken great interest in climate science. On 23 April he

filed what amounts to a subpoena ordering the University of Virginia to hand over, by 26 July, all available documents, computer code and data relating to [Michael] Mann's research on ... five [federal and state] grants. He also demanded all correspondence, including e-mails -- from 1999 to the present -- between Mann, now at Pennsylvania State University in University Park, and dozens of climate scientists worldwide, as well as some climate sceptics.

His motivation? He claims that Mann may have violated the 2002 Virginia Fraud Against Taxpayers Act, an act used to prosecute those who defraud the Commonwealth of Virginia by "[k]nowingly mak[ing], us[ing], or caus[ing] to be made or used, a false record or statement to get a false or fraudulent claim paid or approved by the Commonwealth" (source).

If the name Michael Mann rings a bell, it's because he's a very prominent climate scientist. He hasn't been shy about sharing his data, and his data show that global average temperatures had been relatively constant until a sharp spike upward in the 20th century, coincident with a massive expansion in the burning of fossil fuels.¹ Those who deny that climate change is happening really don't like his data, and the attorney general of Virginia appears to be among them. Here's how Nature's editors describe the attorney general's actions:

Cuccinelli's actions against Mann hark back to an era when tobacco companies smeared researchers as part of a sophisticated public relations strategy to raise doubts over the science showing that tobacco caused cancer, and delayed the introduction of smoking curbs for decades. Researchers found themselves bogged down in responding to subpoenas and legal challenges, which deterred others from the field. Climate-change deniers have adopted similar strategies with alacrity and, unfortunately, considerable success.

Certainly data on global climate should be made widely accessible, and much of it already is. I pointed out last November that the folks at RealClimate.org have compiled a list of publicly accessible data sets that everyone, including the attorney general of Virginia, is welcome to examine to their hearts content.

Mann's work has been examined and vindicated by the National Academy of Sciences and by Pennsylvania State University, where he's now on the faculty. Apparently that's not good enough for the attorney general of Virginia.

Mann is part of an overwhelming scientific consensus (a) that climate change is real and (b) human activities are contributing significantly to it. It's conceivable that the consensus is wrong, conceivable but not very likely.² But even if the consensus is wrong that wouldn't make any of the scientists, including Mann, who were part of the consensus guilty of fraud. Science progresses by showing how previously accepted ideas are wrong. Being wrong doesn't make you guilty of fraud, cooking your data does. And there's no evidence that Mann cooked any of his data. As the editors of Nature conclude,

Scientific organizations must respond quickly and forcefully any time political machinations threaten to undercut academic freedom. And, rather than complying, the University of Virginia should explore every avenue to challenge the subpoena.

The integrity of climate research has taken a very public battering in recent months. Scientists must now emphasize the science, while acknowledging that they are in a street fight.

That's how the editors of Nature start their lead editorial in this week's issue.¹ Later in the same issue, Jeff Tolleson reports on the disclosure of another batch of e-mails among members of the National Academy of Sciences discussing how climate scientists should respond to a report released by Senator James Inhofe claiming that climate scientists obstructed the release of data, manipulated data to fit preconceived conclusions, and threatened journal editors who published dissenting views. Paul Ehrlich is quoted as saying that the climate change skeptics "are trying to keep the scientists busy and to keep the scientists from doing their job, and they are doing extremely well".

I suggest that we pause for a moment and catch our breath.