Last year, with much ballyhoo, NASA held a press conference about a team of biologists claiming that they had found microorganisms that could use arsenic instead of phosphorous as a basis for biological processes.

However, it didn’t take too long before the work was under serious attack by other biologists. Some were snarky, others more reserved, but the message was clear: not too many professional biologists felt the arsenic claim held up to scrutiny. In fact, some said the research paper was so shoddy it should never have been published.

This whole event comes to my mind from time to time, and I’ve been meaning to revisit it. I’ll admit I’m a little embarrassed by how I participated in it — I reported it straight, writing up a blog post relaying what I had learned from the press conference and from reading the paper itself. I am not a biologist, so the details of the paper were beyond me. But being a scientist myself I could glean what I needed for a blog post, especially coupled with the comments from the press conference.

Or so I thought. As the criticism came in, and I looked into it more, I found myself agreeing with those who disagreed with the original findings. I’ll note that I can’t say for sure if the research was done poorly or not, but it has become more clear that the work itself needed more outside commentary before being released in a press conference at the level it was. My own mistake was trusting NASA PR to have vetted this thoroughly before holding the press conference, and not getting an outside opinion myself. I wrote a followup a few days later with my doubts.

The reason I bring all this up now is that PopSci recently posted an article about the lead scientist in the arsenic story, Felisa Wolfe-Simon. I read it yesterday, and felt that it was coming from an odd angle — as a personality profile on Dr. Wolfe-Simon it’s interesting, but as a background piece on the arsenic story itself it read to me as if it were slanted to support her.

Both my Discover Magazine colleagues Ed Yong and Carl Zimmer have a similar opinion — Carl is actually pretty blunt in his opinion on the article, giving details (and a timeline!) on where it goes astray. Wired Magazine blogger David Dobbs also write an interesting piece where at first he says he likes the article, then updates his post later saying that after more thought a lot of doubts arose. Given the size and impact this story had last year, I suggest you read those links. They’ll give you something to think about.

There are many lessons to be learned here, but the one I’ve walked away with is that since this story came out, I’ve been more careful to check with sources if I’m not familiar with the science myself (and even if I am familiar with it). It’s impossible to prevent all mistakes in writing science articles — in writing anything — but a healthy dose of skepticism is required to minimize them.

In fact, I’d say a healthy dose of skepticism is always required, in every situation. It’s a highly useful tool.