Another reason why Data must be Open

Ben Goldacre (The Guardian columnist on “Bad Science”) has unearthed a superb interchange between a scientist and the creationists. Richard Lenski’s replies are tours de force and quite apart from rebutting the criticisms they could be read by undergraduate scientists as an example of excellent scientific practice. It’s worth emphasizing that good scientists live in the daily concern that their data must be validatable, reproducible.

The discussion is classic and a must-read. It’s long and according to your makeup will leave you laughing, weeping or raging. I append Ben’s summary below. But read the discussion

Inter alia RL was attacked by the creationists for failing to provide data to support his claims. RL replies that the data were in the paper but his attacker wilfully failed to understand this (and possibly failed to read the paper anyway). It is absolutely clear that RL provided everything that any responsble scientist would.

Here are some snippets:

Schlafly (Creationist): Submission guidelines for the Proceedings of the National Academy of Science state that “(viii) Materials and Data Availability. To allow others to replicate and build on work published in PNAS, authors must make materials, data, and associated protocols available to readers. Authors must disclose upon submission of the manuscript any restrictions on the availability of materials or information.” Also, your work was apparently funded by taxpayers, providing further reason for making the data publicly available.

Please post the data supporting your remarkable claims so that we can review it, and note where in the data you find justification for your conclusions.

RL:

Dear Mr. Schlafly:

I suggest you might want to read our paper itself, which is available for download at most university libraries and is also posted as publication #180 on my website. Here’s a brief summary that addresses your three points….

All these issues and the supporting methods and data are covered in our paper.

Schlafly: Dear Prof. Lenski, This is my second request for your data underlying your recent paper…

If the data are voluminous, then I particularly request access to the data that was made available to the peer reviewers of your paper, and to the data relating to the period during which the bacterial colony supposedly developed Cit+. As before, I’m requesting the organized data themselves, not the graphs and summaries set forth in the paper and referenced in your first reply to me. Note that several times your paper expressly states, “data not shown.”

RL: Finally, let me now turn to our data. As I said before, the relevant methods and data about the evolution of the citrate-using bacteria are in our paper. In three places in our paper, we did say “data not shown”, which is common in scientific papers owing to limitations in page length, especially for secondary or minor points. None of the places where we made such references concern the existence of the citrate-using bacteria; they concern only certain secondary properties of those bacteria. We will gladly post those additional data on my website.

PMR: RL has taken great pains (many pages) to refute the claims of fraud and to assert that the data were visible: BUT this was only possible because the fulltext pf the paper was available:

Imagine what would have happened if RL had replied:

“I am sorry, but the article was published in a closed access journal and I have no rights to make the text , which contains the data , publicly available. You will just have to believe that the reviewers and editors agreed with my arguments and that the data supported it. Or each of you and your acolytes will have to purchase the article at a cost of 30 USD from the publisher. And don’t post it on your website – even just the graphs containing the data – or the publishers will send legal letters to you”.

So, in this case, Open visibility was essential to RL’s successful defence. However the “data not shown” was a potential serious weakness, which was not imposed by the author but by the publication process. Electrons and magnetic disks are infinitely cheaper than “pages” and there is no reason whatever to have “data not shown” in a modern article.

Except of course if, as a publisher, you want to stop us re-using it for free…

===================================================================

All time classic creationist pwnage (PMR: follow this link)

June 24th, 2008 by Ben Goldacre in bad science |

Richard Lenski is a biologist who recently found evidence for the emergence of new traits among E.coli bacteria, in a fascinating experiment which he has described in a paper in PNAS (best lay coverage here). His results look a bit like evolution. You will note that his paper includes the original data.
Andrew Schlafly is a startlingly predictable right wing christian activist who runs Conservapedia. I highly recommend a look around there if you’ve not already had the pleasure, because even the people who run Conservapedia find it hard to tell whether the edits are being made by god-fearing americans or naughty satirists.
Schlafly read Lenski. He got angry. He demanded the original data. It was pointed out to him that the original data was in the paper. He demanded the original data again. With menaces.
The following exchange is mirrored humbly and verbatim in case of disappearance. It represents pwnage on a scale most of us can only dream of. (PMR: read under All time classic creationist pwnage)
This entry was posted in Uncategorized. Bookmark the permalink.

3 Responses to Another reason why Data must be Open

  1. Martin says:

    Hi, nice to see another Cambridge-based blogger!
    On the subject of Conservapedia, I just did an back-of-the-envelope “analysis” of their reach here, and to be honest I don’t think it’s just Lenski that owns them – by my calculation the criticism in the science blogosphere will get a far bigger audience than their efforts.
    See: Conservapedia’s Reach: A Back-of-the-Envelope Study

  2. Pingback: The Lay Scientist

  3. pm286 says:

    “Bill” had a comment which got into the spam by mistake supporting the idea that no data should be withheld in a paper.

Leave a Reply

Your email address will not be published. Required fields are marked *