.comment-link {margin-left:.6em;}

Drexel CoAS E-Learning Subscribe with Bloglines Drexel CoAS E-Learning Podcast

Monday, February 13, 2006

Peer Review in the Google Age

Peer review in science has been the subject of significant controversy. Peggy Dominy has links to some recent articles discussing the matter. Dave Bradley points out that some journals are moving towards removing the anonymity of reviewers.

In a world where finding information is laborious, the knowledge that a document is from a "“trusted source" is valuable. I remember spending tremendous amounts of time as a graduate student in the early 90's using the Chemical Abstract books looking for articles about my research. An article from the Journal of Organic Chemistry was likely to have a reliable experimental section and it was worth my time to physically locate that article and photocopy it. Other sources might not have warranted the effort.

But the situation is different today. The only reason I have for visiting the library now is to see my librarian friends. The fact that I can download articles from my office or my home has made things easier. But just as importantly, my ability to quickly filter through databases using flexible queries saves a tremendous amount time. The source of the information is really not as important as my ability to find it online immediately.

The idea that peer review is useful to "authenticate" research has always seemed a bit strange to me. After all, the targeted audience for most scientific articles consists of (by definition) peers of the author. What makes the reviewers selected by an editor any more capable of validating an article than the targeted audience?

Of course reviewers do not repeat the experiments in an article and cannot check for fraud. This can only be determined over time, after other researchers have had a chance to try to use the reported techniques. Once in a while a big scientific fraud case makes the news and that would happen with or without peer review.

Sometimes it is clear that reviewers do not even read the articles, as was made evident from the green light given to a computer generated document of random jargon.

From an editor's perspective, peer review is a cost-effective way to maintain the quality and focus of a journal. And certainly for the well-known journals in a given field, it is essential. However in an age when anyone can start an online "journal" and select reviewers that remain anonymous, the term "peer review" is not a very good indicator of quality. Unfortunately it is still used a kind of gold standard in academic promotion and tenure when counting publications.

I think the real issue is that we have to separate the problem of efficiently communicating scientific information from the problem of convincing a committee of the impact of a faculty member's scholarship. And when doing open science, the first concern is the communication of the information. Heather Morrison, Peter Suber and others are also commenting on this. That's why we're blogging our entire undigested laboratory notebook as close as we can to real time.

When I was looking a synthesis of a key compound for our malaria project, I was not interested in peer review. Others who are thinking about repeating our experiments will probably not care either.

This does not prevent the subsequent write-up of a peer reviewed article summarizing the experiments, just like talking about experiments in a conference usually does not prevent publication in most journals. There is plenty of room for both types of communication.

First disclose, then discuss and finally convince when necessary.


  • Dear Jean-Claude,
    I think you have a very good point here. Blogging today helps science along at a much faster pace.

    To me this seems to be going back to some very old ideas. When the Royal Society made their archive freely accessable through the web a couple weeks ago (unfortunately only for 2 months - it was fascinating to read). The very first issues of the Philosphical Transactions journal with articles by Newton, etc. were made available. What did intrigue me was that the journal at that time (1665ff) was only partly made up by original articles. A large portion consisted of letters, reviews, discussions of the discoveries of collegues. This was about work in progress, offering ideas and commenting on them etc. It was like reading a 340 years old blog - just in paper form.

    So is today's blogging just a return to some good scientific ideas? It might be only a little faster and reaching a larger (global) audience.

    By Blogger Ulrich, at 2:57 PM  

  • Ulrich,
    Absolutely, there is definitely a sense of going back to the roots of science. But this time the process can be sped up because machines will be contributing more and more.

    By Blogger Jean-Claude Bradley, at 3:05 PM  

Post a Comment

<< Home

Locations of visitors to this page Creative Commons Attribution Share-Alike 2.5 License