What Good Science Looks Like

Yet another article appeared yesterday on Excite News on how Web Journals Threaten Peer-Review System. (For another view see Deepak Singh.) There were some good points made. For example, on tenure and evaluation:

Researchers whose work appear in traditional journals are often more highly regarded. That attitude appears to be slowly changing. In 2002, the reclusive Russian mathematician Grigori Perelman created a buzz when he bypassed the peer-review system and posted a landmark paper to the online repository, arXiv.
Perelman later won the Fields Medal this year for his contribution to the Poincare conjecture, one of mathematics' oldest and puzzling problems.

On quality control:

Editors of traditional, subscription-based journals say the peer-review system weeds out sloppy science. The traditional process isn't designed to detect fraud (referees rarely look at a researcher's raw data), and prestigious journals have unwittingly published bogus work. Last year, for example, Science retracted papers on embryonic stem cell research by a South Korean cloning scientist who admitted falsifying his results.

I think we need to highlight good Open Science when we find it. Org Prep Daily is a nice recent example. It is a blog with synthetic procedures. From the way it is written, any competent organic chemist can tell that the information is self-consistent and of high quality, with enough information to repeat the experiment and confirm that it worked. Those who can't tell have no business doing the chemistry. And those who want to learn need to discuss it with peers and mentors to try to understand how to tell if it makes sense, just like they would with any other type of publication.

Org Prep Daily is very similar in scope to Synthetic Pages. Both provide recipes for making chemicals at a similar level of detail. However, they differ in some important respects. Synthetic Pages is organized in a central database and users must abide by the website's terms. For the authors, the copyright is relatively friendly, at least compared with standard journals:

Can I publish after submitting a procedure to SyntheticPages?

Yes. Although you transfer copyright of the SyntheticPage to us, you may also incorporate the information into a paper (subject of course to the third party's terms and conditions) or elsewhere. For this, no copyright notice is required. You should, however, consider that submitting a SyntheticPages may affect your right to claim a patent on the material at a later date.

But for users, no re-mixing with attribution allowed:

You are not authorized to:

* alter the material in any way
* reproduce or store any part of this web site in any other public or private storage medium, electronic or otherwise, without written permission from SyntheticPages.

In contrast, using a blog like Org Prep Daily enables the authors to set their own format and copyright terms. Since they control the blog, in principle they could selectively remove negative comments, which would not be the case with Synthetic Pages. However, it is hard to unlink things in the Blogoshere and if someone really wanted to indicate that something was seriously wrong with a procedure they could do so on another easily found forum. One advantage of the Synthetic Pages approach is that it offers a third party time stamp, which is more difficult to do on a blog. Using a hosted wiki solves that problem.

This basically boils down to a top-down vs. bottom-up approach. Which is better? For me it comes down to findability. Unless you had heard of Synthetic Pages, you probably would not search on their database. However searching for a compound's name on Google pulled up the relevant posts on both Synthetic Pages and Org Prep Daily. If they are both equally findable, which one to use?

When in doubt duplicate. Post on both. That's why it is so important to retain the right to republish, which is almost always given away when publishing in a journal.

Reference to the compounds in redundant formats (i.e. InChI, SMILES, CAS number, etc) would also make the information much more findable. Read some Peter Murray Rust for more details on that.