Ranking chemistry and blogosphere metrics

I’ve been pointed to ChemRank – a system that allows you to comment on and rank the chemical literature. I hadn’t seen this before and haven’t looked in depth, so I am only commenting on the idea and technology. (As always I also would like to know what the Openness of the supporting organisation was). I copy the post in full and make some comments.

Chem 2.0: ChemRank – You decide the quality of the literature.

Dissociating the good literature from the bad literature is an endeavor we all do individually(but not for long: ChemRank). If only there was some website where we could tell whether that syn. prep. is accurate or that physical model is valid. The current methods to determine the validity of the literature are: to perform the same experiment, try to determine this from how many people cite that article, go ask around the department for someone who did something similar, try to relate the quality of the paper from the h-index of the author.

But, what if you still want more? What if you want something more than a numerical qualifier of a paper’s worthiness, or of an author’s scientific quality? What if you want to have a discussion beyond the scope of your labmates? What if you want to have an intellectual discussion about the current chemical literature with every other chemist on the planet? How do you do this and where do you begin?
The chemical blogosphere has done a rather good job in keeping the community of internet savvy chemists a breasted on some of the latest and coolest research (http://wiki.cubic.uni-koeln.de/cb/index.php). But, unless you have a huge audience and build a faithful readership, no one will ever know your views and it can be rather difficult to have an intellectual conversation with just yourself.
With all this in mind, I set out to create a website that would overcome these historic limitations to academic communication. I created a new website called ChemRank: http://www.chemrank.com Screen Shot shown [deleted in this post – PMR]
At ChemRank you can add a paper to the database and then vote whether you find it a good or bad paper. You can also leave comments critiquing the yield or congratulating the authors on a job well done. Most importantly, there is a public record of your views! Your pain does not need to be repeated if you point out the problematic reaction or the incorrect eqn in the comments section.
An other cool feature of ChemRank is the building of a database that knows the good chemical literature from the bad chemical literature. Currently, there is no database to my knowledge, except you own brain or PI’s brain, that tracks this information. Although, Noel’s experiments with Connotea is the step in the right direction.
There is even something for the already established pioneers in online chemical literature discussions (aka chemical bloggers) too. For each paper added to the database a script, behind the scenes, will generate a little code snippet to allow voting on the literature your discussing on your blog too. For example, if you are discussing the recent paper “Atomic Structure of Graphene on SiO2” you can generate the following from copy/pasting a code snippet. Assuming you’ve already added the paper to the database and clicked the link: “add to your blog”

Atomic Structure of Graphene on SiO2

code snippet:

Code: (html)

Atomic Structure of Graphene on SiO2

This by no means is a completed project, there is still much more to do.

• Make links that show the papers posted last 7days, 1month, 1 year, all time
• Make users register before posting.
• Add the ability to tag the articles with relevant keywords
• Make a pipe’s rss feed of the author’s most recent papers, on the fly and behind the scenes, display in the description as well.
• Make an api so others can access the database and use it for cool-new mashups.
• Add a search feature, but that is kind of pointless while the database has less than 10 papers in it.

All of the above can be done in time, but it depends on the feedback from the community and the popularity of the website. As with all projects, I don’t necessarily expect it to catch on right away if at all. A lot of times I code, simply to show how to do it, and how to do it well. Allowing public comments on the current literature is something the chemical publishers should be doing anyways, now we no longer have to wait for them…
Mitch

Comment:
This and similar approaches are very important. We don’t know where metrics like this will take us but it can’t be worse than the current citation index which is managed by … by whom???
Before anyone puts too much effort into their own system the should check out carefully what the “Web 2.0” community does for review. As I blogged earlier, Revyu.com – Review Anything has tools for managing and collecting reviews – no technical reason why they shouldn’t actually include chemistry AFAIK. I would strongly encourage using RDF for reviews – this will happen anyway and makes your material discoverable more easily.
There are reviews in the chemical blogosphere already – TotallySynthetic.com runs a poll for the best synthesis of the month/year, etc. I’m not sure whether there is any technology that publishes this in RDF form but it would be relatively easy. Commentaries in the blogosphere also have a greasemonkey viewer from Chemical blogspace
and other sites, so that when you read the table of contents the monkey alerts you to any commentary. It would be technically easy to fix the monkey so it counted eyeballs on papers – how often was this paper READ (not cited, not downloaded – but at least displayed in the browser presumably with a human somewhere near). And to add tools so that the monkey could ask for opinions from the viewer.
I’m not suggesting there should be one coordinated effort – anymore than we need just one movie review database. But I do think that reviewers should try to use the same technology so that their results can be aggregated – and the reviewers can themselves be reviewed!