Ranking chemistry and blogosphere metrics

I’ve been pointed to ChemRank – a system that allows you to comment on and rank the chemical literature. I hadn’t seen this before and haven’t looked in depth, so I am only commenting on the idea and technology. (As always I also would like to know what the Openness of the supporting organisation was). I copy the post in full and make some comments.

Dissociating the good literature from the bad literature is an endeavor we all do individually(but not for long: ChemRank). If only there was some website where we could tell whether that syn. prep. is accurate or that physical model is valid. The current methods to determine the validity of the literature are: to perform the same experiment, try to determine this from how many people cite that article, go ask around the department for someone who did something similar, try to relate the quality of the paper from the h-index of the author.

But, what if you still want more? What if you want something more than a numerical qualifier of a paper’s worthiness, or of an author’s scientific quality? What if you want to have a discussion beyond the scope of your labmates? What if you want to have an intellectual discussion about the current chemical literature with every other chemist on the planet? How do you do this and where do you begin?
The chemical blogosphere has done a rather good job in keeping the community of internet savvy chemists a breasted on some of the latest and coolest research (http://wiki.cubic.uni-koeln.de/cb/index.php). But, unless you have a huge audience and build a faithful readership, no one will ever know your views and it can be rather difficult to have an intellectual conversation with just yourself.
With all this in mind, I set out to create a website that would overcome these historic limitations to academic communication. I created a new website called ChemRank: http://www.chemrank.com Screen Shot shown [deleted in this post – PMR]
At ChemRank you can add a paper to the database and then vote whether you find it a good or bad paper. You can also leave comments critiquing the yield or congratulating the authors on a job well done. Most importantly, there is a public record of your views! Your pain does not need to be repeated if you point out the problematic reaction or the incorrect eqn in the comments section.
An other cool feature of ChemRank is the building of a database that knows the good chemical literature from the bad chemical literature. Currently, there is no database to my knowledge, except you own brain or PI’s brain, that tracks this information. Although, Noel’s experiments with Connotea is the step in the right direction.
There is even something for the already established pioneers in online chemical literature discussions (aka chemical bloggers) too. For each paper added to the database a script, behind the scenes, will generate a little code snippet to allow voting on the literature your discussing on your blog too. For example, if you are discussing the recent paper “Atomic Structure of Graphene on SiO2” you can generate the following from copy/pasting a code snippet. Assuming you’ve already added the paper to the database and clicked the link: “add to your blog

Atomic Structure of Graphene on SiO2

code snippet:

Code: (html)

This by no means is a completed project, there is still much more to do.

  • Make links that show the papers posted last 7days, 1month, 1 year, all time
  • Make users register before posting.
  • Add the ability to tag the articles with relevant keywords
  • Make a pipe’s rss feed of the author’s most recent papers, on the fly and behind the scenes, display in the description as well.
  • Make an api so others can access the database and use it for cool-new mashups.
  • Add a search feature, but that is kind of pointless while the database has less than 10 papers in it.

All of the above can be done in time, but it depends on the feedback from the community and the popularity of the website. As with all projects, I don’t necessarily expect it to catch on right away if at all. A lot of times I code, simply to show how to do it, and how to do it well. Allowing public comments on the current literature is something the chemical publishers should be doing anyways, now we no longer have to wait for them…
Mitch

Comment:
This and similar approaches are very important. We don’t know where metrics like this will take us but it can’t be worse than the current citation index which is managed by … by whom???
Before anyone puts too much effort into their own system the should check out carefully what the “Web 2.0” community does for review. As I blogged earlier, Revyu.com – Review Anything has tools for managing and collecting reviews – no technical reason why they shouldn’t actually include chemistry AFAIK. I would strongly encourage using RDF for reviews – this will happen anyway and makes your material discoverable more easily.
There are reviews in the chemical blogosphere already – TotallySynthetic.com runs a poll for the best synthesis of the month/year, etc. I’m not sure whether there is any technology that publishes this in RDF form but it would be relatively easy. Commentaries in the blogosphere also have a greasemonkey viewer from Chemical blogspace
and other sites, so that when you read the table of contents the monkey alerts you to any commentary. It would be technically easy to fix the monkey so it counted eyeballs on papers – how often was this paper READ (not cited, not downloaded – but at least displayed in the browser presumably with a human somewhere near). And to add tools so that the monkey could ask for opinions from the viewer.
I’m not suggesting there should be one coordinated effort – anymore than we need just one movie review database. But I do think that reviewers should try to use the same technology so that their results can be aggregated – and the reviewers can themselves be reviewed!

This entry was posted in "virtual communities", chemistry. Bookmark the permalink.

6 Responses to Ranking chemistry and blogosphere metrics

  1. Tom Heath says:

    Hi Peter,
    An interesting post! You’re absolutely right, Revyu could certainly accommodate reviews of chemistry material. I’d love to see people having a go at this (or any other discipline for that matter).
    If you’d like to review resources that are already online then just follow the normal reviewing procedure on the site, and put the URL of the resource in the “Web-only Resource” link field on the reviewing form. This will then do the right thing semantically between the scenes (shout if you want more info on what it will be doing exactly). If you want to review things that aren’t strictly online, or where this could be debated (e.g. journal/conference papers are a bit of a grey area), then get in touch and we can delve into the whole DOI issue.
    Of course once the reviews are in there, then you can get them out in a number of ways: one line of JS in your HTML source, RSS, straight RDF, or best of all SPARQL queries to the Revyu SPARQL endpoint. They’ll also be crawlable and immediately contributing to the web of data. Let me know if there’s anything I can do to help out.
    Cheers, Tom.

  2. pm286 says:

    (2) Thanks Tom,
    This is very useful.I’ll leave the others to pick it up – it’s good to hear this definitive statement. FWIW I picked up revyu from Dan Connolly and Harry Halpin at WWW2007 who were giving examples of bringing together several RDF vocabularies – FOAF, Revyu, vCard and DC to show how someone could get advice of the best hotels but only from her trusted friends. This would be very useful for chemistry – what papers do my friends think are good.
    I am very conscious of the journal/DOI thing. I have no idea what the journals think about these blossoming communities of comment. It should certainly change their mindset. I don’t see the DOI as an issue because as long as *someone* can comment on it then they can leave a revyu. Of course the others may not all be able to read it – but that’s a bit like reading a film review when you can’t afford to go to the cinema.

  3. Tom Heath says:

    Hi Peter,
    Interesting to hear that Revyu featured in Dan and Harry’s session in Banff. The ultimate goal for Revyu data (and the real focus of my research) is exactly this sort of scenario: recommendations from trusted friends (but who do you trust ;).
    Is there any chance you could summarise the current DOI situation as you see it? I got very excited about them several years ago, then someone suggested there was a big political issue around them and poured water on my enthusiasm. Now with Linked Data really taking off it would seem time to revisit the issue. An expert’s precis of the state of the art would help immensely. Are people really using them?

  4. pm286 says:

    (3) Are people really using them?
    Yes. Universally. Who owns them? ??? They are one of the more successful unique IDs. I may blog about this later…

  5. baoilleach says:

    I feel a Greasemonkey script coming on…

  6. pm286 says:

    (5) 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *