Article Level Metrics – how reliable are they? (I prefer to read the paper)

I am on the board of a wonderful community voluntary organization – the Crystallography Open Database (COD) http://www.crystallography.net/ . For 10 years it has been collecting crystal structures from the literature and making them Open – more than 300,000. It’s the only Open database for small structures (the others CSD and ICSD are closed and based on subscriptions even though the data is taken from public papers. This morning we heard of a great paper using the COD for data-driven research. Here’s the landing page http://www.mdpi.com/2073-4352/5/4/617 :
articlemetrics0
 
The paper is trawling through hundreds of thousands of structures to find those with a high proportion of hydrogen atoms – a clever idea for finding possible Hydrogen Stores for Energy. The closed databases couldn’t be used without subscriptions.
I think this is a clever idea and tweeted it. I’m a crystallographer and structural chemist so it’s not surprising that a few other people retweeted it.
However I noticed that the article had a daily count of accesses and that there was a small glitch of 3 accesses today. I tweeted this and – surprise – the accesses went up. after 12 hours there have been over 100 accesses
articlemetrics
 
You’ll see there have been over 100 accesses today because I and 3-4 others have tweeted it. This is nothing to do with the contents of the paper because not many have actually read it today. People have clicked to view the graph, and every time they visit the graph goes up. It’s nothing to do with the quality of the science (which I think is good) or the fact that the paper is Open Access – it’s just a Heisentwitter.
So what does 100 accesses today mean? Nothing.
What does my opinion of the paper count? Something, I hope (I would have recommended publication).
The point is that to decide whether science is good or useful
you have to read the paper
 

This entry was posted in Uncategorized. Bookmark the permalink.

One Response to Article Level Metrics – how reliable are they? (I prefer to read the paper)

  1. Peter, to me #altmetrics are still in the first place numbers that reflect the amount of attention (as proxy to impact) has, but importantly, the tools that calculate them provide the provenance of the count and it is this network that has value to me. It is not the number that tells me much about a paper, about just as much as the citation count, but the links to other papers (citations) and other communication channels points me to related discussions. This way, it provides a richer scientific communication and therefore dissemination channel. In a lot of cases the things that boost the #altmetrics are not really about the science, as you note too; this is not really different from citation counts either, where many citations are more of the likes of “hey, look, they did something that smells like X too” (which is a reason why we must adopt a solution like the Citation Typing Ontology, but that’s only a related discussion; http://www.jbiomedsem.com/content/1/S1/S6). But I compare tweeting about a paper like putting a paper on someone desks, or blogging like having a poster at a conference to make people aware of my research.
    It indeed says nothing about quality, but also not less that more traditional ways to make people know about your own or research by others. Or?

Leave a Reply

Your email address will not be published. Required fields are marked *