I asked a simple question:
“What is the single raison d’etre of the Journal Impact Factor in 2011?”
And have had two useful answers:
Zen Faulkes says:
For me, it’s to ensure that the journal I submit to is a real scholarly publication. There are a lot of new online journals opening up. Some of them are not credible. For me, that a journal has an Impact Factor lets me know that sending a manuscript there is not just the equivalent of burying the paper in my backyard.
and
Laura Smart says:
Ultimately it boils down to evaluating academics. As Zen Faulkes says, academics do use it as a measure of quality for journals where they may choose to publish, however flawed it may be. It’s an easy shorthand. Everybody within the current academic publishing system uses it in this fashion whether it be grant reviewers, hiring committees, tenure committees, peer reviewers, or faculty considering where to volunteer their effort as editors/editorial board members. Grant providing bodies use it when evaluating the publications produced from awards. Publishers may use it slightly differently: as a marketing tool for selling value. But who are they marketing to? The academics who are using the journal impact factor to evaluate one each worthiness.
It’s been said for 15 years (or more) that the responsibility for changing the scholarly publishing system rests with changing the organizational behavior of the institutions producing the scholarship. People have to stop using journal impact factor as a judgment tool. This won’t happen until there is incentive to change. The serials pricing crisis and usage rights issues haven’t yet proved to be incentive enough, despite lots of outreach by librarians and the adoption of Open Access mandates by many institutions.
Scholars won’t change their behavior until the current system affects their ability to get funding, get tenure, and advance their careers.
These are valuable comments and I’ll use them to introduce why I think we have created Monsters of the Scholarly Id. The JIF is probably the worst as it is not only flawed but its use shows that academia does not really care about measuring quality. The JIF was not created by academia, it was created by publishers as a branding instrument. And that is precisely what it is – a branding tool, created by the manufacturer. It was neither designed nor requested by academia but, as Laura says, it has been adopted by them. They do not control it and so they are in its grip (more later).
Brands can be valuable. http://en.wikipedia.org/wiki/Brand gives The American Marketing Association defines a brand as a “name, term, design, symbol, or any other feature that identifies one seller’s good or service as distinct from those of other sellers “. The branding of household products in the 19th century by pioneers such as William Hesketh Lever (http://en.wikipedia.org/wiki/Lever_Brothers ) where “The resulting soap was a good, free-lathering soap, at first named Honey Soap then later named “Sunlight Soap“. Until that time soap had been of highly variable quality and the branding by Lever allowed customers to associate a brand with consistent and high quality. Many other businesses followed suit.
It is fairly easy to determine whether soap is of good quality or substandard. Whether a car is reliable or breaks down. For many other products (beer, clothes, fragrances, …) the association depends on subjective judgments including a large amount of personal preference. Which brings us to the branding of journals.
I am going to argue later that we do not need journals and that they are increasingly counterproductive. However, assuming that we do need them, is branding useful? Branding is now common – the journal carries the publisher’s name and may have a consistent visual look-and-feel. But visual consistency does not mean valuable or even consistent science.
Journals are – unless you tell me otherwise – unregulated. And that’s how it should be. Anyone can set up a journal. http://en.wikipedia.org/wiki/Liebig founded a journal “The volumes from his lifetime are often referenced just as Liebigs Annalen; and following his death the title was officially changed to Justus Liebigs Annalen der Chemie.” Many blogs have the effective status of journals and many contain high-quality scientific content. (Certainly I would encourage anyone who had something to communicate about scholarly publishing to blog it rather than using a scholarly journal such as Serials Review). So Zen quite rightly asks (implicitly) about journal regulation.
I think he is right to ask for it, though the JIF is not a regulation – it’s a branding sought by the publisher for the benefit of the publisher. Nothing specifically wrong with that, but to assume it acts in the interests of academia is to misunderstand branding. It’s primary purpose is to give a single, apparently objective and regulated, number giving an apparent indication of quality. I use the word “apparent” as that is what academia consumes, but since the process of JIF creation is not transparent it is not objective. (That is separate from whether it measures anything useful – which I believe it does not).
Because the number of journals has risen so rapidly it is impossible, even within a field, to determine the standing of any particular one. (Why it has risen I’ll try to deal with later, but it’s not because readers are asking for more journals). So presumably we can rely on the reputation of a publisher justifying the quality of a journal.
Unfortunately not. (This news is 2 years old…) See http://classic.the-scientist.com/blog/display/55679/ where The Scientist in 2009 revealed that
Elsevier published 6 fake journals
|
Scientific publishing giant Elsevier put out a total of six publications between 2000 and 2005 that were sponsored by unnamed pharmaceutical companies and looked like peer reviewed medical journals, but did not disclose sponsorship, the company has admitted. |
Read more: Elsevier published 6 fake journals – The Scientist – Magazine of the Life Sciences
http://www.the-scientist.com/blog/display/55679/#ixzz1SH3VUqmM
This is not denied by Elsevier who stated:
“We are currently conducting an internal review but believe this was an isolated practice from a past period in time,” Hansen continued in the Elsevier statement. “It does not reflect the way we operate today. The individuals involved in the project have long since left the company. I have affirmed our business practices as they relate to what defines a journal and the proper use of disclosure language with our employees to ensure this does not happen again.”
It is gratifying that Elsevier have indicated that there was – in 2011 language – a “single rotten apple” and that the problem has been cleaned up and we can relax for the future. And I am sure they are grateful to the Scientist for discovering the problem which lay undetected for several years. Nonetheless it shows the commercial pressure to publish journals. Unlike the journals I grew up with, which were the outputs of learned societies and were to promote science, the primary purpose of most (not all) of today’s journals is to make money. (In the MGS we subsidised the journal from the membership – tempora mutantur). I talked about 4 years ago to someone whose business was creating new journals. His recipe:
- Find an area (his was medical) where he could create a niche demand. The demand didn’t have to exist, it just had to be creatable.
- Create a journal, with luminary editorial board. Find the senior editor. Academics like to be on boards. It makes them look good on their CV. Sometimes they even get jollies. (Disclaimer: I have had one free jolly – a (working) breakfast from the J. Cheminformatics: Coffee, fruit, donuts – probably 10USD).
- Get a reasonable number to submit papers for the first issue. They won’t be critically reviewed will they? After all it’s the editorial board. And we need it to look good. Doesn’t really matter if you take and old paper, rework it a bit as a review with some new work. And get the grad student to do the hard work of the references and some pretty pictures.
- Get academic libraries to subscribe (this was closed access, reader pays). Most very large universities would do this.
- Wait two years and sell the journal to a major publisher for ca 100K GBP
Everyone benefits.
Except academia, who has subscribed to yet another albatross. But there’s lots of money in the system. And anyway the researchers don’t pay, the library does. And we need the freedom to publish, don’t we?
Checklist of monsters (MOTSI) so far:
- The branded journal
- The new journal
- The journal impact factor
(there’s more to come). But since this is already a long post, let’s have a separate post on the worst of the new journal…
So, Zen, we do need an independent reviewer of scholarly publishing. A consumer magazine “Which Journal?” But the impact factor, which is negotiated by publishers with non-answerable commercial companies in a closed process does not provide it. http://en.wikipedia.org/wiki/Quis_custodiet_ipsos_custodes%3F . It should be academia, but it doesn’t seem to be. After all there are more urgent things to do than monitor our own quality. We’ll do the research and let the commercial sector tell us how. (And in “commercial” I include the major non-profit societies which have become unbalanced and use publishing to fund their activities rather than the other way around).
So your next assignment (after all we rely on citations so much)
“What is a citation?”
Answers within 24 hours welcome
With respect, Peter, I don’t think the Journal Impact factor was “created by publishers as a branding instrument”. In fact, Academia (in the guise of the NIH) seems to have been involved, and originally it was just a method of measuring the influence journals had. Read this article by Gene Garfield, the ‘father of citation measurement’ (my characterisation of the man): http://jama.ama-assn.org/content/295/1/90.full
As with just about any technology or methodology, it can be used for different purposes, including those they were not intended for. The fact that publishers use it for branding, and academics use it as a lazy man’s way of putting their colleagues in a convenient pecking order, cannot be blamed in the JIF. If anybody should be blamed, it’s the academics and publishers themselves for using inappropriate tools, and gaming the system to boot.
Einstein’s well-known quote applies here and should be heeded more: “Not everything that can be counted counts, and not everything that counts can be counted.”
Journals as a selection and ‘stratification’ method of the scientific literature was probably necessary in the print era. Now we have entered the electronic era, journals as selections or collections op papers make less and less sense, and PLoS One – although still called a ‘journal’ – is an indication of where it is likely to go: a platform on which a wide variety of articles can be published after only ‘technical’ peer review, subsequently to be sliced and diced in whatever collections or selections one might find useful. Journal ‘branding’ will disappear as a mark of quality of content, I think, and if it becomes anything, it is likely to be a mark of service to authors (is the publishing fast, easy, open, the content easily navigable, nicely presented, computer-readable, etc.). With article-level metrics, the real brands to be reckoned with will be authors, collaborations of authors, and possibly research establishments.
Many thanks Jan,
One of the features of blogs is that I can get feedback from the community and correct my knowledge and idea.
And I will do so.
And I agree fairly completely with your analysis of the need to move onwards from journals and the value of PLoSOne-like structures