Is "peer-review" holding back innovation?

As part of my talk at Open Scholarship I’m going to show two pieces of scholarly work of which I am proud, which I believe fit all the criteria of publication and for which I get no formal credit. (I also regard this blog as a scholarly work, and also get no credit)…
The first is an invited talk at Google. (Yes, I can claim some minor formal credit for an invited talk, but probably not to a company!) This was videoed and has received 1727 downloads and 12 5-star ratings. (Of course some of this may be donw by robots or my friends, and probably some of them only watch the first few minutes, but there must be some serious viewers). It has everything a scientific publication requires:

  • accessible
  • peer-reviewable
  • formal record
  • re-usable
  • archivable (When I have time I’ll put it in our DSpace…)

The second is our WorldWideMolecularMatrix (WWMM). This is an evolving system for open access to the world’s molecules and properties and as part oif it we have put 175, 000 objects in the Cambridge DSpace. But it has never been formally published in a full paper. That’s partly because it’s not finsihed and partly beacuse everyone can see it. Why publish it?
But it has been peer-reviewed! Someone – I have no idea whom – started a Wikipedia entry. I’m naturally proud of this. The entry quotes extensively from the talk I gave at OAI4 in 2005 at CERN (“CERN Workshop on Innovations in Scholarly Communication (OAI4)” ) (Video). Joanne Yeomans recorded this talk as a video and this has – I gather – been regularly accessed. Again it has most of the features of a publication – but I can’t get any formal credit for it.
So to the current UK Research assessment exercise (RAE) – 4 citable papers in peer-reviewed journals does not allow for this type of innovation in scholarly publishing. Should I abandon the new approaches and concentrate on paper? It’s what the management would like…

This entry was posted in open issues. Bookmark the permalink.

7 Responses to Is "peer-review" holding back innovation?

  1. Pingback: Unilever Centre for Molecular Informatics, Cambridge - petermr’s blog » Blog Archive » Presentation to Open Scholarship 2006

  2. Peter – this is a fundamental issue that we need to continue discussing. I think the real issue is that we have to separate the problem of efficiently communicating scientific information from the problem of convincing a committee of the impact of a faculty member’s scholarship. And when doing open science, the first concern is the communication of the information. More here:
    http://drexel-coas-elearning.blogspot.com/2006/02/peer-review-in-google-age.html

  3. Egon says:

    One part of the discussion is that ‘paper’ journals get stored down in cellars of libraries. So, theoretically it is possible to dig up papers published 100 years ago. Digging up a website of 10 years ago is impossible. I think initiatives like DSpace help, but things need to formalize first. And I do think peer review is important, on whatever way of transporting the science. I don’t think peer review itself is holding back science (just delaying it some 3-24 months, depending on the journal).

  4. Egon, actually a lot of the web has been archived through the Waybackmachine. For example, here is my institution’s website from 1997
    http://web.archive.org/web/19971024094507/http://www.drexel.edu/
    Although I think that formalizing (and by that I assume you mean rendering the information within a semantic framework) is an important part of the process, I disagree that it has to come first. I think that it is most important to get the information out there first in some findable format. The chemists looking to make DOPAL (one of the chemicals we have made successfully) are not going to care about where that information is hosted or if all possible metadata have been generated. They are also not going to care that it has been blessed by some anonymous group of “peers”. They are going to read the protocol, look at the NMRs, TLCs and other experimental proof and then decide if it is crap or if it is worth following. If we insist that no scientific information be disseminated without some type of peer review we will be holding back science because there is no way that every piece of data can be examined by a group of non-paid volunteers. By this standard, we would also have to audition every speaker at a scientific conference to make sure that all their words are peer-reviewed before speaking to the masses.

  5. pm286 says:

    (2) Yes – this is exactly the issue. In the area of Open Source software the communication is 100% important. At one stage in the past we paosted a darft of a CML paper for comment (not widely advertised but still publicly accessible). The journal editor said it had to be taken down immediately or it couldn’t be published. This is an interesting view of quality control!
    (3/4) Archiving is distinct from publication and from peer-review. Unfortunately witn electronic journals the archive is often only held by the publisher (some of whom have promoted the idea that they alone are the true holders of the record). The reader only has a time-limited access to a database (the publishers’ word) – not a journal. Oh dear…
    (4) In chemistry I agree. I don’t need peer-review – though it is valuable to have quality control (and many peer-reviwers do this). However in some disciplines (e.g. medicine) it is vital to have peer-review of clinical trials.

  6. stelt says:

    You amazed me with a demo at an SVG meeting in London about 2 years ago.
    Open semantic web technologies more powerful than many people could even imagine, let alone think it exists, let alone make it exist.
    I put the Google video up on an open technologies blog at my university. This video is a great tool for open technology advocates. I’m a fan

  7. pm286 says:

    (6)Thanks!
    It is still a pity that SVG is not universal. We find problems everytime browsers are upgraded. And the animation only works on MSIE. The scientific power is shown at http://www-mitchell.ch.cam.ac.uk/macie/
    see 100 Entries and (say) M0002.
    I assume you are referring to http://video.google.com/videoplay?docid=5871900250712259806 the talk I gave at Google. That was fun.

Leave a Reply

Your email address will not be published. Required fields are marked *