I realised with considerable disappointment ( Can I data- and Text-mine Pubmed Central?) that I might not be able to text- and data-mine the material that the NIH has required to be deposited in Pubmed Central in its mandate. Now I have got confirmation by email from an authoritative source (who asks not to be named in case the information is not quite precise). But in general terms the answer is simple:
NO-ONE MAY DATA- OR TEXT-MINE PUBMED CENTRAL
In short Pubmed Central is “free access” (no price barriers), not “open access” (no permission barriers). You may not download material from it (except to expose it to your own eyeballs), and certainly not redistribute it. You may not data-mine it.
I am aware of the struggle that was required to get George Bush to sign the mandate and it certainly wasn’t the time to break ranks. But now that the mandate is passed (and starts tomorrow) we must press ahead immediately to campaign for full access to the text.
We have the right and the duty to submit our views to NIH. For example Stevan Harnad has argued (recommendations to the NIH) that it is better to reposit in institutional repositories (“green”). Whether or not this is a good idea (and I personally don’t think so as it make datamining almost impossible) it is clearly outside the current approach from Pubmed Central. For example, I gather, the mirrors of PMC have to agree to the same absolute permission barriers that PMC imposes – it would be impossible to ensure that thousands of libraries enforced this – almost draconian – contractual system.
So we have to argue to the NIH that bioscience is desperately impoverished by the unreasonable permission barriers that are now in place. I’m not a (US) politician and I think the NIH and advocates have done well to win the first battle. But at present the policy is seriously hindering modern science.
So the whole area is incredibly complex. The goal is simple – use scientific publications to further our understanding of science and – hopefully – make progress in enhancing human health. For this we MUST have robots. We cannot do it with humans alone – every week we get thousands of new papers.
I’d be grateful to know what the position is with Wellcome. I thought they had removed permission barriers.
-
Recent Posts
-
Recent Comments
- pm286 on ContentMine at IFLA2017: The future of Libraries and Scholarly Communications
- Hiperterminal on ContentMine at IFLA2017: The future of Libraries and Scholarly Communications
- Next steps for Text & Data Mining | Unlocking Research on Text and Data Mining: Overview
- Publishers prioritize “self-plagiarism” detection over allowing new discoveries | Alex Holcombe's blog on Text and Data Mining: Overview
- Kytriya on Let’s get rid of CC-NC and CC-ND NOW! It really matters
-
Archives
- June 2018
- April 2018
- September 2017
- August 2017
- July 2017
- November 2016
- July 2016
- May 2016
- April 2016
- December 2015
- November 2015
- September 2015
- May 2015
- April 2015
- January 2015
- December 2014
- November 2014
- September 2014
- August 2014
- July 2014
- June 2014
- May 2014
- April 2014
- March 2014
- February 2014
- January 2014
- December 2013
- November 2013
- October 2013
- September 2013
- August 2013
- July 2013
- May 2013
- April 2013
- March 2013
- February 2013
- January 2013
- December 2012
- November 2012
- October 2012
- September 2012
- August 2012
- July 2012
- June 2012
- May 2012
- April 2012
- March 2012
- February 2012
- January 2012
- December 2011
- November 2011
- October 2011
- September 2011
- August 2011
- July 2011
- May 2011
- April 2011
- March 2011
- February 2011
- January 2011
- December 2010
- November 2010
- October 2010
- September 2010
- August 2010
- July 2010
- June 2010
- May 2010
- April 2010
- August 2009
- July 2009
- June 2009
- May 2009
- April 2009
- March 2009
- August 2008
- July 2008
- June 2008
- May 2008
- April 2008
- March 2008
- February 2008
- January 2008
- December 2007
- November 2007
- October 2007
- September 2007
- August 2007
- July 2007
- June 2007
- May 2007
- April 2007
- December 2006
- November 2006
- October 2006
- September 2006
-
Categories
- "virtual communities"
- ahm2007
- berlin5
- blueobelisk
- chemistry
- crystaleye
- cyberscience
- data
- etd2007
- fun
- general
- idcc3
- jisc-theorem
- mkm2007
- nmr
- open issues
- open notebook science
- oscar
- programming for scientists
- publishing
- puzzles
- repositories
- scifoo
- semanticWeb
- theses
- Uncategorized
- www2007
- XML
- xtech2007
-
Meta
There is a subset of PubMed Central called the Open Archives Service. You are allowed to index this since it consists of articles where publishers have deposited specifically with open access in mind. This subset is OK to index/datamine and link back to (provided you use the PubMed e-utilities – not a web spider directly to the URLs which is forbidden in robots.txt – and respect request delays – one per 3 secs I think)
If you want to extract and host (as opposed to link back to / datamine for personal use) I am less sure.
(1) thanks Will – I should have made this clearer.
Yes – I am aware of the Pubmed Open Archives. This effectively reflects the material which would be available without the mandate. (FWIW I always try to creare robts which are friendly.)
Unfortunately relatively few authors either publish in Open Access journals or bought the full open Access rights from the publishers. The mandate requires the publication of the material with “free” or “public” access. My understanding is that this material cannot be spidered,
Did your private correspondent say anything about license issues? I’d say that this NIH policy violates the CC-licenses used by some of the more liberal publishers.
I have discussed my questions on the legal aspects of PMC versus the CC-BY 2.0 license requirements. The latter forbids putting technical obstacles to control access, exactly what PMC seems to be doing.
See http://chem-bla-ics.blogspot.com/2008/04/legal-advice-needed-nih-restricting.html
So what does BioSearch actually search?
http://biosearch.berkeley.edu/
(3, 4). Egon, the articles do not carry a CC licence – they retain the publisher licence AFAIK.
(5) It says Open Access journals, so I assume that…