As regular readers will know we are applying text-mining to chemistry in Open theses. The problem is finding fully Open theses – so far we have got Alicia’s. Alicia has captured all here molecules in semantic form so text-mining isn’t required – and I’m hoping to do some fun stuff with XML on it.
I’ve searched for large collections of theses. MIT has a promising collection which would be ideal but they are only TollFree, not OpenAccess. I’m still appealing for readers to help. But in one of those quirks of Googling I ended up at the digital repository of the University of Stirling.
I am delighted about this since I spent 15 very happy years on the staff at Stirling in the Chemistry department. It doesn’t have one now – that’s why I left 🙁 – although we’re having our 40th anniversary later. But perhaps there are theses with chemical concepts Now the repository announces:
The copyright in theses in this collection remains with the author, unless it is stated to have been assigned to the University of Stirling. The University of Stirling reserves the right to keep electronic copies for consultation in both cases.
so I wasn’t very hopeful. But I thought I’d have a look and found one in aquaculture – one of the successful disciplines in Stirling:
Feeding behaviour of the prawn Macrobrachium rosenbergii as an indicator of pesticide contamination in tropical freshwater.
which carried the licence:
This item is protected by original copyright
Items in the Repository are protected by copyright, with all rights reserved, unless otherwise indicated.
still not hopeful, until I read the license:
License granted by Kriengkrai Satapornvanit (ffiskks@ku.ac.th) on 2007-03-26T06:34:59Z (GMT):
[...]
END USER LICENCE
This work is licensed under the Creative Commons
Attribution-NonCommercial-ShareAlike 1.0 Licence.
YOU ARE FREE:
- to copy, distribute, display, and perform the work
- to make derivative works
Under the following conditions:
ATTRIBUTION
You must give the original author credit.
NON COMMERCIAL
You may not use this work for commercial purposes.
SHARE ALIKE
If you alter, transform, or build upon this work, you may distribute
the resulting work only under a licence identical to this one.
For any reuse or distribution, you must make clear to others the
licence terms of this work. Any of these conditions can be waived
if you receive permission from the author. Your fair dealings and
other rights are in no way affected by the above.
Many public thanks, Kriengkrai Satapornvanit and I hope your future work prospers. Now I am completely free to see if chemicals can be mined from the thesis:
- I download the PDF.
- I convert it to ASCII using pdftotxt. This destroys the formatting, diacritics, tables, subscripts, superscripts – in fact almost everything except the words. And – for most theses – these are still in the right order. (Unlike Eric Morecambe, this is not a joke – PDF often has the words in an arbitrary order).
- I start OSCAR (http://oscar3.sf.net) as a Server and paste in the text. It takes about 30 seconds for OSCAR to read the whole thesis and interpret the chemistry. (This portable version of OSCAR does not have a complete lexicon – full versions need to be run on a server).
- OSCAR (originally dubbed the “journal eating-robot”) eats the text. Here’s a typical section:
You can see that OSCAR has recognised many words as likely chemical terms (in yellow) and knows the structure of the underlined ones (the full version would know all of them). It’s not 100% accurate – you can see it thinks “P” is the element phosphorus – but Peter Corbett has addressed this in later versions.
So this allows us to collect metadata from theses automatically. OSCAR can tell us in a few seconds that this thesis is concerned with specific pesticides. That’s part of the basis of the SPECTRa-T project. Since we’ve benefited from Open Source theses, maybe we should do the whole project on an Open Wiki…
June 12th, 2007 at 3:37 am eOpen Access: What Comes With the Territory
Peter Murray-Rust’s worries about OA are groundless. Peter worries he can’t be be sure that:
Pay no attention. Download, print, save and crunch (just as you could have done if you had keyed in the text from reading the pages of a paper book)! [Free Access vs. Open Access (Dec 2003)]
It will. The University OA IRs all see to that. That’s why they’re making it OA. [Proposed update of BOAI definition of OA: Immediate and Permanent (Mar 2005)]
Versions are tracked by the IR software, and updated versions are tagged as such. Versions can even be DIFFed.
You may not create derivative works. We are talking about someone’s own writing, not an audio for remix, And that is as it should be. The contents (meaning) are yours to data-mine and reuse, with attribution. The words, however, are the author’s (apart from attributed fair-use quotes). Link to them if you need to re-use them verbatim (or ask for permission).
Yes, you can. Download and crunch away.
This is all common sense, and all comes with the OA territory when the author makes his full-text freely accessible for all, online. The rest seems to be based on some conflation between (1) the text of research articles and (2a) the raw research data on which the text is based, and with (2b) software, and with (2c) multimedia — all the wrong stuff and irrelevant to OA).
Stevan Harnad
American Scientist Open Access Forum