SPECTRa and SPECTRa-T

Every chemistry department spends millions each year on determining crystal structures, calculating properties of molecules and measuring spectra for newly synthesised compounds. The data are potentially extremely valuable, but almost all of them are lost. Of those that are saved and published, many are given to the publishers or aggregators who then sell them back to the community.
We have just completed an 18-month project, SPECTRa, (a collaboration between chemists and libraries at Cambridge and Imperial College – see home page) to see what could be done. We verified that there was a massive problem of data loss and that most of the factors were people-related, not technical. The final report will be out soon (and I and/or Jim Downing will probably blog it then) but the additional point is that we are looking at how to sustain the momentum (sustainability is a major problem with many short-term IT-based projects) – particularly without specific funding. Our approach is to develop informal partnerships with other institutions (libraries and chemists) specifically to use the Open Source tools we have developed. For example we have collaborated with the University of Southampton (who run the national Crystallographic Service). We have also got 2-3 early adopters outside the UK who are working out how they can adapt the process to capture their data (crystallography in the first instance).
How can we base a sustainable business model on zero funding? We can’t, of course, and it will only work where the institution cares about preserving data which it spent millions to create. Our tools are Open and therefore free, but they require local business processes to be developed and installed. Departments and institutions have to realise that preservation of data is a necessary investment. It needn’t be large and it can be aggregated with the current costs of (say) promoting research output, preparing publications, thesis preparation, etc. Chemists may also find that their LIS (Library Informations System) group is interested in this type of project – it’s an ideal place to start for data capture and preservation and the likelihood of technical success is high. In general any departmental service should cost information systems as part of their operation.
The great thing about the Web, of course is that it allows you to find the best and most enthusiastic collaborators. So if anyone out there is interested in saving their chemical data…
SPECTRa itself has finished but JISC has funded a new project (SPECTRa-T). It’s not a continuation of SPECTRa, but it uses the ideas gathered there and applies them to chemical theses. Probably at least half the chemical data generated in departments is only ever published in theses (the supervisor moves, the student’s data cannot be found, etc.). So SPECTRa-T is looking at how robots can manage the semantic preservation of this automatically. Our OSCAR3 tool (Peter Corbett) can read a thesis in a few seconds and extract thousands of instances of chemical metadata along with structured objects (crystal strucures, spectra, etc.).
So we are interested in institutions which have Openly available chemical theses… If these are available in *.doc format that is much better that than the hamburger PDF that most institutions require for deposition. But we may even be able to do something with than.