Peter Suber has picked up a point that I made at the RSC Open Access Meeting and I’m happy to address it:

I missed this in the chemical blogosphere and was alerted by Peter Suber: Facilitating the exchange of chemical data

Chemspider describes the NMR prediction service provided by the Ecole Polytechnique Fédérale de Lausanne. NMR Prediction Now Available Via ChemSpider

00:56 22/05/2008, Antony Williams,

[…]. The NMR prediction service is provided by Luc Patiny’s group out of Ecole Polytechnique Fédérale de Lausanne at the Institute of Chemical Sciences and Engineering. Their nmrdb.org webpage offers a series of services, not just NMR prediction and I offer the details below from their website.
NMR Predictor – This page allows to predict the spectrum from the chemical structure
NMR Assigner – Upload and assign NMR spectra on-line. The assignment of NMR spectra may be decomposed in 4 steps:

1. identification of the signals
2. integration and multiplicity determination
3. assignment of each signal to the corresponding atom in the molecule
4. exportation of the data for publication and/or for database storage

NMR Resurrector – A great amount of NMR information is currently available in the form of scientific publications. However, this information is not readily accessible in the format required for complex searches. The Resurrector enables the user to easily import these in-line spectral descriptions and creates an assigned visual representation that can be seamlessly integrated in the attribution process.

PMR: Firstly, I think it’s very important to provide free online predictions of NMR spectra and Lausanne are to be congratulated. The technology is very valuable in the age of machine-readable chemical information. I’m not sure quite what the primary site at Lausanne provides – it states that the prediction is provided by a (presumably commercial) engine from Molecular Networks. I don’t know whether there are restrictions on the volume or re-use of the calculations done at Lausanne. If there aren’t any, this is a useful advance (though I am still keen to see the algorithms of prediction tools exposed ).
Again I may have missed it, but the primary means of calculating NMR spectra at Lausanne seems to be through an applet. I’d like to see Web Services that can be used by machines, rather than needing to enter structures by hand (I’m pleased to see Pubchem is taking the WS approach and I’ll blog this later). Computing is becoming indefinitely cheap and it makes sense to compute the spectrum of every new compound published.

Christoph Steinbeck has given a huge account of the RSC meeting so I don’t have to say anything. Here’s what I seem to have said (As I have no powerpoint or linear trajectory my talks do not follow a set course and are determined in part by audience reaction).

Open Access Publishing in the Chemical Sciences

May 23rd, 2008 · No Comments

I was invited to give my views on some new chemistry in European Bioinformatics at a Meeting held by the CICAG group of the Royal Society, held at Burlington House, London.
Peter Murray-Rust set the scene by emphasising the importance for Open Data. He showed some fantastic work on data extraction by OSCAR from theses, where his group had parsed a synthetic chemistry thesis into an interactive graph of a reaction network. He also showed an SVG animation of this graph as a reaction sequence, all automatically generated from an OSCAR run. Peter pointed out in the subsequent discussion that data cannot be copyrighted, which was acknowledged by all publishers in the audience. The reality is different, however, because publisher’s licenses often prevent downloading of more than few articles in a row. Detection of a robotic download for text mining comes with the danger of the whole university being disconnected. It is unclear to me how robotically parsing papers and extracting data would damage the bushiness model of publishers. It could, of course, lower the number of subscriptions from

PMR: The main thing we took away was the importance of factual data. No-one disputed that facts could not be copyrighted (though not all realised that copyright was only one of the methods used by publishers to control access and re-use – server-side beheading is completely effective). I asked the audience – > 30 composed of publishers, librarians, software companies, etc. – no actual chemists of course – whether anyone would object to our robots reading the literature and extracting the data from the papers whether as text, images of tables. Half the audience thought I should, the rest didn’t vote against.
So, publishers, I’m going to start mining data from your sites. I hope you welcome this as a way forward t a new exciting era of data-rich science publishing. I hope that if you don’t agree you’ll let me know. I wouldn’t like to start and then get the lawyers sent. So please comment – it’s very important. I shan’t attack anyone who sends a reply. And you can send it by confidential email if you like.
There are a million new compounds each year in the scholarly literature. Our robots can produce huge amounts of good information from it. In some cases we get over 90% recall and precision – it depends on the type. This must be good for science. So please, publishers, let us know we can do it and we’ll publicly thank you. And if you don’t like the idea, please let us know why.
I’m in Barcelona at COST D37 helping to develop computational tools using CML. It is really changing the way things are done.
More on both fronts later.

I normally try to blog some of my presentations before the event so that at least there is some sort of record. It also allows for feedback from readers. So I’m talking on Open Data in Chemistry. I’ve been working very hard to create a new demo of the future that chemistry could have if it wished and I think it’s working. I believe – not surprisingly – that every publisher of chemistry should look at it carefully. Because it helps to change the shape of the technical chemical publishing.
The theme is “Open Data”. I’ve recently written a review of this in Elsevier’s Serials Review and it’s coming out RSN in a special issue on Open Access. It’s already on Nature Precedings. So if you want detailed aspects – a few months out of date, they are there.
Some bullet points:

• Data are different from text. Open Access generally does not support data well (I make exceptions for ultra-strong-OA such as CC-BY and BBB-compliant. Of the sort that PLoS and BMC provide. Green Open Access is irrelevant to Open Data (I think it makes it harder, others disagree).
• Data matter. Chemistry is a data-rich science. We throw away over 90% of our data. We are all part of the problem, but publishers are one of the worst places for data loss.
• Data must be made available by the authors. It’s now simple to do this. There is no technical excuse for not publishing chemistry data.
• Data must be Open. It’s that simple. It can be done independently of Open Access
• Data should be semantic. That’s harder but it’s happening. In our group we are producing the next generation of semantic tools for chemistry. I had hoped to announce a very important new project but the legal details haven’t been completely signed off. You’ll see it first on this blog.
• Graduate students can – if they wish – provide semantic theses that can be checked and enhanced by machines. Free of many common errors.

What of the future? I’m not going to talk about business models or the rights and wrong of Junk Science through government mandates. But I should make it clear that:

• Closed access is harmful to chemical data. That’s a fact, not a political stance. We are 10+ years behind other data-rich sciences because we protect data in archaic silos.
• Publishers have to choose, one way of the other. “Mumble” is no good. Either you are an enthusiastic publisher of Open Data or you are a closed publisher. Your choice.
• The formal aggregators (Chemical Abstracts, Inorganic Crystal Structure Database, Cambridge Crystallographic Data Base) will see their market and importance steadily decline. I predict that in 5 years’ time there will be no role for ICSD in its current form. The CSD may follow. Chem Abs will survive, but in form marginalised from the main web.Unfortunately at the moment several publishers (Wiley, Elsevier, Springer) do not expose crystallographic data and sent it to the data centres where we have to pay to get it out. This type of restrictive practice harms chemistry – I shall show how – and will be increasingly difficult to defend. Unfortunately when I write to these publishers they simply don’t reply.

So that’s what I intend to say. Roughly. And show some demos of the publication of the future. The datument.
Enthusiastic publishers can make it happen and chemistry is the best subject. Or you can delay it. You’ll succeed in delaying it, biut eventually it will happen.

Yesterday I received a request from a publisher. I won’t name them but I don’t think the material is sensitive, and I need your help anyway. It’s very simple

We're in the process of producing new [High School] Chemistry
teaching and learning materials
We are looking at producing a number of rotatable
molecular models as part of our digital publishing offer.
[We would like someone] who can write CML files for us.

The molecules are apparently simple:

• neopentane and 4 other simple organic molecules

• buckminsterfullerene

• diamond

• graphite

• sodium

• sodium chloride

The organic molecules are easy (the publisher wasn’t concerned about what conformation was provided). I actually used ChemAxon’s Marvin – which emits CML – and it took about 5 minutes. So thank you ChemAxon, who make the software available for free, though it’s not Open.
In the near future it will be different. We shall have extracted all the molecules from CrystalEye and put them in an Open Repository. We shall have added the 250,000 molecules from the NCI which we computed with MOPAC and so it’s certain we would find the molecules we wanted in there. Moreover I would expect that the Blue Obelisk will soon have a complete workflow for drawing molecules and creating 3D structures.
But where can I get a 3D structure of sodium chloride? No, it’s not in Wikipedia (which is an encyclopedia, not a data- or knowledge-base). How long would it take you? And, before you think it’s simple remember it’s for commercial use. You will have to negotiate with the supplier of the information to determine whether you are allowed to redistribute the derivative work. Yes, it’s only data and data shouldn’t be copyright, should it?
So my question to the blogosphere is:

“where can I get redistributable coordinates for the last 5 substances? and how long did it take you to get them and assure me that they can be re-used for commercial purposes”

I expect buckminsterfullerene to be fairly easy. I think I have an answer for the solids. It took me about 10 minutes of half-remembered browsing by a strange route. I’ll accept coordinates in CIF, PDB or CML (no other format I know of supports crystallography and we need the space group and cell dimensions). As a second best I’d accept a filled-out unit cell with Cartesian coordinates. But the coordinates aren’t the problem. Finding the structures is. Please reply and tell me how long it took. Remember I’m not interested in pictures, only coordinates. And I didn’t get any joy with the ICSD database of inorganic crystal structures (http://icsd.ccp14.ac.uk/icsd/icsd_help.html). I may not have navigated through the complex interface but I only got:

Access forbidden!

You don’t have permission to access the requested object. It is either read-protected or not readable by the server.

The sodium chloride was determined about 100 years years ago. It’s in the public domain. But where can I get it?
The publisher has offered me a fee. I don’t know how much, but I will suggest they donate it to support education in Africa unless anyone has a better idea.

Open Visualization Workshop this Saturday in London (UK). Not sure what time, but it will be clarified, I am sure.

Just to confirm that the most popular date for the Open Visualisation
Workshop voted on at our Doodle page [1] is this coming Saturday 24th
May. The event will be an informal, hands-on workshop focusing on open
source visualisation technologies. More details are at:
http://okfn.org/wiki/OpenVisualisation/Workshop
Trampoline Systems have kindly offered to host the first workshop:
Trampoline Systems
8-15 Dereham Place
London
EC2A 3HJ
United Kingdom

Open Visualisation Workshop

Visual representation is a time-tested way of making large, complex bodies of information manageable – whether in the form of maps, timelines, graphs, or charts. Emerging digital technologies have revolutionised what is possible in this domain.

This is an informal, hands-on workshop for those who work with, or are interested in, open-source visualisation technologies, whether based on the web or on the desktop, whether for data or other kinds of knowledge.

Participating

If you are interested in participating, please:

1. add your name to the list below;

2. specify which dates in May you are free on the event’s doodle page;

3. join the ok-london-announce and the open-visualisation list for further details.

List of participants

• Jonathan Gray, Open Knowledge Foundation
• Ed Murfitt, RCA-IDE
• Jonathan Lowe, Giswebsite LLP
• Rufus Pollock, Open Knowledge Foundation
• Saul Albert, The People Speak

• Charles Armstrong, Trampoline Systems

• David Aanensen, Imperial College London
• Peter Murray-Rust, Cambridge
• Julie Tolmie, KCL
• Joanne Harrison, Univ. of East London

• Nils Gehlenborg, European Bioinformatics Institute

• Gregory Jordan, European Bioinformatics Institute

• DeborahMacPherson, Accuracy&Aesthetics

Activities

Feel free to add to the following list of things you would like to see on the day:

• Examples and use cases of visualisation in different fields

• Demos, walkthroughs, testing and comparing of different open-source visualisation software packages (see list on Open Visualisation page)

• Client-side visualisation (SVG?)

Publicity

Please post here if you publicise the event to avoid doubling up!

• Sourceforge Prefuse forum, 2008-04-17

• Processing forum, 2008-04-17

• Nature Network group “Visualization and Science”, 2008-05-02

I’ll try to post at least twice in what I shall say at RSC on Thursday. (FWIW at least 2 readers have recently applied to go to the meeting – I should have started blogging earlier…)
The posts look to a positive future based on the ready technological ability of high-quality chemical data as a result of the information and instrumentation revolution. For those who espouse this view there’s a great future. For those who traditionally have seen data as something hard-won and chargeable there are turbulent times. The reality is that they will have to change. Not because I tell them so, but because the world does. Not explicitly, but in the relentless change in the way we do things – credit cards vs banknotes, mobile phones over landlines (they are knocking down the iconic red telephone boxes in UK) – you know the list as well as me.
So the model of “you publish your data in fragmented form; we type it up and sell it back to the community” is now no longer necessary or viable. We’ve seen gigabillion companies flourish in the last 10 years based on openly available data. (Yes we have concern about some of the Openness but that’s another post). There isn’t a market for micropayments for scientific information. (I exclude 40 USD to read a paper as it is hardly “micro” -it’s quite a good meal in some places)
When I started scientific research data were hard to come by. I was a crystallographer and I’m going to use that discipline as I understand it and also as it contrasts different approaches well. And it has exceptionally high-quality data.
To solve a crystal structure I had to record the diffraction pattern. This was done using methods developed by Karl Weissenberg (I was appalled that I couldn’t find a Wikipedia article). The diffraction pattern looks like some here. (or search Google for “Weissenberg photograph”). I solved 6 crystal structures for my doctorate and each might produce several thousand “spots”. The spots were of different intensity and this could be used to determine the positions of the atoms in the structure – incredible, even now. The intensities were meaured by eye (usually compared to a calibrated scale). So I measured some tens of thousands of spots.
This was the raw data. It mattered. It was hard-won, every spot hand-recorded in the lab book. Then each was typed (on a teletype) onto punched tape and fed into a Mercury computer (later the KDF9).  When the strucrure was published every spot was published in the journal. It took pages but the journals and editors required them.
Why? Because they were the primary record of the experiment. They were the proof that you had made the right deductions about the atomic positions. Over-ambitious or sloppy claims were regularly demolished by  self-appointed critics such as Jerry Donohue, and Richard Marsh (and several others). If they saw a structure they didn’t feel was correct they would re-type the data and re-analyse the structure. No-one likes being corrected in public print but we all accepted it was completely appropriate. It’s that public criticism which has helped to keep crystallography at the top of data quality.
There’s a complementary aspect. As science evolves it’s often possible to re-analyze existing data. So, for example, many Weissenberg photographs recorded so-called anomalous dispersion which can be used to determine the cirality of molecules. In many cases the effect was clear enough to observe in retrospect but the authors weren’t aware of the phenomenon. It would be possible to revist the data and re-analyse. Similarly the advance of theoretical methods and programs in crystallography means that more effects can be corrected or analysed in the computation. If I revisited my DPhil data I could use anistropic refeinement to get imporived coordinates. It’s probably not worth it – cheaper to re-synthesize and re-collect the data. But it’s possible.
Then the journals started to worry about page space. These intensities took up a lot of space. So they were no longer published (although they might be microfiched). And then even the final coordinates were no longer published – the only way you could get them was by requesting microfiches from the publishers.
And this culture has often stayed with us. There is now no technical reason why all the scientific data shouldn’t be part of the publication. But it usually isn’t. And that leads to several consequences:

• Loss of data
• A market in data

I’ll address these later.

2-3 people have mailed me to say that their comments don’t get through. This may be due to “upgrades” in WordPress which we made a week or two back.
If your comment doesn’t appear, please mail me. I never fail to post comments unless they are obvious spam.
P.