I’m at OR08 – normally I would try to blog the meeting but (a) I am recovering from my preentation and (b) I’m off to NL to defend Egon Willighagen’s thesis. So these posts will be bitty…
I was honoured to be asked to kick of the meeting and chose the title above. In my presentations I generally don’t know what I am going to say in detail – I have a menu of several hundred slides and choose on-the-fly, depending in part on the audience (their backgrounds, interests, etc.) . This occasion took this to new heights.
I’d prepared a number of interactive demos – OSCAR, OSCAR3, videos from JoVE: Journal of Visualized Experiments , Eclipse Unit tests, and a movie from Andrew Walkingshaw. At least three of these have never been shown in public, so this was fairly ambitious.
Andrew had prepared his movie on MAC (and bought the pukka Apple movie generator). He mailed it to me – I downloaded QuickTime and iTunes on Vista. It showed a movie of polar bears in a think snowstorm (i.e. white on white). No matter what I and Jim did in the small hours of the morning we couldn’t get it to run. So we planned to run it on Jim’s machine and switch the video when necessary.
Then I found out that because the conference was oversubscribed there was an overflow room. So the presentations were geared to use Powerpoint running on a dedicated Mac….
Um… So the Soton techies (who did a great job) linked my output not through the VGA but through the VPN on an RJ45. Linked through a VPN to the other room.
Years of presenting have thrown up some fun situations. Henry Rzepa and I created chemical presentations (Mage, RaMOL) for WWW1 at CERN in 1994. We prepared it two days in advance. Then, before the meeting, someone deleted the shared libraries (*.so) to “save space”. We got the demo working at – I think – T+5 (i.e. 5 mins after Henry had started presenting). Another when stewardess spilt coffee over my laptop – it recovered except that the semicolon key crashed the machine and to type an “a” it had to be cut and pasted from existing text.
You can see what is coming. The complexity of the system – movie -> Vista -> VPN -> Mac -> Screen bombed. We abandoned the VPN (in real time) and rebooted. When Vista reboots it is nice and leisurely so gives natural spaces for me to fill in. When it finally came up there were two displays – one on my machine and one on the screen. But they were different. So I had to drive the talk from the screen …
I have no idea what I said, so I can’t blog it. I hope I got most of the messages over. I’ll try to remember some of the things I’d like to have said and blog this later.
But it was fun for April 1…

From Cameron Neylon:

Semantics in the real world? Part I – Why the triple needs to be a quint (or a sext, or…)

… This definitely comes with a health warning as it goes way beyond what I know much about at any technical level. This is therefore handwaving of the highest order. But I haven’t come across anyone else floating the same ideas so I will have a shot at explaning my thoughts.

The Semantic Web, RDF, and XML are all the product of computer scientists thinking about computers and information. You can tell this because they deal with straightforward declarations that are absolute. X has property Y. Putting aside all the issues with the availability of tools and applications, the fact that triple stores don’t scale well, regardless of all the technical problems a central issue with applying these types of strategy to the real world is that absolutes don’t exist. I may assert that X has property Y, but what hppens when I change my mind, or when I realise I made a mistake, or when I find out that the underlying data wasn’t taken properly. How do we get this to work in the real world?
[… lots more – on provenance, probability, etc. snipped …]

PMR: In essence Cameron outlines the frustration that many of us find with the RDF model. It makes categorical assertions which have 100% weight and – in its default form – are unattributed. Here are three assertions:

• The formula of water is H2O
• The formula of snow is C17H21NO4
• Snow is frozen water

Assuming I have the implicit semantics that frezzing does not change the chemical nature of a substance (not always true), these three statements taken at face value create a contradiction.
I can remove the contradiction by introducing the semantic that a formula may be associated with more than one name and that a name may be associated with more than one formula. This taken at face value prevents us from making any useful inferences.
What I have felt a great need for (echoing Cameron) is that the triple should be enhanced with two properties:

• the provenance (the person or software making the assertion)
• the weight of the assertion

“At Dagstuhl it is continuing to snow.”
If I pass this sentence to OSCAR it may mark up snow as a chemical substance. In doing so it now gives every annotation a weight based on the confidence (I shan’t explain how here). So, for example, it is much more likely that 2-acetylfoobarane is a chemical than HIV (hydrogen-vanadium-iodide) and OSCAR addresses these concerns.
It’s possible to add provenance and confidence to RDF but I don’t know of a standard approach for doing this. If we start doing this we need to make sure we have consistent schemas.
(Interestingly we’ve just been discussing the value of adding “strength of statement” to the results of text mining.

From Peter Suber’s blog:

The high cost of the lack of open data

14:43 25/03/2008, Peter Suber, Open Access News

The Value of Spatial Information, a report by ACIL Tasman prepared for Australia’s Cooperative Research Centre for Spatial Information and ANZLIC, March 2008.  (Thanks to Baden Appleyard.)  From the executive summary:

…Constraints on access to data are estimated to have reduced the direct productivity impacts in certain sectors by between 5% and 15%. It is estimated that this could have resulted in GDP and consumption being around 7% lower in 2006-07 (around $0.5 billion) than it might otherwise have been….

Comment.  These are big numbers and it takes a minute to put them in perspective.  In one country (Australia) in one year (2006-07), lack of OA to one kind of data (spatial data) cost the economy $500,000,000.

PMR: I hardly need to comment. However in our current discussions at Dagstuhl on Text Mining and Ontologies in the Life Sciences it is clear how valuable Open Data is. It’s also clear how much the lack of open data in chemistry holds back innovation. I don’t have numbers, but it would be great to have an economist look at this…

A brief update. I’m privileged to have been invited to a meeting at Dagstuhl in Germany. It’s on Text Mining and Ontologies (I expect that we shall all post abstracts over the next few days). It’s a heavyish program and – rightly – wireless is forbidden in the seminar room so there won’t be many posts. I’m applying Chatham House rule although you can see the participants on the web.
The discussion today included

•  whether ontologies modelled reality
• can we create ontologies from text-mining (general answer no, except in limited cases, but it may be a useful help)
• whether ontologies should always be created by domain experts (generally yes – any contracted-out ontology is garbage).

I presented our group’s work today – OSCAR is now well known and being used elsewhere. It’s nice to be in an area where software and resources are freely available. My optimism level about free knowledge in science has risen.
Also some useful unattributed conversations about repositories – the computer scientists are not impressed with DSpace etc. and the resources spent by universities. I’m gathering ideas for my presentation at OR08 next week on Data Repositories. I am gearing up to generate lively discussion.

Catalysed by a recent comment on a 2007-12 post (Exploring RDF and CML) :

1. Tim Berners-Lee Says:
   March 21st, 2008 at 1:09 am ePeter,Sounds exciting!Do you have any public RDF molecule data others could explore? URIs?Tim BL

here’s an update of where we are at with molecular repositories. (We shall have a clearer idea when several of our group present at Open Repositories 2008 (OR08) in 10 days time. (Lots of progress can be made in 10 days). I’m omitting details here (so as not to spoil the show next month).

• We are committed to RDF+XML/CML as the future for molecular information. This is the only way that we can manage such diverse information as documents, recipes, results of calculations, spectra, crystallography, physical and analytical properties, etc. The CML schema is now being used in many places and has remained stable for 15 months. Almost all parts have now been tested in the field (the main exception is isotopic variation – e.g. in geoscience). We can easily go from CML to RDF – the reverse is not always possible. The value of CML is that it is currently easier to use for chemical calculations as there is a knowable coherence of related concepts. Note that the CML community is developing a number of subdomains (“conventions”) which allows some degree of local autonomy as in CMLComp.
• We are enthusiastic partners in the OREChem project (Chemistry Repositories, and from Jim Downing ORE! Unh! Huh! What is it good for?). This uses named (RDF) graphs to describe local collections (“Aggregates”) of URIs. The project will have several molecular repositories of which we shall contribute at least 2 (CrystalEye and our neascent “molecular repository”). All content will be Open Data.
• Jim Downing has developed a lightweight repository (MMRe) based on Atom/REST and RDF. I won’t give too much away except to say it is deployed and over the last few days Joe Townsend has been adding data from chemistry theses (SPECTRaT) and Lezan Hawizy has been adding our collection of “common molecules” (scraped from various sources). This can now be queried through SPARQL.
• Andrew Walkingshaw has converted the CML in CrystalEye to RDF – 100,000 entries and probably about 10 million triples. He’s been working with a well-known semantic web company (not sure if this is public yet) and has done some very exciting extraction and mashups. SPARQL searches work over this size. Andrew has also developed Golem – a system which extracts dictionary links (cml:@dictRef) from CML computations and is able to build dictionaries (ontologies) automatically and then to extract data.
• In the last four days Thomas Steinke has converted VAMP to emit CML. We have run a few hundred calculations automatically (by extracting molecules from the NMREye repository, converting them to input, running the calculation, and then converting to RDF). The results – which contain coordinates, energies and NMR peaks – are being fed into another local repository.

So we have a variety of sources which will all be available. We face a number of exciting questions.

• How do we express a molecule in RDF? We are gradually converging on an “aggregate” where a molecule has identifiers, properties, and special resources such as chemical formula, the CML connection table, and a list ofg chemical names.
• How do we assign identifiers. This is a really hard problem. Although for many chemicals there is little doubt about the relationship between names, identities and properties there cannot, in general, be a “correct” structure or a “unique URI” for a chemical. Look, for example, at “Phosphorus Pentoxide” (In WP). Experiment shows that there are several different forms, with different chemical connectivities. There are 2 formulae (P2O5 and P4O10) each with a different CAS number (Chemical Abstracts is a major authority in chemistry). Are these different chemicals or do they represent our changing chemical knowledge? Is one used for early publications and another for later ones? Only CAS can say when one number is used and not the other. It is because of this uncertainty that we cannot know exactly how many different chemicals there are in the CAS collection.

There cannot be a platonic semantic description of chemical identity – many chemicals do not have a “correct” structure. Antony Williams has been doing a heroic and valuable job in detecting inconsistencies in reporting chemical structure and resolving them where possible (eMolecules and ChemSpider – A Respectful Comparison of Capabilities). But he is not establishing a “correct” structure – he is making authoritative statements about the relationship between names, structures and identifiers.
This brings us to why RDF – probably in its quad form (i.e. with provenance) – is important to describe chemical structure.

• Many substances occur in several forms and there is no single structure. We hope that RDF can manage these relationships.
• Many name-to-structure assignments have changed over time as out experimental techniqures become more powerful. Thus the C19 chemists would first write PO5 (atomic weights were not “correct”), then P2O5 and only after X-ray crystallography P4O10. To understand historical chemistry we have to know the relationships used at the time.

We have scraped about 10000 compounds from the web including Wikipedia and have a variety of triples associated with each. There is little overlap of triples – names, CAS, formulae are present or absent. So we now need to use RDF technology to reconcile this information. It’s a complex task and we will probably have to add weights/probabilities to some of the statements – some authorities are less reliable than others.
In the first instance we’ll probably use some of the commonest identifiers to assert identity and that’s the version we should be releasing in a few days.

I have been invited to give a keynote lecture at Open Repositories 2008 (see the programme – about 25% down) and have chosen the title “Repositories for Scientific Data”. I’d value help from the repositarian blogosphere and elsewhere.
My thesis is that the current approach for Instituional Repositories will not translate easily to the capture of scientific data and related research output. In some fields of “big science” (e.g. High Energy Physics) the problem is or will be solved by the community and their funders and institutions have effectively no role. However much – probably most – science is done in labs which are the primary unit of allegiance. Typical disciplines are chemistry , materials science, biochemistry, cell biology, neuroscience, etc. etc. These labs are often focussed on local and short-term issues rather than long-term archival, dissemination of data to the community, etc. Typical worries are:

• My grad student has just left without warning – can I find her spectra?
• How can we rerun the stats that our visitor last year did for us?
•  My laptop has just crashed and I’ve lost all the images from the microscope
• My chosen journal had to retract papers due to recent scientific malpractice. Now they want me to send them all my supporting data to prove I have adopted correct procedures. This will take me an extra month to retype in their format.

If we are to capture and preserve science we have to do it to support the scientist, not because the institution thinks it is a good idea (even it is is a good idea). So we have to embed the data capture directly into the laboratory. Of course in many cases there is a key role for the Department, particularly when – as in chemistry – there is a huge investment in analytical services (crystallography, spectroscopy, computing).
I am developing this theme for the presentation and would be very grateful for anecdotal or other information as to where the institution or department has developed a data capture system which ultimately feeds into medium-term (probably Open) preservation. Two emerging examples are Monash which has acquired a petabyte for storage of University scientific data and will layer a series of access mechanisms (SVN, Active Directory, Samba, RDB, SRB, etc.) on top of it. Recently Oxford has announced a Data Repository.
If you have material that will help give a balanced picture of data reposition in institutions I’d be grateful for email (or comments on the blog but I’ll be offline for a few days from Monday). I’m aware that some disciplines have domain repositories independently of institutions (e.g. HEP, bio-sequences, genes, structures, etc and David Shotton’s image repository for biology) – I’m after cases where the institution has invested in depertamental or lab facilities and which are actually being used.
Many thanks in advance.