petermr's blog

I mentioned yesterday that we had been funded by JISC to develop a departmental repository – starting from C3DER (crystallography) and expanding to spectroscopy and chemical syntheses. We shall be working with a commercial supplier of Electronic Lab Notebooks in a tightly coupled project where both will benefit from the synergy – we shall have the use of a robust platform and can add on many of the innovations we’ve been developing here, which should then get a wider currency. We think this is a new and exciting way of exploring the next generation of chemical informatics which will be semantic, enhanced and guided by ontologies.

The vendor has not been selected so I am keeping my mouth shut…

CLARION project – Cambridge Chemistry Department

 

The data challenge: Chemistry laboratories produce many types of information and data – raw data, processed data, observations, chemical structures, reaction schemes, experimental write-ups, conclusions, graphs, images, crystallographic, spectroscopy data, papers, references, and so on.  It is challenging to store this variety of information such that it is accessible and usable by a variety of users.  The challenges include:

 

Storing data in formats that allow its use by specialist data processing tools

Using data formats that are suitable for publication and long-term preservation

Allowing certain data to be used by people outside the department

Motivating researchers to open their data

Enhancing the meaning and context of the data to improve its usability

Making the data searchable and easily navigable

Ensuring that the system has minimal support overheads, yet continually evolves as required to meet changes in the IT environment.

 

Using an ELN:  The Cambridge Chemistry Department has a basic repository which stores crystallographic data.  Project CLARION (Cambridge Laboratory Repository In/Organic Notebooks) will create an enhanced repository that captures core types of chemistry data and ensures their access and preservation.  The Chemistry Department is implementing a commercial Electronic Laboratory Notebook (ELN) system; CLARION will work closely with the ELN team to create a system for ingesting chemistry data directly into the repository with minimum effort by the researcher.

 

Enhancing and expanding data usage:  CLARION will provide functionality to enable scientists to make selected data available as Open Data for use by people external to the department.  The project will use techniques for adding semantic definition to chemical data, including RDF (Resource Description Framework) and CML (Chemical Markup Language).  Much of these techniques will be extensible to other disciplines.  CLARION will address general issues such as ownership of data, and it will publicise its results to the chemistry and repositories communities.  Effort will be put into developing a sustainable business model for operating the repository that can be adopted by the department after project completion.

 

Timelines: The project runs for two years from April 2009. The initial pilot deployment of the ELN is scheduled for late 2009, and we hope to be publishing open data from it in early 2010.

 

Project blog:                http://clarionproject.wordpress.com/

Twitter:                       CLARIONproject    http://twitter.com/CLARIONproject

Contact:                      “Brian Brooks” <bjb45@cam.ac.uk>

Plenary – Trends from the Trenches Chris Dagdigian, BioTeam, Inc.
Some brief notes from plenaries
What’s Mainstream:

virtualization, partly because of power requirements. The simplests and most powerful thing you can do. Protects the web apps, databases that are “lashed up”. Valid use case, but not enterprise. So virtualization allows enterprise-like environment preserves this innovation without danger. Vital for science

not coming soon – Vms for Grids and Clusters. Too much admin hassle

Storage – first 100TB single namespace project. Jobs lost over data loss. Data triage is a given. Examples Single namespace for Mac has 80 TB, 1.1 PB on Linux system

Users have no idea of true cost of storage. $124 for 1TB fort hardware is misleading. Individual labs put in 100Tb+ systems

Unlimited data storage days are over. – need triage. Cheaper to repeat experiment than keep data

Data loss – exemple – double disk failure in metadata – 10 TB in goverment lab. You will get double disk failures. Need RAID6

Backup is becoming a thing of the past, no “nightly full”.

Amazon, Google MS can store for 80cent / GB / year. Can you do that??

IT cannot be sole decision maker for triage or for storgae optimization

Rate limits are chemistry, regagent costs and human factors

Proeblem is somewhat scary but most people surviving

Amazon is is the cloud – has mutli-year headstart

Security in the cloud – don’t expect things that you don’t provide. Objections are often political

Compute power is easier than IO. He believes that Amazon are working on data ingestion.

Will be big move of science data into storage cloud. Science data will take 1-way trip. Data will stay in cloud. Only derived data will return.

McKinsey report on Cloud Computing very good, also James Hamilton

Watch Amazon, Google and MS.
Best data practices are starting to trickle out. Google is now showing what it did 5 years ago so they must be up to very exciting things now
Finally – federated data storage – something for the future

My talk is “Open Semantic Data in Science”. I’ll probably write 3-4 blog posts on the various aspects of this, and at present I’m thinking of:

• What is Open? (this post)
• What is semantic? And what do we require for it?
• What is data?
• What are we able to offer (with some modest emphasis on our own endeavours).

I am starting with the assumption that for science now and in the future Open Data will be essential. The culture, especially among young people, is that “the answer is out there” and is retrievable within seconds or less. There’s also a realisation that increasingly we don’t know in detail what we are looking for when we start a study. We read bits of papers, skim around till we get a feel for the subject, ask our colleagues, post questions on blogs, etc.

We’re also using machines much more to help us with the data, both in the volume and the diversity. This is a central theme at BioIT. So the fundamental postulate of Openness is:

• ANY barrier to access and re-use, however small and seemingly trivial COMPLETELY destroys public semantic data.

(Note that I accept that there are closed worlds – companies, healthcare, etc. which require access controls, but their technology can feed off what we are trying to create in public view).

Why am I so insistent on this? I’ll leave the moral and ethical arguments aside here and concentrate on the technical aspects. The Open Knowledge Foundation has addressed this point in its definition and I’ll quote from that highlighting particular points (and abbreviating occasionally)

A work is open if its manner of distribution satisfies the following conditions:

• 1. Access

The work shall be available as a whole …, preferably downloading via the Internet without charge. The work must also be available in a convenient and modifiable form.
Comment: This can be summarized as ‘social’ openness – not only are you allowed to get the work but you can get it. ‘As a whole’ prevents the limitation of access by indirect means, for example by only allowing access to a few items of a database at a time.

• . Redistribution

The license shall not restrict any party from selling or giving away the work either on its own or as part of a package made from works from many different sources. The license shall not require a royalty or other fee for such sale or distribution.

• . Reuse

The license must allow for modifications and derivative works and must allow them to be distributed under the terms of the original work. The license may impose some form of attribution and integrity requirements: see principle 5 (Attribution) and principle 6 (Integrity) below.
Comment: Note that this clause does not prevent the use of ‘viral’ or share-alike licenses that require redistribution of modifications under the same terms as the original.

• . Absence of Technological Restriction

The work must be provided in such a form that there are no technological obstacles to the performance of the above activities. This can be achieved by the provision of the work in an open data format, …

• 5. Attribution

The license may require as a condition for redistribution and re-use the attribution of the contributors and creators to the work. …

• 6. Integrity

The license may require as a condition for the work being distributed in modified form that the resulting work carry a different name or version number from the original work.

• 7. No Discrimination Against Persons or Groups

• 8. No Discrimination Against Fields of Endeavor

The license must not restrict anyone from making use of the work in a specific field of endeavor. For example, it may not restrict the work from being used in a business, or from being used for military research.
Comment: The major intention of this clause is to prohibit license traps that prevent open source from being used commercially. We want commercial users to join our community, not feel excluded from it.
9. Distribution of License
The rights attached to the work must apply to all to whom the work is redistributed without the need for execution of an additional license by those parties.
10. License Must Not Be Specific to a Package
…
11. License Must Not Restrict the Distribution of Other Works
…
and now the absolute requirement for Openness.

NONE OF THE ABOVE CONDITIONS ARE OPTIONAL

• This is the crux. There are many data resources which are described as “Open” but they fail in one or more aspects. The commonest failures are:
• to expose only part of the data. A database system with a query interface is normally not Open Data even if individual items can be downloaded without barrier. It is generally impossible to extract the whoel work as its boundaries are concealed by the search interface
• to limit the amount downloaded. This is very frequent (“you may use a maximum of 100 entries”).
• To forbid re-use. “This data is copyright X and may not be re-used without permission”)
• To require access through specific technology. A search form limits the access.
• To require any form of signin, even if free. Robots are illiterate in this aspect
• To restrict purpose of re-use. Thus CC-NC (“no commercial reuse”) is NOT OKF-compliant
• To fail to provide a clear statement that the data are open and comply with the Open Knowledge definition. It’s almost universal that data are NOT labelled as Open. This is easy to fix – just add the OKF’s tags

• 

• So the message is simple, though it will take time to spread
• Use the OKF definition for all your data and tag it as such

This blog authored with ICE + Open Office; thanks to PeterSefton and USQ

I am talking on Wednesday at 2009 Bio-IT World Conference which “unites life sciences, pharmaceutical, clinical, healthcare, and IT professionals. It is the perfect place to learn, be recognized and network”. Well I hope I can be recognized as I’m meeting Steve Heller at 1600 and Antony Williams tomorrow. And networking is good – they have wireless everywhere which is great relief as so many of these conferences have no wireless or charge zillions per day. And I’v found a free lunch (again many of these places are really mean, but you can often sneak in at the back to special functions.

So what shall I say? I was going to talk about the Chemical Semantic Web, but then looked at my program and found I had offered:

Open Semantic Data in Science

which is much better because (a) a Pfizer article in the conference magazine says the SW doesn’t exist, but Semantic Data does and (b) it will fit well into the revised program. Antony Williams is now just before me and Rajarshi Guha is just afterwards. So we shall try to meet beforehand and see if we can make a seamless program.

The conference asked us to upload slides a month before – I argued that my style of presentation didn’t use slides and anyway I didn’t know what I was going to say. I therefore generally work it out the day or night beforehand. This isn’t because I am casual about it – I take my presentations seriously and put a lot of work into them – but a lot of this goes on in my head in the preceding days. I don’t recommend this to others – and in Cambridge we require colleagues to present “dry runs” a week beforehand.

But the main thing is that what matters to me is what I say rather than what I write. I try to adjust my words to the actual audience – there could be 10 or 100 people in this session – I have no idea. I often use a blog to work my ideas out and I’ll do that here. The blog serves as a record and – now that I have ICE as an authoring tool – I should be able to add images to the blog. Here goes…

[Question – what is the name of the panda? Who is his famous companion ? And if you didn’t know how did you find out]

So, yes, images seem to work – great! So I can start to use blogs with things other than words.

So how can this technology compete with the awful Powerpoint? Its main virtue – and it’s an important one – is that it has a good editing tool and it acts as a container for various types of content. Not a robust one, as anyone who has transferred it from or to Mac or Unix knows.

So what alternative is there?

I suggest Word or Open Office. Ideally I’d like to use this for the complete presentation but I have got fouled up with the page flips. I like my HTML presentations – I like the ability to scroll. The problem is packing them together afterwards (I can’t do this beforehand as I don’t know what slides I will use).

So my current approach is to blog my main message before the presentation. By doing this with ICE I can author the blog as ODT. Then I can convert this to DOCX and upload it to the conference site. I’ve talked about this with Peter Sefton and we’re thinking about a way to manage some of this.

Ideally I’d like to be able to record which HTML slides I showed (I assume this requires Javascript). Then it would be useful to combine this into a single ODT/Word/HTML/PDF document for the later reader. That’s exactly what Peter does for the courseware at USQ.

So I’m blogging with ODT and will upload several blogs of my talk during the meeting.

I don’t know whether they will make sense, but at least they’ll me more semantic than Powerpoint.

This blog authored with ICE + Open Office; thanks to PeterSefton and USQ

I am learning more about the proposed (and terrifying) control of the European Internet by vested interests through ISP control. Since I’ve only known about this for 1 day forgive errors in the learning process..

Here’s a useful, recent, post explaining it.

In simple terms, if you infringe copyright three times you can be cut off from the Internet by your ISP. I don’t know for how long. Probably ever. Or at least as long as the foul-ups on credit ratings and gas bills to people with no gas.

This is the modern equivalent of having a hand cut off for stealing, or being transported to Australia.

So what’s the problem? None of us infringe copyright do we? We sit through those awful adverts from DVD rentals “you wouldn’t steal from a pensioner” – so “don’t infringe our copyright”.

I try to abide by copyright. But it’s technically hard. The copyright on many works is highly dubious. Anyone can add copyright notices to the public domain with a very good chance of making it stick. Publishers such as Wiley and ACS require their copyright to be added to supplemental scientific data. And they – see the Shelley Batts case (Sued for 10 Data Points) – pursue those who copy this.

So it’s entirely possible that if I copy a graph or table of numbers or a spectrum from a Wiley paper I will be “infringing ‘their’ copyright”. Now all they have to do is tell the ISP to cut me off. I will have to argue with an ISP who has no knowledge or interest in the rights and wrongs and simply sees “Copyright Wiley” stamped on the document.

And it will get worse. Just like universal video cameras the front-line policing of the Internet could be through ISPs.

I will be a second-class citizen for the rest of my life.

Glyn Moody has confirmed that there is a real threat to the European Internet:

Glyn Moody says:

April 25, 2009 at 4:14 pm (Edit)

Yes, this is very serious. It’s threatening Net neutrality – which means that bits are passed end to end without caring what they are. Telecoms companies want to be able to block certain kinds of traffic, or charge more (for Skype, for example). This is likely to kill innovation.

Here’s my commentary from a few weeks back, together with the letter I wrote to my MEPs. There’s also a link to what Sir Tim Berners-Lee said on Net neutrality, and why it matters:

http://opendotdotdot.blogspot.com/2009/03/save-european-internet-write-to-your_30.html

Feel free to contact me with questions.

I have therefore written to my member of European Parliament (MEP)

Dear Andrew Duff,

I am writing to urge you to vote to keep Europe’s Internet free when the issue is voted on May 5th.

I am sure you are aware of the issues, which are summarised on http://www.blackouteurope.eu/.

I am a chemist in the University of Cambridge, deeply involved with the Digital Information Environment and have been supported by grants from the UK’s eScience program (DTI/EPSRC)  and the JISC (www.jisc.ac.uk) to develop new radically new information systems for scientists. The Internet and the new generation of Web activity (Sir Tim Berners-Lee’s Semantic Web) are a revolution in human history and will bring major advances and be a major generator of wealth. This will be critical in pulling the world out of recession.

Cambridge and the East of England are leaders in this revolution, which depends on uncontrolled and ultra-rapid innovation. This innovation depends absolutely on the free flow of information, typified by TimBL’s “Linked Open Data” where different information sources are “mashed up”  through software. Often these systems are created by single, inspired, programmers with nothing more than a PC connected to the Internet. Any barriers, however small, destroy this innovation completely. That is why many of us campaign for the removal of barriers such as software patents, DRM and copyright that restrict the digital revolution. For example, the Open Knowledge Foundation (http://www.okfn.org), which started in Cambridge, works to create the instruments and the advocacy for unrestricted knowledge. Any control over the flow of digital information will have major adverse effects on the innovation we are now seeing.

Besides the economic argument, the proposed changes will have a very serious impact on digital democracy. Again Cambridge has been in the forefront of developing new web-based democracy such as seen in mySociety (http://www.mysociety.org) with products such as TheyWorkForYou (http://www.mysociety.org/projects/theyworkforyou/) which gives immediate and comprehensive digital access to information on Westminster MP’s records and activities. (Some of their work is supported by the Cabinet Office).

The future of our planet will also depend on access to data and its unrestricted use. A typical example is AMEE (http://www.amee.com/), again a single person initiative, which produces carbon and energy calculators which are now used by major multi-nationals to unify the reporting of carbon footprints.

The European debate is characterised by large multinationals with powerful lobbies arguing for wealth creation through restriction and monopolies. This is backwards looking and slow-moving and economies and governments who are governed by this will be rapidly overtaken by those, like the new US administration, who espouse Openness as a primary approach.

Yours

Peter Murray-Rust

This blogpost was prepared with ICE+OpenOffice.

I got the following mail yesterday which alerts us to a potential threat to our Electronic Freedom. I’d value a reality check. I have checked on Twitter and got a little feedback – it’s a real issue. But – like all Euro stuff – I am sure it’s complex. I haven’t seen anything on EFF, for example. (EFF were active in fighting European Software Patents).

So – should I write to my MEP – before May 5^th? – and if so with what message?

Do NOT Touch The Internet
Citizens Ready For A Fight Across The European Union.

The privatization of the Internet could be decided in the next few weeks. On May 5th, the European Parliament will vote on a package of measures which will affect the national laws of all EU countries.

The European Parliament is about to give up our rights to an open access to the Internet to protect the interests of Entertainment and Communications multinationals. A civil campaign by organizations of the whole European Community will be simultaneously launched tonight in order to prevent the privatization of the Internet and to defend the democratic right to access to information and digital tools.

This campaign includes:
1 – An informative website explaining what the so called ” Telecommunications Package “, which is about to be voted on in the European Parliament, is, why it is so dangerous for the future of citizens. the impact on their daily economy, as well as useful information and tools created by citizens for citizens, including open letters, press releases, a blog and video material:
http: // www.blackouteurope.eu/ 

2 – A letter to members of the European Parliament that it is being sent by hundreds of organizations and citizens (one Euro Parliamentarian recognized that they are receiving as many as 200 letters a day):
http://www.blackouteurope.eu/act/letter_to_meps.html

3 – An automatic system that allows citizens to send the letter directly to all the Members of the European Parliament:
http://blackouteu.wordpress.com/directly-send-to-parlamentarians/

4 – A system to monitor their vote in the matter:
http://blackouteu.wordpress.com/follow-their-vote/

And

5 – A Europe wide casting for “grandmothers” to create an inter-European viral video:
Fortheinternet.wordpress.com/

This blogpost was prepared with ICE+OpenOffice.