petermr's blog

A wonderful launch of Cambridge Open Science / research at the Panton last night. 14 people came which is a wonderful number – enough to get a sense of critical mass. Lovely warm fire, comfy sofas, British beer, projector, screen, internet – what more could we want?

This was driven by Keren Limor who put great energy into the event. Mailing, tweeting, distributing flyers. And she had a great introductory program. Something like:

• Mixer game
• Keren presentation on the issues of Open Access and open research (e.g slides of Mat Todd’s work on Open Drug Discovery).
• PMR talking about the history of OKFN – what I thought of Rufus Pollock in 2004 and why I joined the OKFN then.
• Excellent video from Jonathan Eisen + Nikc Shockey + PHDComics (http://www.youtube.com/watch?v=L5rVH1KGBCY) on Open Access. 5-minutes of must-watch
• Discussion
• Video of Jack Andraka getting Wired award. http://www.wired.co.uk/news/archive/2013-10/17/jack-andraka-wired-2013 . This is sensational.
• Discussion about what we might do next. There’s a great Idea come up… I’ll tweet when more details

Playing the videos was a stroke of genius, Keren. It’s like being at the movies with friends when you have a sense of community, even if you’ve seen them before (which most hadn’t).

A real feeling of community and going forward. The next meeting’s in a month’s time, in the Panton (Keren, can you comment the date/time?) It’s by Fiona Nielsen who’s developed a really great idea (http://dnadigest.org/news/ ) DNA Digest.

Fiona was one of the four of us who planned the group. Everyone can understand What Fiona will talk about, so please all come. (Cambridge is an easy ride from London, as well).

UPDATE from Keren:

Hi,

Thank you Peter!
Here is the link to the event next month: http://www.eventbrite.co.uk/e/would-you-share-your-genome-sequence-tickets-9293969513

what are we going to talk about?
The genomic era is at our doorstep together with a lot of promises to personalized medicine – But what exactly is a genome? Do I have to share it? Do I have to share it all? With whom should I share it? What ethical issues might arise? What are my rights concerning my genome?

For more updates/links/discussion join our facebook group Open Research Cambridge
All the best
Keren

If you found someone writing software which di some of the same sort of things that your software did would you be pleased or upset? If you’re an academic in the modern world you might wellb be upset. “Bugger, we’ve been scooped!” “we’ll have to replicate their functionality”, etc. Because academic success depends on being the first and beating down competition. I have seen so many cases where programs have been rewritten solely for career advancement.

So you might think that when I heard about Tabula 6 months ago I would have been upset. Tabula uses PDFBox (as does our AMI) to turn PDFs into something useful for machines. But I was delighted. Because turning PDFs into semantic form is one of the most soul-destroying activities on earth. There’s no map, a new form of PDF can knowck you back, people don’t understand why we spend this time hacking. It’s lonely.

So first and foremost we welcome Tabula as friends. So let’s see them:

It was then wonderful to meet Mike and Manuel at #MozFest. Mike and Manuel are not concerned about their journal impact factor. They want to make the world a better place.

By hacking PDFs?

Yes! Today’s journalist – e.g. @ProPublica – uses data to find and justify stories. Stories are contained in expense slips, company reports, government spends, etc. The UK MP expenses scandals used crowd-sourcing to analyse zillions of MP’s expense receipts. NHS waste in prescribing non-generic drugs has been highlighted by hacking data. My “crusade” – to liberate factual science from journals – is similar. Most science is destroyed into PDF.

But it can be liberated. So next Wednesday in Oxford (sold out!). I’ll be demonstrating Tabula to get data out of scientific PDFs. It’s immediately understandable and easy to use.

And we face the challenges together. The really horrible aspects of:

• Optical character recognition (words and numbers in diagrams are often bitmaps)
• Recognising table formats – many tables are simply layout for humans
• Restructuring lists
• Analysing graphs

Etc.

So Mike and Manuel and I spent an hour swapping our experiences, making friends. We’re looking at each others’ codes. We’ll try to avoid duplication.

And most importantly, our community has now grown. Growing from 1 to 2 increasing the impact by a factor of 4. It makes our individual efforts more believable. It helps new people join.

I’m going to have to get a better tweetpic…

In about 15 hours from now there will be a thunderclap: https://www.thunderclap.it/en/projects/5675-open-access-button-launch

Joe and David – two medical students are ANGRY.

They’re angry that paywalls to the medical literature still exist. That they can’t read the medical literature that they want. That Jack Andraka, still at school, has to pay PUBLISHERS for “permission to read papers which can save lives.

Joe, David, Jack are the future.

And they are angry about the present. The mess that our generation has bequeathed to them. “Green” and “Gold”, mandates that non-one honours and believes in, repositories that are unfilled and no-one uses. Ten years of failure to implement the BOAI principles of access for everyone.

They want to sweep all this away.

So Joe and David have created a Thunderclap. They’ve created an Open Access Button which tells the world every time someone fails to get past a paywall.

Because (as I said):

“Closed access means people die”.

And they understand the modern world. They are using modern methods to tell people that they are unhappy. A LOT of people. They’ve asked us to support them by donating our Twitter followers to the cause. I’ve done this, with 2000 followers. So have people with many more followers (such as Creative Commons – Yeah!)

They want to get 800,000 – they might even beat that.

When things get unbearable we have to protest. I’ve protested against Nuclear weapons. I’ve been to Greenham Common. And Molesworth. I’ve often stood in Marble Arch protesting until UK removed cruise nuclear missiles. I was among 2 million who told Tony Blair not to invade Iraq. (He didn’t listen to me or anyone else).

I’ve demonstrated electronically against software patent in Brussels.

And the OA Thunderclap is telling the world that we’ve had enough. We’re sick of the prevarication.

Join Joe and David. Donate your Twitter followers. Support them to fight for a new world of Openness. Because what they have is the support of the citizens.

(Here’s some of their original post):

If someone hits a paywall in the forest, does it make a sound?

Every time you hit a paywall is an isolated moment of frustration, that is unlikely to shake the ivory tower of academic publishing. By putting these moments together using the Open Access Button, we will capture your individual moments of injustice and frustration and display them, on full view to the world. Only by making this problem impossible to ignore can we change the system.

This project was started by two students, good friends frustrated by the current system and driven to change the publishing system we will have to work with. The project was made possible by the invaluable support of developers, advocates and the open access community at large.

Our team has worked extremely hard in the past few months to develop a prototype which we’re finally ready to show to the world. At launch, the button will be able track and map every time a user hits a paywall, help them share their struggle and finally help them get access to the paper for free. Advocates can use the stories and data the button collects to push for change. Our data and code will all be available for others to use, improve on and do things we couldn’t have dreamed of.

Everyone is affected by this problem, patients, students, doctors and academics. We need your help to make this problem too obvious to ignore. Please help us. Share this thunderclap, and download the button November 18th

Find out more : http://bit.ly/1bEH7XT

We’re meeting tomorrow at the Panton Arms Cambridge for the first public Open Science/Research OKFN meetup, organised by Keren Limor. (Four of us met at Keren’s house a month ago to plan it). http://www.eventbrite.co.uk/e/cambridge-open-research-tickets-8650372497 The Panton Arms is the second most famous science pub in Cambridge – or am I being too modest?

It’s open to EVERYONE. It’s easy to get to from London (50 mins half-hourly trains and 10mins walk).

Keren’s asked me to introduce it. We have a tradition of Panton Discussions, captured on video, so I’ll bring my video camera along and give a short (promise! Limited by battery) rambling account of the history of OKFN and science in Cambridge. We’ll take a trip to the holy shrine of 37 Panton Street and kiss the doorstep. There’s a nice warm fire

I’ll bring a slide projector and we can (I think) connect to the net so there’s a chance for people to talk about what they are excited about.

The whole idea is to build a community of people in Cambridge interested in Openness and Science. You don’t have to be a scientist! I shall be there from ca 1800 and we expect to sort-of start around 1830. The Panton sells

• Beer
• Soft-drinks
• Acceptable food
• Tomato ketchup and mayonnaise (yum!)
• Chips

This is what the M-R group survived on for 12 years and they are all still alive.

Until this week I and other scholars had been generally forbidden to use machines to read the scientific literature and extract facts (“Text and Data Mining”, TDM or “Content-mining”). The STM publishers had prepared a draft licence which can be summarised as 20 different ways in which “the readers’ machines have no rights”. You’ll remember that we were invited to discuss this in Licences for Europe” and we indicated we didn’t want licences, we wanted rights.

So I felt I had to raise this at UKSG on. In my slides I outlined principles:

“The right to read is the right to mine” and noted that “Unrestricted TDM saves lives”

And recommended that we all make some changes:

• Libraries – reject TDM restrictions
  
• Publishers – Damascene conversion J
  
• Funders – insist on CC-BY
  

(A Damascene conversion is a sudden change of heart from the dark side to the light) see Wikipedia:

So I expected some gentle flak for being unrealistic.

But WOW! Just before my talk Vicky Gardner from Taylor and Francis talked about T+F OA and posted a slide which said TDM was allowed for non-commercial purposes. Gulp! So I asked her. Something like:

PMR: Does TF allow me to mine your non-OA content?

VG: Yes

PMR: The subscription material??

VG Yes

PMR: Wow! Christmas has come early [Then gets so excited crashes down step and into people, chairs, audience.]

It turned out the STM publishers had released a statement on Wed (the day before and I’d missed it). It starts

COMMITMENT BY STM PUBLISHERS TO A ROADMAP TO

ENABLE TEXT AND DATA MINING (TDM) FOR NON COMMERCIAL SCIENTIFIC

RESEARCH

IN THE EUROPEAN UNION

“Signatories [STM] commit to granting the necessary copyright licenses to permit the text and data mining of copyright protected content and other subject matter on reasonable terms for non-commercial scientific research purposes in the European Union.

“the purpose of non-commercial text and data mining of subscribed journal content for non-commercial scientific research, at no additional cost to researchers/subscribing institutions”.

The document’s a bit abstract in places, but it’s a political not technical one. The general message is that I and my friends can go ahead and mine content as long as I don’t burn out the publishers’ servers or publish copyright material (e.g. licensed pictures of Mus Michaelis (copyright Disney Corp)).

Dear STM publishers I won’t do either of those deliberately and if I make a mistake I’ll rectify it and say sorry.

So we’re starting today! Ross and I hacked AMI yesterday to read and emit species from HTML, XML PSF and SVG. We’ll start extracting species next week and publishing them daily or even more frequently.

The big day is Wed 27^th November – Oxford Open Science run by Jenny Molloy. We are launching our TDM kit for researchers (completely Open). The idea is thate researchers will find it easy to use, saves them time, and they’ll enhance and distribute it. The idea is to get it widespread into research labs where it grows its open culture of mining and sharing the results.

I’d be delighted for offers of help. Over the next few posts I’ll detail what TDM for science is and how to get started.

.

see also…
http://www.booktrade.info/index.php/showarticle/50730/

http://www.publishers.org.uk/index.php?option=com_content&view=article&id=2630:the-pa-joins-european-commitment-to-enable-text-and-data-mining&catid=503:pa-press-releases-and-comments&Itemid=1618

Can Felix and AMI index Lucy’s repo? YES!

 
At UKSG (which I’ll blog later) I talked about repositories and said that without indexing they were of little value. So #animalgarden were excited to get a tweet from
 

Lucy Ayre @lastic
 

#uksglive hard not to get excited by Peter talking about content mining. Turning PDFs into CSV/XML files. Index my IR Peter!

 
Lucy is from LSE. Professor Felix Q Potuit is also from LSE. He’s trying to index #animalgarden repository:

He’s asked AMI to do the electronic material:
 
So Felix and AMI would love to help Lucy. What does PMR think?
PMR thinks it’s a great idea! We could have a hack day or two. Let’s get in everyone who might be excited (not just from LSE, but everywhere – Birkbeck, OKFN, and London hackers and Brit Lib, and Cottage Labs, and booklovers and …). Buy them lunch and provide wifi.
And let them loose and see what they can do.
Yes – we have the prototypic technology to read the contents of repositories. We can turn PDFs and DOC into XHTML. We can extract facts , which can be used for research, or indexing.
#animalgarden look forward to hearing from Lucy!
 
 

In my presentation to the @UKSG tomorrow I shall argue that the scholarly literature is a vast untapped source of high quality data and can supplant traditional expensive, human abstracting services. The organizers asked me not to present vapourware (which is easy because I never do!). The Content Mine is not vapourware, but it’s only just started the first lode.

FACTS are uncopyrightable. A purchaser of a book can extract and republish facts without permission. A subscriber to a journal can extract and republish facts – how else have the abstracting services worked? So my machines can extract and analyse and check and republish and enhance scientific data.

We have been doing this for 6 years in crystallography. But we weren’t the first. The Crystallography Open Database (COD) celebrates 10 years of machine abstraction from the literature. Started in France, by Armel Le Bail, it’s moved to Vilnius in Lithuania where Saulius Gražulis runs it. They’ve got 240,000. Our own Crystaleye (CY) was developed in 1 year by Nick Day, who needed structures for computational chemistry and decided to abstract the whole visible, Open literature for crystal structures; he’s got about 250,000. Obviously there’s a lot of overlap but it’s probably fair to say there’s a total of 300,000 high quality useful crystal structures.

And it’s all Openly available. It can be used for drug discovery, new electronic materials, capturing carbon dioxide, and education.

How does this happen? The crystallographic community requires all published structures to have data available (Supplementary data; Supporting Information) on the websites alongside the “fulltext”. Our machines can hoover these up and, importantly, associate them with the fulltext. Most publishers, RSC, ACS, Nature… regard SI as public data even for closed fulltext. (However Wiley, Springer, Elsevier put it behind the CCDC access wall where structures are only available by email, and then with a limit of < 100).

You might think that COD and CY are rivals – that’s the traditional academic view. Not at all – we are friends and we are starting serious collaboration! We have jointly obtained a grant from the Lithuanian Academy of Sciences for me to travel to Lithuania (Vilnius) and work together to integrate our data collections.

And our tools – which are Open and powerful. For example I’ll be integrating the Content Mine tools which will allow a complete indexing of the crystallographic literature and data from inside the fulltext! That means that our data base will contain material not available anywhere else – open or closed.

We shall simply have the best Open small-molecule database on the planet.

And we invite volunteers at add material and make the software and services better!

We’re now well under way with The Content Mine – a project to extract 100 million facts from the scholarly literature (mainly journals). Here’s how it will work , (also see neighbouring blog posts posts and the video (https://vimeo.com/78353557 , a 5 minute summary of what we are going to do and why it will work). I summarised the problem: as:

• How we are going to get them?
  
• How are we going to process them?
  
• Where do we put them?
  
• Who will help?
  

And addressed (3) in part (Wikimedia, but there are additional places).

The overall summary of the people I want to talk to (not necessarily those that have given any approval, and nowhere near complete) is shown:

It’s a bid hard to read (some animals don’t have very good handwriting). But in the period since the video was made I’ve talked with BMC (+Ross Mounce, and we are going back), PLoS, and we shall visit, British Library (and we’ll go back), EuropePMC (AGM tomorrow), might bump into eLife in the pub… emails with Greg Wilson (SW carpentry), Wikimedia (and will visit), etc.

And I’m presenting at UK Serials group (“journals” ) on Thursday and I’ll invite legacy publishers to get involved. There’s all sorts of reasons why they should like the idea of their data being mined, without their having to do anything other than hold the lawyers back). If I make a good case I’m sure I’ll convince them. [With possibly one or two exceptions].

So Q (1). Where are we going to get the facts?

We’ll get them from standard journal articles that I and and millions of others can read (either Openly or because our universities have paid to read). They normally consist of:

• “the full text”. This is normally PDF, which used to be impossible to read, but I have cracked enough of the (awful) typesetting that you can regard this as a problem solved in principle. The good news is that some articles contain vector diagrams in the PDF and I can read these. About 40% +- 10 of BMC is vector. Most of ArXiV is.
• OR html. This is useful because we convert PDF to XHTML and we analyse that. Most Open publishers and many legacy publishers expose HTML. This has some advantages (no errors converting from PDF) but has the problem that the figures and tables may be removed. Sometimes high Unicode points (especially math) are represented by images.
• OR XML. This is available from Open Access publishers but jealoulsy guarded by legacy ones. Since I can recreate it from the PDF I don’t need it. It’s useful because it uses a single DTD (JATS/NLM).
• Supplemental data. This can be anything. In good cases (CIF) it’s highly structured and well described. In almost all other cases it’s messy, ranging from CSV (parsable, but without ontology) to XLS (parsable but even worse) to data destroyed as PDF or WORD. (Except I can recover some of this).

Some closed publishers (e.g. Nature, Royal Soc. Chem., Amer. Chem. Soc.) expose their supplemental data and let us mine it already. Others hide it behind paywalls.

I’m going to do this in stages, starting with Open Access publishers (i.e. those who have 95%+ of their output as BOAI-compliant – I do not use confusing terms like “Gold”). I can do this without seeking their permission as the CC-BY licences grant me the right. However I am keeping in touch because it’s good to have a harmonious community – for example I wouldn’t want to burn their servers out by bad practice. (Actually this is complete FUD (see Cameron Neylon) – there is no way that PMR and friends *could* burn out anyone’s servers). But I’d like to be seen to be responsible.

So I’m talking with them. If you are an open access publisher who would like their content mined, and have a community that would appreciate it, get in touch.

I’m starting with BioMedCentral. Why?

(Not because they have a mascot (Gulliver Turtle) and the others don’t). But because Ross and I need what’s in BMC Evolutionary Biology and we’ve developed the tools on that. And because BMC (unlike PLoS) keep vector graphics. BUT! I have talked with PLoS and as a result they are going to …

And we also know that the Royal Society allows content mining by subscribers. So there’s a lot to get started on.

And on Thursday I’ll ask the legacy closed publishers to allow content mining by subscribers. We don’t need their help (i.e. we don’t need XML (because it has no pictures)) – we just want them to chain up their lawyers.

Would I be breaking the law to mine FACTS from content I had paid to read? I don’t think so. “The right to read is the right to mine”. I don’t want to break the law. As I said last week “I have kept within the law – up till now”. I want to be able to continue to say that.

But at some stage, next year, we shall move onto FACTS in the whole literature.

I’ve been invited to talk to the UK Serials group. (“Serials” == journals for most people). http://www.uksg.org/event/NOVCONF2013. “Open Access Realities:
global experiences of implementing OA”. Here’s the programme:

10.10  Welcome and introduction by the Chair Charlie Rapple, Associate Director, TBI Communications

10.15  How are institutions and publishers placed to “really do” OA? Damian Pattinson, Editorial Director, PLOS ONE

10.45  Make Open Access work!: the moment of truth for academic libraries  Lars Bjørnshauge, Director, SPARC Europe and Managing Director, DOAJ

11.45  Finch one year on – a review of progress Michael Jubb, Director, Research Information Network

12.00  How are universities putting policy into practice, from both library and research perspectives? Adam Tickell, Provost and Vice-Principal and Jill Russell, Digital Assets Programme Manager, University of Birmingham

12.30  How can existing Open Access models work for humanities and social science research? Caroline Edwards, Lecturer in Modern and Contemporary Literature, Birkbeck, University of London (Co-Founder and Co-Director of the Open Library of Humanities)

14.00  How are subscription publishers making the transition to OA? Vicky Gardner, Open Access Publisher, Taylor & Francis

14.30  What impact will open educational resources (OER) have on teaching and learning? Steven Stapleton, Project Manager, University of Nottingham

15.30  Scientific data costs billions but almost all is thrown away – what should be done? Peter Murray-Rust,

I shall listen carefully to what is said and shan’t decide exactly what to say till I start speaking. But unlike (I think) several of the speakers I am not happy with the state of or progress of OA. I talked at Spoton2013 to scientists from the European Periphery – Greece, Spain, Ireland (and others). They are now effectively excluded from scholarly publication. They cannot buy serials (at all) and they cannot publish in “Gold” APC-based journals because of the extortionate publisher-driven fees. Their only option, and I support it, is to re-invent scholarly publishing. I’ll publish more when I get some of their details.

But that’s not what I shall talk about. I shall talk about the almost complete failure of the scholarly publishing system to communicate science. It has fallen into a dysfunctional status where the means of scientific publication is driven by corporates (including rich “scholarly societies”) who wish to perpetuate their status at the expense of innovation and progress. I shall contend that this leads to tens of billions of dollars of funded work being effectively unpublished and therefore wasted. Scholarly publishing still thinks in terms of C19 pages and C20 information abstracters and this destroys change.

But I am not primarily whinging. I intend to change this and I shall describe C21 ways that I shall be using including the use of modern web technology and community. I shall show tools which should change the face of data in serials

I shall hope to change delegates’ minds. I shall ask for commitment and help from those I have changed. Even a modest number will be priceless.

And to get you interested here are two questions I shall ask now and answer on Thursday:

• Who are the 5 most important Open Access Publishers in Science (I include BioMedCentral and PLoS so I want 3 more)?
  
• Which are the 3 most important Open Access repositories?
  

A massively valuable meeting at #solo13. I’ve told you how we are going to extract 100 million facts from the scientific literature. That’s an act of faith and we have to start building the reality. We’ve got to work out:

• How we are going to get them?
• How are we going to process them?
• Where do we put them?
• Who will help?

At #solo13 there was a session on Revolution run but Alok Jha, Science correspondent of the Guardian. (Alok’s been very helpful in the past, highlighting our battle to have lawyer-free content-mining). After the panel, Alok asked for people who were running revolutionary campaigns.

I volunteered The Content Mine (blog post and 5-minute video https://vimeo.com/78353557 ). I said this was revolutionary. I also said “‘I work within the law… Up to now…” – that’s a factual statement, not a threat. Afterwards I met up with Toni Sant, who is education organizer at Wikimedia UK.

Before I continue, a clarification. Wikimedia is a foundation, Wikipedia is an encyclopedia (albeit the greatest the world has seen), Wikileaks is completely separate, MediaWiki is a useful tool from Wikimedia. Wikimedia has created many Wiki-projects, see http://www.wikimedia.org/. There’s 16 projects. That’s too much for me to hold in my head (Greg Wilson has told me this). So If I am confused that’s an objective problem. So Wikimedians, please forgive and correct errors.

Anyway the really exciting thing is that Toni and colleagues would love to host the data coming out of the Content Mine.

Ross and I have been invited to visit the Wikicommunity down in Wikiland London (Leonard Street, off Silicon Roundabout). The most likely receptacles will be Wikidata and Wikispecies.

We’ll need to discuss details – does our output need curation? If so how? And by whom? Or is the quality produced by machines sufficiently good (I think it is for species, and maybe chemicals, and some identifier systems). We are all communally very excited?

Why am I not doing this in my University or the University of Bath?

Because Universities don’t understand modern information as well as and as enthusiastically as Wikimedia. Wikimedia understands:

• Distributed systems
  
• Version control
  
• The semantic web
  
• Communities and crowd-contributions
  
• The world of citizens outside academia
  
• Identifier systems
  
• Domain-specific information
  
• The law
  

Universities (with a very few exceptions) aren’t interested and aren’t competent. I have tried to use repositories for data and effectively failed. Anecdote: A senior retired academic asked his library to store his database. “Oh we don’t support academic databases, only ones we buy in”. (He “solved” that by turning his database into LaTeX, then PDF and then the library was happy to accept it as it was a “book”). I might have thought that my own University would be interested in my experience of text-and-data mining, but they don’t answer my mail.

So I go where the energy, the vision, the community is. I think Wikimedia will largely replace university libraries for most people (and certainly the #scholarlypoor). Wikimedia states:

Imagine a world in which every single human being can freely share in the sum of all knowledge. That’s our commitment.

I share that vision. Universities, unfortunately, don’t.

The only thing that prevents Wikimedia – and it’s a horrible thing – is the stranglehold of copyright and contract law from megacorporations, abetted by the supine-ness of Universities. If your University really values citizens of the world it should work WITH Wikimedia, not against it.