Open Data in Climate Research?

Dictated and Scraped into Arcturus

Yesterday evening I went to a discussion at the Royal Institution. I’ll first give the abstract of the occasion and then my motivation and conclusions. Please read what I write very carefully, because I am not commenting on the primary science – I am commenting on how the science and its conclusions are, or are not, communicated.

The Climate Files; The battle for the truth about global warming

 

In November 2009 it emerged that thousands of documents and emails had been stolen from one of the top climate science centres in the world [PMR: The Climate Research Unit (CRU) at the University of East Anglia (UEA), UK] . The emails appeared to reveal that scientists had twisted research in order to strengthen the case for global warming. With the UN’s climate summit in Copenhagen just days away, the hack could not have happened at a worse time for climate researchers or at a better time for those who reject the scientific consensus on global warming. Yet although the emails sparked a media frenzy, the fact is that just about everything you have heard and read about the University of East Anglia emails is wrong. They are not, as some have claimed, the smoking gun for some great global warming hoax. They do not reveal a sinister conspiracy by scientists to fabricate global warming data.

To coincide with the launch of his new book, The Climate Files, the veteran environment journalist Fred Pearce discusses how the emails raise deeply disturbing questions about the way climate science is conducted, about researchers’ preparedness to block access to climate data and downplay flaws in their research.

This will then be followed by a panel involving Dr Myles Allen (University of Oxford) and Dr Adam Corner (Cardiff University).

Fred Pearce was the main speaker and described in detail his analysis of the emails which had been exposed from UEA. I would agree from his analysis that there is no “smoking gun” and that many of the emails were unfortunate rather than malicious. He was then answered by Drs. Allen and Corner, and there was clearly some disagreement between them and him. The discussion was then opened to the audience (which included scientists, journalists and many others) and a lively and valuable debate took place.

I should make it clear that I am making no comment at the moment as to whether global warming is a reality and if so how important it is. And I am deliberately taking the position of an agnostic because I want to find for myself what the evidence is and how compelling it is. For that, it is important that the information is Open and so it is as a “data libertarian” (a useful phrase which I heard last night) that I attended the meeting.

As a result of the presentations and the discussions within the panel it seemed to me that there was a serious lack of Openness in the Climate Research community. It is important not to judge from just one meeting but given the enormous public reporting and discussion I was disappointed to find that there were still parochial and entrenched attitudes about ownership and use of data.

My superficial analysis is that the CR community has retreated into defensive mode and has not changed its communication methods or interaction with the community. This is perhaps understandable given the hostility and publicity of much of the media coverage and further comment (and UEA has put a ban on staff speaking on the issue). Such bans can recoil, as it is then easier to believe there is something to hide. It may be difficult, but it seems essential to radically overhaul the governance and communication.

On more than one occasion the panel asserted that Climate data should only be analysed by experts and that releasing it more generally would lead to serious misinterpretations. It was also clear that on occasions data and been requested and refused. The reason appeared to be that these requests were not from established climate “experts”. This had led to the Freedom Of Information Act (FOI) being used to request Scientific Data from the unit. This had reached such a degree of polarisation that of over 100 requests only 10 had resulted in information being released by the University. I had no idea that this “FOI battle” had been going on for several years and that nothing had been done to try to solve the problem. This in itself should have been a signal that change was necessary – however inconvenient.

We should remember that climate research is not an obscure area or of science but something on which governments make major and lasting decisions. It surprised me that there was not an innate culture of making the data and research generally available. The CRU is effectively a publicly funded body (as far as I know there is minimal industrial funding) and I believe there is a natural moral, ethical and political imperative to make the results widely available. The FOI requests should have been seen as a symptom of the problem of not making data available rather than as, it appears, being regarded as irritation from outsiders. Whatever the rights and wrongs, it was a situation with a high probability of ending in public disaster (as it did).

I was sufficiently concerned that I spoke at the end and although I do not have my exact words I said something like the following:

“I am a Chemist and a data libertarian. I am not an expert in climate change but I believe that I could understand and contribute to some parts of climate research (e.g. data analysis and computational science and I do not accept the need for a priesthood. In my advocacy for publishing Open Data I encounter many fields where scientists and publishers are actively working to make data openly available. The pioneers of genome research and structural biology fought their culture (which included major commercial interests) to ensure that the results of the work was universally available. I see other areas where scientific papers cannot now be published unless the scientists also make their data available at time of publication. Climate research appears to have generated a priesthood which controls the release of information. For a science with global implications this is not acceptable.”

This will not be my last blog post on this issue. I was sparked into action when I heard a talk in Cambridge by Nigel Lawson (Margaret Thatcher’s Chancellor of the Exchequer). Lawson argued (using proof by political assertion) that climate change research was a conspiracy. He has now set up a foundation to challenge the mainstream view (The Global Warming Policy Foundation). However I realized while listening to him that I did not have compelling incontrovertible Scientific Data and arguments that I could use to challenge his views. This is an untenable position for a scientist and so I believe I must educate myself and my fellow scientists about which pieces of information are genuine.

To do this we have to develop a culture of Openness and a number of us discussed the problem at the Open Knowledge Foundation’s OKCon earlier this year. Although much has been written and continues to be written on climate research there is no Open repository of information.

The OKF’s goal is to create or expose Open resources. We are currently thinking about how to do this for climate research. We have to be extremely careful that we do not “take sides” and that our role is strictly limited to identification of Open resources.

This entry was posted in Uncategorized. Bookmark the permalink.

8 Responses to Open Data in Climate Research?

  1. David Jones says:

    Perhaps you could kick off with what data you would like to see open.
    Data that I would like, that you might have a professional opinion on, is a reference library for the IR spectra of the Kyoto Protocol gasses (CO2 and other greenhouse gasses). I had a look, but I couldn’t find an open archive of IR spectra. Do you know if one exists?

  2. Martin Ackroyd says:

    I’d suggest that essential reading for anyone interested in these issues are:
    Climategate The Crutape Letters by Steven Mosher and Thomas W. Fuller
    and
    The Hockey Stick Illusion;Climategate and the Corruption of Science by A W Montford
    It has often been said that the climategate emails were taken out of context. But with the full context, as revealed by Mosher and Fuller, they are utterly damning. And these were emails exchanged between leading authors of the IPCC reports.
    Once you understand how the famous IPCC Hockey Stick Graph was based on erroneous statistics and dodgy manipulations of proxy data, as set out in verifiable detail by Montford, you wonder if anything at all from “climate scientists” can be trusted.

  3. Fay Kelly-Tuncay says:

    Hi, may I also suggests:
    Richard Lindzen The Peculiar Issue of Global Warming Fermilab Colloquium 2-10-2010.
    http://www.youtube.com/watch?v=-z5AeI2DUgM&feature=related
    Or Prof Bob Carter’s new book, I’ve just got it from Amazon.
    http://www.youtube.com/watch?v=FfMB1BpPqsU
    Or if you are feeling bold you might like to join my face book group: Repeal Cimate Change Act 2008 Campaign
    http://www.facebook.com/group.php?gid=107337052636523
    Good luck with your project. Bye. Fay

  4. Richard J says:

    Nigel Lawson sat on the House of Lords Select Committee which reviewed Climate Change and its Economics in considerable detail in 2005, taking a wide range of technical submissions. If I recall correctly, they concluded that the scientific and economic aspects were potentially flawed by the overtly politicised nature of the IPCC process.
    Lawson’s concerns should not be dismissed lightly. They are shared by many scientists in Earth Science disciplines closely related to climate science, but perhaps not directly reliant on research funding in this field.

  5. David Holland says:

    Peter,
    As the individual whose Freedom of Information Request resulted in the infamous email “Mike, can you delete any emails you may have had with Keith”, I have to tell you that Fred Pearce is well wide of the mark.
    If anyone wants to know why someone would try to procure the deletion of AR4 emails just two days after I asked for them, just ask me for a confidential copy of my submission to the Russell Enquiry and confirm that you will not publish it. It is not on the Enquiry website because Sir Muir’s Enquiry does not have Parliamentary privilege and it is worried about being sued. I guess that also limits what the Enquiry will report.
    The email address is crusub@t***o.net (the biggest UK supermarket).

  6. David Bishop says:

    Peter
    I have linked here from Andrew Montford at Bishop Hill; like him I welcome your interest in the (so-called) Climategate emails. I am an observer, no more; as a non-scientist I am not in a position to verify or otherwise the science itself. However, as a reasonably widely-read person of a certain age (nearing retirement) who has followed the climate debate with concern for several years now, I judge that I am able to make a sound assessment of the state of play, and in my view Fred Pearce’s views on the emails are somewhat off target.
    Your focus was the emails, and so I will limit myself to those. Rather than pigeon-hole people as being poles apart (“alarmist” v “denier”), I judge that there is a range of views along a spectrum. Fred Pearce offers one perspective, but from my wide reading not one of the centre, tending much more towards the alarmist view. I would therefore recommend to you a perspective offered by John Costella. His very thorough analysis of the emails (in pdf format) is at this link:
    http://www.lavoisier.com.au/articles/greenhouse-science/climate-change/climategate-emails.pdf
    You will see that John Costella’s study is dedicated to John Daly, who died in January 2004. What you may not be aware of is this email message from the corpus; it is from Phil Jones to Michael Mann in reaction to that event:
    From: Phil Jones
    To: mann@vxxxxx.xxx
    Subject: Fwd: John L. Daly dead
    Date: Thu Jan 29 14:17:01 2004
    From: Timo H‰meranta
    To:
    Subject: John L. Daly dead
    Date: Thu, 29 Jan 2004 12:04:28 +0200
    X-Mailer: Microsoft Outlook, Build 10.0.4510
    Importance: Normal
    Mike,
    In an odd way this is cheering news ! One other thing about the CC paper – just found
    another email – is that McKittrick says it is standard practice in Econometrics journals
    to give all the data and codes !! According to legal advice IPR overrides this.
    Cheers
    Phil
    “It is with deep sadness that the Daly Family have to announce the sudden death of John
    Daly.Condolences may be sent to John’s email account (daly@john-daly.com)

    Reported with great sadness
    Timo H‰meranta
    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    That reaction cannot, in any terms whatsoever, be acceptable. And in my considered judgement the attitude thus shown, which is only the tiniest snippet from the voluminous material, permeates the whole sorry saga, and taints climate science as currently practised virtually irremediably.
    You will also note from the email message above the incredulity expressed by Jones that data and codes are released as standard practice in other disciplines. This points to the heart of your own concern regarding openness in science, and the fact that it is woefully lacking in ‘climate science’. In that respect I wish you well with the Open Knowledge Foundation, which I learned of from Bishop Hill; I will now be exploring it keenly.
    With kind regards
    David Bishop

  7. PaulM says:

    I am sure that as a scientist you are well aware of the importance of obtaining information from the primary source rather than relying on other people’s opinions.
    So I find your remark “I would agree from his analysis that there is no “smoking gun” and that many of the emails were unfortunate rather than malicious” rather worrying.
    I hope that you will read the emails yourself and consider whether ‘fortunate’ or ‘malicious’ is the more appropriate term. You could start with the very detailed investigation by John Costella linked above, or the briefer summary from Bishop Hill at
    http://bishophill.squarespace.com/blog/2009/11/20/climate-cuttings-33.html
    Here are a few quotes:
    “I can’t see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow – even if we have to redefine what the peer-review literature is !”
    “If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone.”
    “Can you delete any emails you may have had with Keith re AR4? Keith will do likewise.”
    “Anyway, I’ll maybe cut the last few points off the filtered curve before I give the talk again as that’s trending down as a result of the end effects and the recent cold-ish years.”
    On the more general global warming question it will be interesting to hear the results of your investigations starting from an agnostic position. I embarked on a similar journey about 3 years ago and was quite shocked by the results.
    Where does Nigel Lawson say that climate research is a conspiracy? I suspect that you may be misquoting him.

    • pm286 says:

      >>I am sure that as a scientist you are well aware of the importance of obtaining information from the primary source rather than relying on other people’s opinions.
      I do, though in this case the matter was not primarily scientific but journalistic.
      >>So I find your remark “I would agree from his analysis that there is no “smoking gun” and that many of the emails were unfortunate rather than malicious” rather worrying.
      This is a new area to me and I had little idea of what would be said at the meeting. I will rephrase my statement to “At the meeting I saw no reason to disagree with his analysis… ”
      >>I hope that you will read the emails yourself and consider whether ‘fortunate’ or ‘malicious’ is the more appropriate term. You could start with the very detailed investigation by John Costella linked above, or the briefer summary from Bishop Hill at
      http://bishophill.squarespace.com/blog/2009/11/20/climate-cuttings-33.html
      I have read some. I do not intend to comment on anything other than the question of Openness of data/code/documents. I shall not comment on the content of the science or of the human aspects.
      >>On the more general global warming question it will be interesting to hear the results of your investigations starting from an agnostic position. I embarked on a similar journey about 3 years ago and was quite shocked by the results.
      That does not surprise me. However I shall be publicly agnostic. It requires a great deal of work and time to do justice to the rights and wrong both scientific and human. I shall confine myself solely to trying to label resources as open and trying to get people of whatever persuasion to make material Open.
      >>Where does Nigel Lawson say that climate research is a conspiracy? I suspect that you may be misquoting him.
      He gave a talk in Cambridge that I attended. There was no script so I cannot be sure what he said. At the time I felt he was stating that research had been systematically falsified.

Leave a Reply

Your email address will not be published. Required fields are marked *