scifoo: young scientists and the culture of fear

On the last day, and as an inspiration from the previous sessions and the community atmosphere of the meeting, Andrew Walkingshaw and Alex Palazzo ran a session on the problems of being a postdoc under the pressure of having to publish in high-impact journals. They explained how the very high sense of competition and the pressure of conformance to a single way measure of success constrained innovation – their sense of concern came through very clearly. Here’s their blog entries (AW first):

The Scifoo nature

Scifoo was a blast.
Alex Palazzo and I ran a session today on the politics of scientific communication/open access, particularly for young scientists: he writes about our thoughts here. I was really delighted with how it went; many people, including some very successful academics and editor-in-chief of Nature, Philip Campbell, came along and shared their thoughts.
There’ll be more on what we actually discussed in due course, but the thing happening was itself staggering; from half-formed idea to a really deep round-table discussion in less than forty-eight hours. Creating a space where that can happen is priceless; I can’t thank the organisers enough for inviting me, and, equally importantly, everyone there for their generosity of spirit and openness.

PMR: Then AP. Read this in full, and also the commentary it has generated (and may continue to generate):

Scifoo – Day 3 (well that was yesterday, but I just didn’t have the time …)

Category: art, food, music, citylife and other mental stimuli
Posted on: August 6, 2007 10:46 AM, by Alex Palazzo
Our session on Scientific Communication and Young Scientists, the Culture of Fear, was great. Many bigwigs in the scientific publishing industry were present and a lot of ideas were pitched around. I would also like to thank Andrew Walkinshaw who co-hosted the session, Eric Lander for encouraging us to pursue this discussion, Pam Silver for giving a nice perspective on the whole issue, and all the other participants for giving their views.
Now someone had asked that we vlog the session, we actually tried to set it up but we didn’t have the time. In retrospect I’m glad we didn’t. This became at the last session of scifoo where attendees voiced their comments on the logistics of scifoo, many conference goers preferred to keep video and audio recording devices away from the sessions as they impede open discussion. Conversations off of the record can be more honest and more productive.
So about the session …
The main point that we wanted to make was that there are problems with the current way that we are communicating science and due to developments with Web2.0 applications there is a big push to change how this is done. But we must keep in mind the anxieties and fears of scientists. How we communicate does not only impact how information is disseminated but does affect the careers of the scientists who produce content. Until there is general consensus from the scientific publishing industry, the major funding institutions, and the higher echelons of academia (for example junior faculty search committees), junior scientists are unlikely to participate in novel and innovative modes of scientific communication. The bottom line is that it is just to risky to do so.
There are two main areas that remain to be clarified by the scientific establishment.
1) Credit. How do we ascertain who deserves credit for an original idea, model or piece of data.
2) Peer-review. Although most scientists and futurists who promote much of the open-access model of scientific publishing support some type of peer-review where the science or consistence of a particular body of work is evaluated, there remains some confusion as to whether peer-review should continue to assess the “value” of a particular manuscript. Right now, manuscripts that are submitted to any scientific publication must attain some level of importance that is at least equal to the standards of that particular journal. When evaluating the scientific contribution of any given scientist, close attention is payed to their publication record and particularly where their manuscripts are published. Now whether we should continue to follow this model where editors and the senior scientists determine the scientific validity of any given manuscript is being questioned. In an alternative model many technologists are pushing post-publication evaluation processes which evaluate the importance of any single manuscript after the manuscript has been released after minimal peer-review. These not only include citations indices, but also newer metrics that are currently being developed by many information scientists. There are many problems with these systems, the most critical is that most of the value cannot be assessed until many years after the publication date. An important piece of work may take years to have an impact in a given particular field. Until the scientific establishment reaches a consensus as to whether these post-publication metrics are indeed useful for determining the credentials of a scientist in the shorter term (<2 years post-publication) it is unlikely that any scientists would risk publishing their findings in a minimally peer-reviewed journal.
There was a strong feeling that the top journals do provide a valuable filtering service. They go through all the crap in order to publish the best work. OK they don’t always succeed but competition between all the big journals promotes a high standard. And many scientists are reluctant to give up this model. Journals also help to improve the quality of the published manuscripts, this function would be lost if all we had was PLoS One and Nature Precedings. To all those who think that journals must be eliminated in favour of an ArXiv.org model you are now warned.

PMR: I kept quiet during this session – I have no easy answer. It’s clear that the pressure to get scientific jobs is increasing – whereas not so long ago institutions could choose from those they knew (with all the pluses and minuses) now they try to create a “level playing field”. And what measure do they have when everyone has rave references? It’s difficult not to count the numbers. We did hear that one leading systems biology lab did not simply look at publications but wanted to choose people who could provide a major shift in emphasis and might have a relatively unconventional paper trail. But it’s not common.
Much credit to Alex and Andrew for their bravery in running this session, and to scifoo for it being the sort of place where it could happen.