Jim Downing delicioused me the following link from the UK HEFCE. We are now going to be assessed by robots. (Since I am promoting the concept of robot readers of journals I can hardly complain). Skim the following (which has enthusiastic comments from those who have devised it – including the effect that it will reduce the financial burden of assessment. I comment at the bottom..
Response to consultation on successor to research assessment exercise
Quality stays at the heart of the process, but the bureaucratic burden on universities will be cut, says Johnson
As part of the pre-Budget report, Education Secretary Alan Johnson today published the outcome of the Government’s recent consultation on a new assessment process to replace the Research Assessment Exercise (RAE) after 2008. To reduce the administrative burden on universities, the RAE will be succeeded by a new overarching framework for assessing research quality and allocating funding which is more metrics based. The Higher Education Funding Council for England (HEFCE) will lead further development of the new process, which will use a set of indicators of research income and quality combined with advice from experts including research users, to rate university research and inform the distribution of Government funding for research funding for English universities.
The Government announced in March alongside the 2006 Budget its intention to replace the RAE after 2008 with an assessment system based on metrics – statistical indicators such as the number of times research is cited by other researchers or the amount of research income a department earns. Metrics offer a less burdensome assessment process than the RAE which depends on universities submitting research outputs for review by subject panels. But the proposals recognised that metrics are more readily applicable to assessment of some subjects (broadly sciences) than others.
Nearly 300 organisations and individuals responded to the consultation, which ran from June to October. Some key criteria for the new process were identified: it should continue to use expert advice; should recognise disciplinary differences within a common framework; and it should use an indicator directly linked to research quality.
Alan Johnson said:“The response to our consultation was helpful and we have heeded it. The outcome we are announcing today keeps quality at the heart of the assessment process, whilst reducing the administrative burden on universities. HEFCE will work closely with the sector as it takes our plans forward and the timetable we have set means that universities can continue their work towards the 2008 RAE with the assurance that its outcome will have a reasonable lifespan. But they can also be confident that the new arrangements will build on the RAE’s success, and continue to recognise research excellence in all its forms.”
Welcoming the announcement’s inclusion in the pre-Budget report, Financial Secretary John Healey said:
“A world-class research base is essential to enabling the UK to respond to the challenges and opportunities of globalisation. The new framework for research assessment and funding will ensure that excellent research of all types is rewarded, including that most likely to have an economic and social impact.”
Chief Executive HEFCE, Professor David Eastwood, said:
‘We welcome today’s announcement on the future arrangements for assessing research quality, and for allocating the Council’s research grant, beyond 2008. This provides a stable framework for our continuing support of a world leading research base within HE which is dynamic and responsive to the needs of our stakeholders and research users. We will work with the sector to develop further the new assessment and funding arrangements that will be robust and command their confidence, and to secure a smooth transition.’
The outcome announced today is a new process that uses for all subjects a set of indicators based on research income, postgraduate numbers, and a quality indicator. For subjects in science, engineering, technology and medicine (SET) the quality indicator will be a bibliometric statistic relating to research publications or citations. For other subjects, the quality indicator will continue to involve a lighter touch expert review of research outputs, with a substantial reduction in the administrative burden. Experts will also be involved in advising on the weighting of the indicators for all subjects.
The first assessment exercise for SET subjects will be in 2009, and will begin to inform funding in the 2010/11 academic year. Other subjects will have their first assessment under the lighter touch regime during 2013 and this will inform funding from 2014/15.
Alan Johnson has written to the Chairman of HEFCE, David Young, to ask him to lead the further work and consultation necessary to complete development and testing of the new system. HEFCE will conduct a report on progress to the Department for Education and Skills in time for the 2007 pre-Budget report.
So it is citations that matter. Who creates the citation metrics? HEFCE? No – commercial and quasi-commercial organisations, closely allied to the publishing business. So the publishing industry controls our output and now has a stranglehold on the research economy (e.g. my job) through its metrics.
So how do I get cited? Here’s today’s quiz.
What are the ten most cited papers in chemistry in 1995-2005?
(It’s easy enough to cheat – but have a guess before you do…) I’ll then outline my plan to become the (jointly) most cited chemist of all time. And I’m serious – it has a large element of reality… See if you can work out my plan.