RAE: top dogs don't slice salami

Jonathan Adams, Evidence Ltd. Research Assessment and UK publication patterns.

The UK's Research Assessment Exercise is a research evaluation cycle which considers output, training, funding and strategy across HE institutions. Peer review panels. Forthcoming changes following recent government budget statement -- shift to metrics post 2008.

The RAE has evidently led to an increase in the UK share of world citations; if citations are a measure of research importance, then UK research is now much improved since the early 80s and has this year overtaken the US in biology and health sciences. RAE is a major driver of research activity in universities. It assesses 4 items per researcher -- including books, articles, proceedings, other works.

Analysis of publications submitted to RAE in 1996 and 2001 -- using publication data to assess comparability between subject areas. Evidence shows that journal articles are proportionally the highest output in science, conference proceedings in engineering, book chapters and monographs more common in humanities, and other content (videos, installations etc.) in the arts.

Researchers submit material for assessment which represents their highest quality work; the assessment will affect the amount of funding received, and departmental prestige.

A shift towards journals is evident -- more journal articles were submitted for assessment in 2001 than in 1996 but in comparing these to ISI's content (Web of Science) it is evident that in some subjects, ISI's coverage is comparatively decreasing -- suggesting Web of Science may be less representative of research in some areas than others (e.g. social sciences less well represented).

Changing cultures -- social science researchers do use bibliometric data to evaluate research quality, but do so in an expert way; journals will become increasingly significant.

(3 days at UKSG caught up with me at this point and the notes I made for the remainder of the presentation make so little sense that they would detract from Jonathan's presentation -- so I have quit while I'm still vaguely ahead!)

Q: Greg Kerrison, Qinetiq. How has the RAE influenced the research process?
A: better overall performance; higher level of productivity. In terms of the way people publish, no evidence that we have increased significantly in comparison to other G8 countries. Suggestions of salami slicing don't seem to be justified; it may be that some (less high profile) researchers are focussing on shorter term goals (in order to have adequate content to submit for the next RAE), but the best researchers are not swayed in that direction.