Mammoth UK research assessment concludes as leaders eye radical shake up

Researchers across the United Kingdom are celebrating or commiserating this week as universities receive the results of a years-long research-assessment exercise that dictates how much government funding they will get over the coming years.

The Research Excellence Framework (REF) is one of the world’s most comprehensive research assessments — many nations look to it as an example of how to review research. But its leaders are already looking at how they might change the process.

The 2021 REF results suggest that more than 40% of UK research is world-leading. In an analysis of the results by the Times Higher Education (THE), the country’s biggest research universities rank among the highest-scoring institutions overall. The top five are all located in southern England (see ‘Institution rankings top 10’).

Almost all UK universities took part in the mammoth exercise, together submitting 185,000 pieces of research from more than 76,000 researchers for assessment.

The funders in charge of the REF are reviewing how the next assessment might change. Currently, research is given a star rating by 34 expert panels that fall into four broad categories: medicine, health and life sciences; physical sciences, engineering and mathematics; social sciences; and arts and humanities.

Reviewers also judge institutions on the impact of their work in the wider world and the standard of their research environment. Scores for each element carry a different weight in a formula that dictates the size of each institution’s share of the multibillion-pound pot for public research funding. Currently, the score for research outputs holds the biggest sway, accounting for 60% of the final mark.

The UK government is yet to announce how much money will be up for grabs, and how it will be divided between different institutions. The results of the previous exercise, released in 2014, guided £14 billion (US$17.1 billion) of university research funding.

“[The results] respresent an exceptional achievement for UK university research and demonstrate the huge return on public investment in research,” says Steven Hill, director of research at the funding body Research England.

Changing priorities

Those who administer the research-assessment exercise — the UK’s higher education funding councils — have a history of changing the rules each time to reflect priorities and to help stop institutions from gaming the process to boost their scores. Ahead of the 2014 results, the exercise was broadened to include a measure of research impact. Researchers now submit case studies to demonstrate the economic, social and policy contributions of their work — a move that has been copied by other countries seeking to widen their approach to research assessment.

The latest change included a rule that institutions must submit for assessment the work of everyone who does research as part of their job. Previously, some institutions put forward only top performers in an attempt to skew ratings in their favour. As a result of the change, the latest exercise saw a 46% increase in the number of staff submitted for assessment compared with the previous one.

Last year, funders began to look at how the process could be used to recognize and reward institutions fostering a positive research culture. So the rules of the next assessment, the date of which has yet to be confirmed, could look significantly different.

“The research system itself is under unsustainable pressure in terms of careers and livelihoods. If we want to improve research culture, then the REF is potentially a powerful ally in that effort because it gives you that reach across the entire system,” says James Wilsdon, a science-policy researcher at the University of Sheffield, UK. “I do think the stars are now aligning to support a more radical overhaul of the exercise than at any point in the past 20–25 years.”

Catriona Firth, the associate director for research environment at Research England — one of the four funding councils that administers the REF — agrees that a radical shake up could be on the cards. One aspect of the current review process, known as the Future Research Assessment Programme (FRAP), looks at how the framework can be used to recognize and reward positive research culture. “The current REF has been quite focused on the research end points, and not focusing so much on the inputs or the research process,” says Firth.

Because research outputs are assessed for their originality, institutions do not submit review articles, negative results or replication studies as part of the exercise, which are all important for research, she says. “There are wider consequences of focusing on excellence. What institutes think is going to be valuable in the REF is what they encourage staff to do and what they invest in.”

The tricky part, says Firth, will be balancing the continued drive for excellence with rewarding healthy research culture, without a placing a disproportionate administrative burden on institutions. The REF already has many critics, who claim it is bureaucratic and expensive (the 2014 exercise cost £246 million to run).

The results of the REF review will be very important for the rest of the world, says Lidia Borrell-Damián, who is head of research and innovation at policy group Science Europe and sits on the FRAP international advisory board. “It will be full of insights on what to consider when reforming the assessment,” she adds.