Rankings don't tell the whole story – Handle them with care

By Phil Baty. In Russia, Prime Minister Dmitry Medvedev recently signed an order awarding official recognition to degrees from 210 leading universities from 25 countries – determined in large part by their presence in the top global university rankings.

The thousands set to benefit from study-abroad scholarships under Russia’s five-billion rouble (US$152 million) Global Education programme will also have to attend a top-ranked university.
A similar scholarship project in Brazil, the £1.3 billion (US$2 billion) Science without Borders programme for 100,000 students, also draws heavily on the Times Higher Education and other rankings to select the host institutions.
And in India this month, the government’s Universities Grants Commission set out new rules to ensure that only 500 universities ranked by two global rankings including Times Higher Education are allowed to run joint degree or twinning courses with Indian partners.
Such high-level official endorsement is, of course, gratifying and since 2009 when we joined forces with Thomson Reuters, we have worked hard to listen to critics of global rankings and consulted widely to develop a new, more balanced, comprehensive and rigorous ranking system.
We argue that Times Higher Education’s global rankings are the only ones in the world to examine all core missions of the modern global research university – research, teaching, knowledge transfer and international activity.
They are the only rankings to fully reflect the unique subject mix of each and every institution across the full range of performance indicators and to take proper account of excellence in the arts, humanities and social sciences, so badly neglected by other rankings, we believe. And they are the only global rankings to employ a rigorous, invitation-only survey of experienced, expert academics – with no volunteers and certainly no nominations from universities themselves.
Authority brings responsibility
But we are aware that such authority brings with it great responsibility. A reputation for integrity must be earned and maintained through open and honest discussion about both the uses and the abuses of global rankings. All global university ranking tables are inherently crude, as they reduce universities and all their diverse missions and strengths to a single, composite score. Anyone who adheres too rigidly to rankings tables risks missing the many pockets of excellence in narrower subject areas not captured by institutionwide rankings, or in areas of university performance – such as community engagement – that are simply not captured well by any ranking.
One of the great strengths of global higher education is its extraordinarily rich diversity and this can never be captured by any global ranking, which judges all institutions against a single set of criteria. In this context, a new declaration from a consortium of Latin American university rectors must be welcomed. The declaration, agreed at a two-day conference at the National Autonomous University of Mexico, titled “Latin American Universities and the International Rankings: Impact, scope and limits”, noted with concern that “a large proportion of decision-makers and the public view these classification systems as offering an exhaustive and objective measure of the quality of the institutions”.
No university ranking can ever be exhaustive or objective. The meeting, which drew together rectors and senior officials from 65 universities in 14 Latin American countries, issued a call to policy-makers to “avoid using the results of the rankings as elements in evaluating the institution’s performance, in designing higher education policy, in determining the amount of finance for institutions and in implementing incentives and rewards for institutions and academic personnel”.
I would – to a large extent – agree. Responsibly and transparently compiled rankings can, of course, have a very useful role in allowing institutions to benchmark their performance and to help them plan their strategic direction. They can inform student choices and help faculty make career decisions.
They can help governments to better understand some of the modern policy challenges of mass higher education in the knowledge economy, and to compare the performance of their very best research-led institutions to those of rival nations.
And yes, they can play a role in helping governments to select potential partners for their home institutions and determine where to invest their scholarships.
But they can only play a helpful role if those of us who rank are honest about what rankings do not – and can never – capture, as much as what they can, and as long as we encourage users to dig deeper than the composite scores that can mask real excellence in specific fields or areas of performance.
Times Higher Education is working hard to expand the range of data that it releases, and to allow more disaggregation of the ranking results and more nuanced analysis.
Rankings can be a valuable tool for global higher education – but only if handled with care.
* Phil Baty is editor of the Times Higher Education World University Rankings.