It’s Time to Move Beyond Rankings

The following is a guest post by Ellen Hazelkorn, vice president for research and enterprise and head of the Higher Education Policy Research Unit at the Dublin Institute of Technology. Her book Rankings and the Reshaping of Higher Education: The Battle for World-Class Excellence (Palgrave Macmillan) was published in March.
As I watch the deluge of press releases and media reports reacting to the latest rankings, I am reaching the point of despair. At a superficial level, we can all join in the ridiculous state-of-affairs as governments and universities vie with each other to promote their institutions and—in reality—their country as a place worthy of investment and talent recruitment. But there are more insidious currents at work.
A confluence of factors has heightened policy, public and student interest in higher education—at the organizational level, and also at the level of the individual faculty member and student. In the post global financial crisis world, calls for greater accountability and transparency are driving change across systems and institutions, in academic contracts, in service-level agreements with students, and with society at large. While higher education yearned for the time it would head the policy agenda, it probably never anticipated the intensity of scrutiny that would follow.
But, the growing influence and power of global rankings over higher education has reached the point that bears little relationship to calls for greater transparency and accountability. Indeed, despite amendments to their methodology, rankings have nothing to do with measuring quality—as opposed to wealth and reputation. Because a so-called world-class university is estimated to have an annual budget of $2-billion, governments are busy effectively buying themselves into the ‘new world order.” This explains some of the quite remarkable changes up and down the rankings scale, as well as the broad structural make-up of the Top 100 or Top 400.
At the same time, the rationale for annual publication of rankings can only be driven by commercial interests aiming to extend their publishing remit beyond their traditional customers. Similarly, Thomson Reuters’ Global Institutional Profiles project seeks to cleverly extend its market share of the rapidly-expanding and lucrative knowledge intelligence business. This involves universities freely supplying institutional data that is then monetized into various products. No one can begrudge commercial interests acting in their own self-interest. But governments are elected to represent the public interest.
Governments have responded to rankings in various ways. Many countries—as I’ve reported before—have used rankings to spearhead profound restructuring of their higher-education systems in the belief that the world-class research university represents the panacea for success in the global economy. France’s determination to establish its own “Ivy League” is the latest in a long line of policy maneuvers that are choosing to reward the achievements of elites and flagship institutions rather than improve the capacity and quality of the whole system.
There has also been a proliferation of national rankings, with many countries using rankings to assess and differentiate its higher education system—often with the help of the ranking organizations themselves. Southeast Europe and African nations have used rankings as a guide for investment levels and to set threshold standards. Arab countries have sought to establish a counter-ranking to better represent its unique educational perspective and purpose. The European Union (EU) has done likewise.
Angered by the way global rankings have ignored the diversity of institutions, downgraded teaching, and underestimated Europe’s research, the EU has sponsored the development of U-Multirank as the alternative global ranking. Addressing many criticisms of rankings, it uses techniques developed by the Centre for Higher Education (CHE) in Germany for the latter’s own ranking. Thus, U-Multirank is user-driven, multidimensional, multilevel and peer-group comparable. It calls itself a ranking, but proposes to band institutions rather than produce a league table.
The feasibility stage has just been completed, and the EU has sufficient confidence in U-Multirank that its latest communication last month, entitled “Supporting growth and jobs—an agenda for the modernization of Europe’s higher education system,” has announced Phase II with a launch sometime in 2013. Their objectives are both ambitious and laudable.
But, as with any new initiative, there are teething problems, inter alia: 1) Despite going beyond existing rankings, the choice of indicators remains problematic, and the absence of meaningful internationally comparative data means it suffers from many of the same problems afflicting other rankings; 2) U-Multirank aims to overcome the problem of measuring teaching quality/student performance by including data from the OECD Assessment of Higher Education Learning Outcomes (AHELO) project, which seeks to evaluate “what students in higher education know and can do upon graduation”; that project is also in early stage of development; and 3) The feasibility stage failed to garner sufficient global recognition, and for the moment, it is primarily a European instrument.
These challenges indicate that in the short or medium term, U-Multirank will be unable to counter the influence of global rankings.
It is, therefore, time for international organizations to take a stand—because I don’t believe individual countries or institutions can act unilaterally. Admittedly, governments have been at fault for failing to exercise the same due diligence they require of any other significant investment. But, realigning policies to conform to indicators, produced by others for commercial or other reasons, represents an abdication of national sovereignty.
Therefore:
- International organizations, such as UNESCO, OECD, World Bank, and EU, should come together to Say No to Rankings as the basis for policy making, and caution stakeholders accordingly. In the same way that governments issue “health warnings” against (for example) cigarettes, an unequivocal public campaign should be waged pointing out to governments the dangers of using rankings for such purposes.
- The EU should make an equally bold statement and reframe U-Multirank as a Benchmarking Tool. In this way, it will be formalizing its purpose as a tool to “help students make informed choices” and “support policy makers in their strategic choices.”  This would be a shrewd move enabling the EU to retake the political advantage.
Rankings have helped catapult higher-education up the policy agenda. But, using them as the basis for decision making, at the national or institutional level, is leading to perverse and unintended consequences, with long-term implications for society. The time has now come for leadership.
Voir aussi sur le blog: Are rankings driving university elitism?, Do rankings promote trickle down knowledge? (by Ellen Hazelkorn), « Hit-parade des universités: la France stagne au huitième rang du classement de Shanghai » et « Nous récoltons les fruits des efforts enclenchés dans l'enseignement supérieur », Les universités françaises à la peine dans le classement de Shanghai, New International Ranking System Has a DIY Twist, Les classements des chercheurs en question, Questions Abound as the College-Rankings Race Goes Global (by Ellen Hazelkorn),  International Group Announces Audit of University Rankings.