New International Ranking System Has a DIY Twist

By Aisha Labi. The highly competitive field of international university rankings gained a new entrant on Thursday, with the unveiling of a benchmarking system that its designers say could provide a revolutionary method for influencing the development of higher-education systems throughout the world.
The new tool, known as U-Multirank, focuses on some of the same areas that the best-known international rankings do, but allows users themselves to determine how much they want to weight each factor. Frans van Vught, one of the leaders of the project, which is being supported by the European Commission, described it as "a new instrument to create personalized rankings." U-Multirank was built around the idea that there is no such thing as an objective ranking.
Continental European universities have long complained that they are disadvantaged by the dominant international rankings, such as the tables produced by China's Shanghai Jiao Tong University and Times Higher Education, in which American and British institutions tend to occupy the majority of the top slots. Some critics believe that ranking systems, in measuring universities' comparative research productivity, they put too much weight on the publication of articles in journals, most of which are in English.
U-Multirank relies on indicators in five subject areas: teaching and learning, research, knowledge transfer, international orientation, and regional engagement. On Thursday, developers unveiled the results of a two-year feasibility study involving 159 universities from around the world, two thirds of which are in Europe. The project generated interest from most countries, with two notable exceptions. Few universities in the United States or China were willing to take part.
"American universities are largely focused on the American system and their own performance in the American system, and do not care so much about the rest of the world," Mr. van Vught conjectured. Just five American institutions ended up completing the institutional questionnaires. Chinese universities, which require permission from the national ministry of education to take part in such a project, were also unwilling to participate, and only one Hong Kong institution completed the questionnaire.
The system's Web-based interface was demonstrated at a conference in Brussels by users representing a student, a university administrator, and a business official. Each showed how they would select indicators that mattered to them, such as the student's focus on student-staff ratios or the business representative's interest in dropout rates or business-studies programs. After the conference, Mr. van Vught described complaints from at least one audience member that the system seemed far too complicated as "surprising" and said that efforts would be made to make the interface more user friendly.
U-Multirank was developed by a consortium of European organizations led by the Center for Higher Education Policy Studies at the University of Twente, in the Netherlands, and the Center for Higher Education Development, in Germany. The consortium hopes to unveil the complete system in the fall and will incorporate feedback from Thursday's conference as it continues its work.
Developers emphasized that the results that were presented on Thursday are not yet final and that there remain areas that need fine-tuning. For example, they noted that it has been challenging to find indicators to effectively measure regional engagement. Ellen Hazelkorn, a leading expert on rankings from the Dublin Institute of Technology, who attended the conference, described the system in an e-mail message as "a good start," although she emphasized that the chosen indicators of quality "remain problematic and controversial."
Ms. Hazelkorn has been critical of the increasing influence of rankings in shaping higher education policy in many countries. "I suspect the real challenge remains the fact that in the absence of a fully credible model—which is probably many years away," she said, "governments will continue to make profound decisions about their higher education systems based on the existing hyperbole about global rankings."