Dissecting world university rankings is a complicated and intricate process

With the release in April of both the QS World University Rankings by Subject 2022 and the 2022-23 edition of the Global 2000 list by the Centre for World University Rankings (CWUR), universities across the globe are again riding the media wave to boast about their ranking.

If they fared better, it becomes the marketing campaign for the next month; if not, they quietly take note thereof. As a US Ivy League university president commented some years ago: “we don’t care much about rankings, but we are so pleased that we are in the top five globally”.

So too is the reaction of the media and the public when such rankings are announced, and questions abound on why some rise, others hold their position, and a few take a drop. Globally, rankings have become an important indicator mainly for prospective students, but also for partners, collaborators and potential funders of an institution’s quality of research and teaching and learning.

Last year saw the release of a new book titled Research Handbook on University Rankings: Theory, Methodology, Influence and Impact, in which the authors give a comprehensive review and analysis of the impact of global university rankings since their establishment in 2003. In an article by these authors, Ellen Hazelkorn and Georgiana Mihut, in University World News (UWN) in January 2022, university rankings are defined as capturing “the zeitgeist of accelerating globalisation and the global battle for talent and increased policy and public focus on performance, quality and accountability”.

The authors discuss how geopolitics are reshaping the higher education landscape, delve into the business of rankings, and dissect indicators and methodologies of rankings. For me, the latter provides the context within which we — universities, the public and the media — should respond to university rankings.

There are several world university rankings, with some having sub-rankings. Besides QS and CWUR, other well-known rankings include Times Higher Education (THE), CWTS Leiden Ranking, and the Academic Ranking of World Universities (ARWU, or Shanghai Rankings). Only some rankings require institutions to participate formally, while others rate universities on their own accord and mainly use data extrapolated from existing databases to which they have access.

All rankings use different methodologies and data sets to determine the positions of universities on their ranking, meaning their criteria differ. For instance, THE requests certain data from participatory universities and uses that to rate institutions, while QS gathers most of its data through stakeholder surveys (therefore, as much as 40% is based mainly on perception). Also, THE uses, among others, data on teaching, research and citations, while the QS ranking focuses on academic and employer reputation, among other indicators.

Within these different methodologies, there are also varied definitions of fields, such as who is considered (and not considered) to be staff members or students. These differences in methodologies and data sets explain why a particular university could fare well on one ranking but poorly on another, and why the interpretation of definitions differs among universities.

https://www.dailymaverick.co.za/article/2022-05-04-unions-demand-10-increase-in-public-sector-wages-setting-the-stage-for-a-bitter-battle/

An analysis by Stellenbosch University (SU) in 2020 revealed that universities within South Africa interpret definitions differently and therefore do not always submit the precise comparable data.

In their article for UWN, Hazelkorn and Mihut view such issues relating to the different methodologies and choice of indicators as the most regularly critiqued aspect of rankings. Also, they argue that we do not yet fully understand what would constitute high-quality higher education, and we are as yet unable to determine how quality in teaching and learning, internationalisation, social impact, or aspects of equality and diversity can be measured.

These are among the pertinent aspects that are not easily defined or can be measured that may have a significant impact on a university’s position in rankings. Today, many of these “intangible” aspects of quality higher education are some of the deciding factors for prospective students or partners.

This is not to say that rankings are not becoming more sophisticated. In fact, THE has established its Impact Rankings, which consider universities’ support of the United Nations Sustainable Development Goals. However, institutions must choose and opt-in to participate in this Impact Ranking. Often universities use the results of this ranking for marketing purposes, without mentioning that they might be one of only two or three universities in their country that opted to participate in this particular ranking.

Our vision at SU is to be Africa’s leading research-intensive university, and as such the African context matters immensely. Universities on the African continent’s social, economic and political contexts differ vastly from those of universities in high-income countries. This we saw especially during the Covid-19 pandemic when the digital divide became apparent, with many African universities struggling to go fully online because of a lack of infrastructure for connectivity.

Also, African universities experience declining funding at home and from international funders, while also having to be large-scale agents of change in local communities. In such circumstances, providing quality teaching and learning while producing globally competitive research becomes challenging.

Ranking agencies are not ignorant of these aspects, and THE’s BRICS ranking aims to address the disparity in global universities’ contexts. The fact that African universities are still able to feature on the fiercely debated international rankings stage is no mean feat. This should be applauded, especially when one considers the playing field.

Although Google would say there are as many as 25,000 universities worldwide, on average about 1,500 universities participate or are included in a particular ranking. Regularly, new institutions join these rankings, which impacts your standing — an institution might drop on a ranking, not because it fared worse but because a “better” institution participated or was included in a particular year.

One should therefore rather consider how universities fare over a number of years and not in a particular year, as well as look at their performance across various rankings to get some idea of their overall global standing. From this perspective, SU has been quite stable in various rankings over the past few years.

World university rankings remain at best a complicated matter impacted by various factors and contexts if one wants to come to a realistic conclusion on the quality of a specific university’s standing within global higher education.

But despite the intricacies and perceptions around rankings, they will remain part of the academic currency of our time and will in future increasingly be used by prospective students and partners as a means of judging an institution’s quality and credibility.

As universities, especially on the African continent, we must continue to advocate for a broader public understanding of what these rankings mean, what their impact is, how they sway public perception about our institutions, and how they position our universities within the global knowledge economy. DM

Prof Hester Klopper is Deputy Vice-Chancellor: Strategy, Global and Corporate Affairs at Stellenbosch University.