The release of the first Open Edition of the CWTS Leiden Ranking in January this year triggered an overwhelming response. It was covered by various news media, including Research Professional, AEF info, and TheMetaNews, and was endorsed by important stakeholders such as the International Science Council, the French Committee for Open Science, and the National Agency for Research and Development in Chile. Université de Lorraine in France even explicitly referred to the Leiden Ranking Open Edition when it announced its decision “to withdraw from the commercial THE and QS rankings”. In the Netherlands, the Leiden Ranking Open Edition has been nominated and shortlisted for the Leo Waaijers Open Science Award.

For us at CWTS, the Leiden Ranking Open Edition plays a crucial role in our strategic agenda. One of our strategic goals is that “within the next few years, we want our work at CWTS … to be fully based on open research information”. To make openness of research information (e.g., openness of the metadata of research outputs) the norm, we co-led the development of the Barcelona Declaration on Open Research Information, published in April this year and already signed by over 80 research organizations worldwide. The key commitment made by signatories of the Declaration is to “make openness the default for the research information (they) use and produce”. The Leiden Ranking Open Edition demonstrates how CWTS is working on implementing this commitment.

Today we are proud to publish the CWTS Leiden Ranking Open Edition 2024. This new release of the Leiden Ranking Open Edition is based on the most up-to-date data from the OpenAlex database. Moreover, in this new release of the ranking, we have made a major enhancement to the web interface through which the ranking statistics are presented. To make the ranking as transparent as possible, the web interface now offers an easy way for users to contextualize the ranking statistics by exploring the underlying publications.

Below we introduce the web interface improvements made in the new release of the Leiden Ranking Open Edition. We also point out how the ranking statistics in the Open Edition and in the traditional Leiden Ranking can be compared, and we briefly discuss next steps in the development of the Open Edition.

What is behind the numbers?

At CWTS we are frequently contacted by universities that ask us for the data behind the numbers presented in the Leiden Ranking. For instance, universities tell us that they find it interesting to see how many of their publications are classified as highly cited, but that this information is of limited value if they cannot see the actual list of highly cited publications. In the traditional Leiden Ranking based on data from the Web of Science database, the agreement under which CWTS uses Web of Science data does not allow us to present information at the level of individual publications. In the case of the Open Edition, however, we use fully open data from the OpenAlex database, for which we do not face such restrictions.

Consider for instance Delft University of Technology. Suppose we are interested in the publications of this university in the social sciences and humanities (SSH). The Leiden Ranking Open Edition shows that in the period 2019-2022 6.1% of the SSH publications (co-)authored by Delft University of Technology belong to the top 5% most highly cited of their field and year. When we click on ‘6.1%’, the list of the underlying publications is shown (see Figure 1).

The list includes 249 SSH publications (co-)authored by Delft University of Technology that belong to the top 5% most highly cited of their field and year. Importantly, many of these publications do not have a full weight of one in the calculation of the Leiden Ranking statistics. These publications have a fractional weight because they belong to multiple main fields (not only SSH) and/or because they have been co-authored by Delft University of Technology with other organizations. Consider for instance the most highly cited publication in the list. As indicated by its citation score (CS), this publication had been cited 622 times by the end of 2023. However, the publication belongs not only to the SSH main field but also to mathematics and computer science, and moreover, of the 35 authors of the publication, only one is affiliated with Delft University of Technology. In the calculation of the Leiden Ranking statistics, the publication therefore has a weight of only 1/2 ✕ 1/35 ≈ 0.01.

In the traditional Leiden Ranking based on Web of Science data, we are unable to share the above information. To a large extent, the ranking presents black-box statistics. The Open Edition solves this problem, both through the improvements in the web interface and by making all data and source codes openly available. By offering full transparency, the Open Edition promotes more responsible use of ranking statistics.

Comparison with the traditional Leiden Ranking

The Leiden Ranking Open Edition 2024 includes the same 1506 universities as the traditional Leiden Ranking 2024 that was released earlier this year. To compare the ranking statistics in the Open Edition and in the traditional Leiden Ranking, we have created this interactive dashboard. The dashboard can for instance be used to explore the correlation between the number of publications of a university in the two editions of the Leiden Ranking (see Figure 2).

For most indicators, the correlation between the statistics in the two editions of the Leiden Ranking is quite strong. In some cases, however, there are substantial differences between the two editions of the Leiden Ranking. These differences may be due to errors in the traditional Leiden Ranking or in the Open Edition. They may also result from differences in the publications included in the two editions of the Leiden Ranking.

In an analysis performed earlier this year, we found that errors in assigning publications to universities are substantially more likely in the Open Edition than in the traditional Leiden Ranking. Improving the quality and completeness of the data in open scientometric databases is a key priority in the open research information community. It was discussed extensively at the recent Paris Conference on Open Research Information.

Next steps

Last January we expressed the expectation that within one or two years the Leiden Ranking Open Edition will be fully mature and will offer a full replacement for the traditional Leiden Ranking. This is still our expectation. We are getting increasingly close to the point at which the traditional Leiden Ranking can be fully replaced by the Open Edition.

While we are moving toward this goal, we will keep working on improving the Open Edition, based also on ongoing discussions about transparency of university rankings and openness of research information. We also very much welcome feedback from users of the Open Edition. If you have any comments on the ranking or suggestions for improvements, please do not hesitate to contact us using the contact form on our website.

We are grateful to our CWTS colleagues Hans den Os, Dan Gibson, Bram van den Boomen, and Martijn Visser for their contributions to the Leiden Ranking Open Edition 2024.