What reforms in how we assess and value research are necessary to better equip public science systems for the existential challenges of the 21st century? How can we understand and tackle issues such as inequitable access to scientific literature, increasing strain on peer review systems, and publisher oligopolies? How best to evaluate emerging knowledge infrastructures and their embedded values, e.g. of openness or sustainability?

These are just some of the urgent puzzles members of the Evaluation & Culture Focal Area at CWTS are currently working on.

Over the years, CWTS has concerned itself with understanding and advancing many of the basic building blocks of academic knowledge-making, from assessment tools & evaluation practices to scholarly communication infrastructure and databases. Recently, there has been an explosion of science reform movements that address a wide range of problems in these domains. For example, researchers in metascience aim to tackle urgent questions of reproducibility and research integrity, while academic institutions and funders have begun to initiate various reforms to improve the fairness and inclusivity of research assessment and research culture. This includes for example attempts to rethink and broaden quality criteria in evaluation through responsible metrics or by means of so-called narrative CVs. In the Evaluation & Culture Focal Area we seek to understand and intervene in this rapidly changing landscape while practising new forms of communicating and valuing research in the daily work of our centre.

The focal area is divided into two complementary clusters: Reshaping Scholarly Communication and Changing Notions of Research Quality.

Our experts in Scholarly Communication will focus on the intertwining of evaluative practices and research culture in the distinct context of academic publishing. More specifically, the cluster will study the changing social, legal, economic and intellectual organisation of scholarly communication practices. Simultaneously, they will work to actively develop new infrastructures for open publishing and peer review, to be used across diverse research communities.

Meanwhile, the Research Quality cluster will expand the knowledge base about new movements, interventions and innovations in research assessment reform. Specifically, we will strive to better understand and intervene in new social formations in science that are seeking to re-shape research quality. Of key concern for this cluster is how efforts to institutionalise alternative forms of research quality fare and what consequences this has for how research is organised and valued. Some examples of such dimensions of quality are Equity, Diversity & Inclusion (EDI); open science; societal engagement; sustainability; and research culture.

Across these two clusters, we aspire to play an active and engaged role in “responsible research assessment transitions” on the local, national, and international stage, for example in the context of the Dutch Recognition & Rewards initiative or the Academia in Motion program at Leiden University. We will also contribute to aligning policy debates around research evaluation with debates around scholarly publishing. Siloing off these domains simply will not do: research assessment reform, after all, requires ‘whole system change’, with transformations in publishing practices needing to be accompanied by changes in funding, universities, research communities and other knowledge production and evaluation settings. One concrete setting in which we already combine our research and intervention activities across the two clusters is the Research on Research Institute, under whose umbrella we carry out a variety of projects co-designed with partner organisations in science funding.

Like CWTS’s other two Focal Areas, Engagement & Inclusion and Information & Openness, our group brings together different skills, passions and interests. Together our staff members engage with various stakeholders on multiple fronts, be it through shaping policy and organisational processes for the better, delivering state-of-the-art evaluations for clients through our contract research projects, or working on new research frontiers – always with an eye to their implications and practical relevance for the broader research culture.

If you are interested in exploring further opportunities for working with the Evaluation & Culture Focal Area, please feel free to contact one of our coordinators. For questions on our activities around Reshaping Scholarly Communication, please contact Wolfgang Kaltenbrunner. If you want to get in touch to discuss our activities around Research Quality, please contact Alex Rushforth, Andrea Reyes Elizondo, or Thomas Franssen. 

Current members of the focal area: Anestis Amanatidis, Rinze Benedictus, Louise Bezuidenhout, Sarah Rose Bieszczad, Carole de Bordes, André Brasil, Clara Calero Medina, Carey Chen, Eleonora Dagiene, Tom van Drimmelen, Soohong Eum, Thomas Franssen, Margaret Gold, Kathleen Gregory, Laurens Hessels, Myroslava Hladchenko, Andrew Hoffman, Tjitske Holtrop, Wolfgang Kaltenbrunner, Dmitrii Kochetkov, Kwun Hang (Adrian) Lai, Erin Leahey, Thed van Leeuwen, Ed Noyons, Ana Parrón Cabañero, Ismael Rafols, Renate Reitsma, Andrea Reyes Elizondo, Sarah de Rijcke, Alex Rushforth, Alexander Schniedermann, Marta Sienkiewicz, Jorrit Smit, Clifford Tatum, Ludo Waltman, Inge van der Weijden, Jia Zhang

Header image: SCRIBERIA

DOI: 10.59350/s1ja4-tca36 (export/download/cite this blog post)