In June 2022, the Council of the European Union gave the green light for a European Agreement on research assessment reforms to go ahead. Plans for this initiative were proposed in a European Commission scoping report in 2021. Under the Agreement, research actors across European member states (including research performing organizations, funders, and national or regional assessment organizations) will be invited to sign up voluntarily and pledge their commitment to translate principles outlined in the report into local assessment reforms.

The initiative builds on ten years or more of campaigns movements to curb the misappropriation of research metrics in academic research assessment contexts, to broaden quality criteria and to change research culture more broadly. The fact that such an influential actor as the European Commission has taken up the research assessment reform baton suggests this agenda truly has landed, at least in certain policy spaces (see also recent reform efforts being coordinated within the Netherlands, Finland, and Norway) (Pölönen and Mustajoki 2022).^[1]

Following the decision to press ahead with the Agreement, I would like to reflect on the strategy the European Commission appears to be adopting for facilitating such reforms, and consider some of the opportunities and challenges their approach may encounter. Before doing so, I briefly outline what principles the European Commission considers to be central to more responsible and fair modes of research assessment.

The European Commission’s scoping report laments the current state of research assessment in Europe, which it states is driven by races for publications and citations, at the expense of quality, and which leads to a publish or perish culture that is damaging for research and researchers. The optimum method for research assessment is said to be qualitative peer review, and while the report adopts a critical tone towards uses of research metrics in research assessment - it does not completely reject their use as long as they are used appropriately in supporting (not replacing) qualitative decision making (citing, for example, the Leiden Manifesto). By signing the Agreement, research actors are effectively committing to ensure that their research assessments will:

• recognize and reward the plurality of contributions researchers make to academic life (not just publishing and bringing in grant money)
• respect epistemic differences between research fields
• reward new (or newly emphasized) quality dimensions such as open science (broadly defined), research integrity, and societal relevance, when evaluating individuals, institutions and research proposals.

As with the majority of efforts to initiate reforms of research assessment over the past decade, the European Commission Agreement is an example of ‘soft governance’, with the emphasis on steering from a distance rather than hierarchical imposition. The scoping report avoids giving precise prescriptions on how to implement reforms – rather it provides a broad vision and signposting to researchers and research organizations on how and why they should seek to redefine research quality, arguing that adhering to such values is commensurate with better (or more responsible) ‘academic citizenship’. To reiterate, signing the Agreement will be voluntary and not a legally binding commitment: while there may be reputational fallout if a signatory were to be seen as not acting within the spirit of the Agreement, there are unlikely to be formal sanctions.

How far can ‘soft governance’ take research assessment reform efforts?

A ‘soft governance’ approach like the Agreement has its attractions for different actors across the European research landscape: in a period of restricted economic growth and belt tightening around public funding, the Agreement requires little fresh money from the Commission, with individual signatories expected to self-fund their internal changes. The fact an actor like the Commission has so visibly put their weight behind this agenda will no doubt help local change agents when lobbying for their own research organizations to revisit current assessment practices. Likewise the lack of top-down prescriptiveness about what exactly the reforms should look like affords agency and flexibility to those on-the-ground in universities, funding bodies or assessment agencies to enact changes in a ‘bottom up’ way. On the flipside, one can envision potential limitations to this soft governance approach: lack of new, centralized funding means universities and other research actors may not wish (or be able) to invest their own scarce resources in enacting this agenda. Furthermore, the voluntary ‘opt in’ nature of the changes may well mean they can be easily ignored by those for whom the shared visions for change do not resonate. Soft governance mechanisms like voluntary agreements ultimately rely on their intended audience getting excited about the vision being set out or feeling a social pressure to conform, but if big players are seen as ignoring these calls, or organizations see them as too difficult to achieve, then the reforms will likely only be taken up sporadically by a handful of enthusiasts.

Another concern mentioned in the scoping report is the risk that research system actors in some member states will adopt reforms much quicker or enthusiastically than others, thus threatening potentially the logic of a common European Research Area (ERA), which envisions a single borderless market for research and for researchers. At national and local levels, influential actors with potential wrecking power may present opposition to the reforms. We have already seen criticisms being voiced against the Agreement from national research actors, for example in Germany, where an alliance of research organizations responded to the plans for the Agreement by stating they are against plans for a ‘harmonized’ European-wide assessment system and declaring they will not be signing up. Opposition and counter-reform movements are challenges for which champions of European research assessment reform should prepare themselves.

As such, there could well be a bumpy road ahead towards reformed research assessment practices becoming mainstream across the European Union. In Leiden, the need to make sense of assessment reform developments in Europe and beyond has prompted us to establish a Responsible Evaluation thematic hub in our institute, CWTS. Through the Responsible Evaluation Hub, CWTS colleagues will meet bi-monthly to discuss ongoing developments around research assessment reforms, including projects we are involved in, external initiatives, and to act as a sounding board for colleagues’ own encounters with ‘responsibility’ dilemmas. We think there will be plenty to discuss.

[1] Pölönen, J. and Mustajoki, H. (2022) ‘European recommendations on responsible research assessment’ Presented at EASST Conference, Madrid, July 6-9, 2022, Session on Responsible Research Assessment and STS, organized by A Rushforth, M Sienkiewicz, J Zuijderwijk, S de Rijcke.