Scholarly book evaluation often prioritises ‘prestige’, which leads to inconsistent and unfair outcomes. My previous research shows that such systems consider neither the intrinsic quality of the research nor the accessibility of the work itself. This is why I have cautioned against judging books by their publishers in multiple posts here, here (in English), and here (in Lithuanian).

After all, how impactful are groundbreaking findings if they remain locked away behind expensive paywalls or inaccessible formats? This disparity demands innovative and responsible metrics that assess both quality and availability, promoting fairer and more effective research evaluation.

To address this challenge, I embarked on a research journey to understand who publishes the books submitted for evaluation in different countries, including my home country of Lithuania.

Specifically, I wanted to know whether Lithuanian policymakers reached their goals by financially incentivising publication with ‘prestigious’ foreign publishers for decades. For this, I needed to identify the core businesses of the book publishers as well as the countries where researchers published their works.

I also wanted to compare Lithuanian results with those of other countries. Thus, I obtained from the Research Council of Lithuania the lists of books submitted to the annual research assessments along with their ISBNs (International Standard Book Numbers).

Two further data sources were necessary: first, the lists of books submitted for research assessment in another country for comparison, and second, a source of reliable metadata for these books.

But finding suitable data proved challenging. First, no single database covers all publications across all countries. Second, not every country has a complete list of national outputs. Fortunately, the UK’s REF (Research Excellence Framework) submissions are available to download and explore freely.

Then came the key breakthrough: I uncovered the power of ISBN codes and their metadata within the Global Register of Publishers, maintained by the International ISBN Agency. Utilising data from this registry, I not only found metadata for every ISBN in my datasets but also gained valuable insights into the publishing landscapes and researchers’ publishing habits in both the UK and Lithuania.

Before diving deeper into the challenge of assessing academic books, let’s first review the potential of ISBN codes and the Global Register of Publishers for answering questions about scholarly publication norms and trends.

Unveiling the book publishing world with ISBNs

Originally designed for book supply chains, ISBNs hold surprising potential for research evaluation because they embed a publisher prefix (Figure 1). This acts like a fingerprint, allowing us to identify the true publisher behind the book, unlike the sometimes-misleading information on copyright pages.

For example, a book might list a specific imprint on the copyright page, but the ISBN prefix reveals it belongs to a larger publishing group. If experts judge books by their publishers, this hidden information can be crucial for accurately assessing research outputs and ensuring fairness across institutions.

And the best part? The publisher’s information is readily available through the Global Register of Publishers, a free database collecting ISBN data from over 150 agencies in 200 countries. As a registered user, anyone can access details about publishers’ main activities, which was crucial for my research.

When I examined the ISBN data for the UK and Lithuanian books in my dataset, I discovered a complex world of imprints and multiple brands that was often at odds with the statements on the copyright page. For example, a single publisher like HarperCollins might use the same ISBN prefix for dozens of different names, making it hard to tell who is truly behind the book.

The business dynamics of the publishing world add a further layer of intrigue to the story. Publishers merge and acquire competitors, and some imprints move back and forth between publishers. The same imprint may change hands multiple times and even be spun off as an independent company. This shifting landscape highlights the limitations of traditional evaluation methods that rely solely on publisher reputation.

But what does this mean for fairer and more effective research assessment? My analysis of ISBN data offers surprising insights about book publishers and their relationships with researchers.

An uneven playing field: Open access vs. prestige in scholarly book evaluation

The metadata of book ISBNs from the UK and Lithuania revealed a disparity between national policies and publishing practices. While the UK prioritises open access, promoting accessibility over publisher prestige, some universities still emphasise established publishers for REF submissions. This contradiction between policy and practice might explain why the REF claims neutrality towards publisher standing. Since my study focused on publishing behaviour only, further research on book accessibility would be valuable to explore the proportion of REF books providing open access.

Lithuania, on the other hand, explicitly encourages publishing with prestigious foreign academic publishers by offering financial incentives. Policymakers assume that prestigious publishers guarantee wide dissemination. No policy mentions accessibility. However, my research shows these incentives have not been effective. To the contrary, many Lithuanian academics struggle to access these coveted top-tier publishers, and a significant portion of their books are published domestically or through self-publishing channels.

The data below illustrates the uneven playing field:

• For the UK’s REF books, the concentration of renowned publishers potentially limits accessibility due to paywalls, despite the national open access policy (Figure 2).

• For Lithuanian books, smaller occasional publishers collectively publish more books than the top ten, but some might lack the resources to ensure accessibility through various formats (Figure 3).

Who publishes these books? UK scholars rely on established academic publishers, while Lithuanians show a shift towards universities, self-publishing, and firms or institutions whose core business lies outside publishing (Figures 4 and 5). This seems to indicate challenges for Lithuanians in securing a prestigious publisher, despite the financial incentive to do so.

Where do researchers publish? Analysing ISBN data shows that UK researchers primarily publish in Europe, predominantly with domestic publishers (Figure 6). In contrast, Lithuanian scholars currently publish more books abroad, with domestic publications declining since only foreign publishers can be awarded the ‘prestigious’ label (Figure 7). This may seem to suggest that Lithuanian policymakers succeeded in incentivising publisher prestige, but Figure 5 shows that academic publishers produced only a quarter of Lithuanian books over 12 years, which casts doubt on the policy’s effectiveness.

Both countries exhibit a disconnect between policy and practice. However, the UK's REF mandates open access from 2024 onward. This policy update should be a boon to accessibility in UK scholarly publishing. It also offers a framework for Lithuania and other countries to prioritise accessibility alongside merit of scholarly books.

Advancing research evaluation: The scientometric potential of ISBN data

Policy recommendations
Based on my findings, I recommend a multi-pronged approach to address the identified issues in Lithuania's book evaluation system.

Firstly, since most Lithuanian books were produced by universities and non-professional academic publishers, Lithuanian policymakers should stop incentivising publisher prestige alone. Instead, they can pilot an open access policy for research outputs, similar to the UK's REF, to ensure wider accessibility alongside merit-based evaluation. Notwithstanding potential concerns about cost or implementation complexity, the long-term benefits in terms of transparency and efficiency should be carefully considered.

Secondly, exploring the adoption of a system that leverages ISBN codes and the Global Register of Publishers for book metadata could streamline data collection and provide valuable insights into the publishing landscape.

Contribution to scientometrics
By unlocking the potential of an important but underutilised data source, my work opens new avenues for scientometrics, policymaking, and book metrics development.

Scientometricians can now access reliable and comprehensive book metadata, enabling them to conduct more accurate and nuanced analyses of each country’s research outputs. Policymakers, meanwhile, can learn about the practicalities of the book industry, guiding the development of more effective research evaluation practices. Developers of book metrics can leverage these findings to create new and improved indicators that reflect the diverse landscape of scholarly publishing.

Future research
My work also suggests several key research questions to further refine our understanding of scholarly book evaluation and publishing practices.

Firstly, a comprehensive analysis of the accessibility of currently submitted books in Lithuania is needed. This would involve identifying the available formats and platforms for accessing these books and assessing any barriers that might exist.

Secondly, exploring the availability and potential sources of additional metadata for national research outputs is crucial. This could entail investigating data from national libraries, repositories, and other book-relevant sources.

Finally, it is essential to determine the feasibility and impact of incorporating ISBN-based metadata into existing book evaluation frameworks. Research along this line would involve pilot studies and comparisons with traditional methods to assess the effectiveness and potential benefits of this approach.

By addressing these key questions, future research can build upon the findings of my study and contribute to the development of more robust and equitable book evaluation systems.

I sincerely appreciate the help of Stella Griffiths, executive director of the International ISBN Agency, who not only answered practical questions about the standards and their application but graciously took the time to review the manuscript "The challenge of assessing academic books: The UK and Lithuanian cases through the ISBN lens." Her insightful suggestions greatly improved the clarity and precision of my research on ISBNs.

Header image: marouh
DOI: 10.59350/j5484-3zf84 (export/download/cite this blog post)