In June of this year, the Open Research Funders Group (ORFG) published an open letter to the wider academic community with a call towards improving research output tracking. Funding acknowledgments were a particular focal point. In the previous blog post in this series, we already addressed several issues at play in funding acknowledgment data sets. In this second installment, we examine a specific real-life example of one funding acknowledgment within which many of the issues presented in that previous post come together. The article in question was published in the Journal of Molecular Biology, with the research being primarily conducted at the College of Medicine, Florida State University.

In this blog post, we suggest a taxonomy of different types of acknowledgment, all of which co-occur regularly in acknowledgment sections of published articles. In the reproduced acknowledgment section below, these different types are color-coded. We will show how different bibliographic databases parse and extract funding agencies and/or grant numbers from this text, and how they can all end up with a different result. By doing so, we would like to make clear that assigning funding agencies to publications is a process with a level of ambiguity, a process that relies at least partly on interpretation.

ACKNOWLEDGMENTS

We thank Dr. T. Somasundaram (X-ray Crystallography Facility) and Dr. Claudius Mundoma (Physical Biochemistry Facility, Kasha Laboratory, Institute of Molecular Biophysics) for valuable suggestions and technical assistance. We also thank Ms. Pushparani Dhanarajan (Molecular Cloning Facility, Department of Biological Science) for helpful comments. We acknowledge the instrumentation facilities of the Biomedical Proteomics Laboratory, College of Medicine. This work was supported by grant 0655133B from the American Heart Association. The use of the “mail-in crystallography” facility of the Southeast Regional Collaborative Access Team for diffraction data collection is acknowledged. The use of the Advanced Photon Source was supported by the US Department of Energy, Basic Energy Sciences, Office of Science, under contract no. W-31-109-Eng-38. All X-ray structures have been deposited in the PDB.

In the above text, we identify three main types of acknowledgment:

• Personal acknowledgment: Individuals being acknowledged for their roles in the research process
• Financial support: Reference to grants, contracts, or any phrasing that elucidates the financial component of the acknowledgment (terms like “funded”, “financially supported”, etc.)
• Logistics acknowledgment: Reference to the use of facilities, materials and/or machines used in the research process

When bibliometric databases count funding acknowledgments, it is important to accurately make the above distinctions, since different analyses require inclusion and exclusion of different acknowledgment types. This can already be difficult to ascertain when the wording in the acknowledgment text is ambiguous (i.e., is “Author A was supported by Harvard” a financial acknowledgment?) or when the logistic support comes with a grant or contract number.

That latter distinction, between grant and contract, is important to note on its own terms:

• Grant: in referencing grant 0655133B from the American Heart Association, the authors are referencing a specific grant number.
• Contract: the authors also reference Contract No. W-31-109-Eng-38, a contract awarded by the US Department of Energy, Basic Energy Sciences, Office of Science.

These types of financial support are distinct in their structure and function. While the case can be made for either type to be included as a funding acknowledgment, adding them up without distinction can feel like comparing apples and reds. Whether the financial support counted pertains to researchers’ salaries, a tailored grant, or a large government contract for a laboratory can be an important difference depending on one’s analysis.

Different interpretations from bibliometric databases

The idea that acknowledgment texts are highly interpretable can be observed in how different databases capture the acknowledgments of this example paper. The tables below show the resulting funding organizations when using the Web of Science, Dimensions, Scopus and Crossref databases respectively.

When comparing these results with the original acknowledgment text, the different databases clearly exhibit varying degrees of accuracy.

Dimensions and Scopus provide the most comprehensive results, identifying financial support from both the US Department of Energy and the American Heart Association. In comparison, Web of Science only identifies the US Department of Energy and Crossref only identifies the American Heart Association.

In addition to this, the results from Dimensions also list the Argonne National Laboratory which is not mentioned in the acknowledgment text. After further investigation it was found that the Advanced Photon Source is located at Argonne National Laboratory. This is a complicated case precisely because it can be partly categorized as logistical support (“Use of the Advanced Photon Source”) and partly financial (the mentioned contract). However, we would argue that having the Argonne National Laboratory as the funder here is debatable since the funding that is in play here is from the US Department of Energy to the Argonne National Laboratory. Arguably, the support from Argonne National Laboratory is logistical, while the support from the US Department of Energy is financial yet of a more indirect type than the American Heart Association.

Furthermore, there is also a variability in the identification of grant numbers by the different databases. Web of Science and Scopus only find a grant number for the US Department of Energy, whereas Dimensions and Crossref only find a grant number for the American Heart Association.

Extracting and identifying funding acknowledgements

The variation in the results can be partly explained by the different approaches taken by the databases for the extraction and identification of funding acknowledgment data.

Web of Science extracts and displays the raw (source) text from the acknowledgment text, while Dimensions only displays the resulting assignment to an organization in the GRID database it uses. To end up with this assignment, Dimensions uses a combination of “text-mining the ‘Acknowledgments’ or ‘Funding’ section” and “structured metadata received from some of our publication data sources, e.g., Crossref or PubMed”.

The funder registry used by Crossref was donated by Elsevier, and is combined with acknowledgment and grant information provided to Crossref by publishers and funding agencies in order to connect the funders in the registry to publications. Scopus extracts organizations from the acknowledgment text and matches them to the Crossref funding registry.

These methodological differences can result in the potential loss of discerning information regarding the identification of funding organizations. For example, in the acknowledgment text the reference to the US Department of Energy consists of three parts: the funding agency (US Department of Energy), the funding programme (Basic Energy Sciences) and the programme office (Office of Science).

Web of Science identifies this information as one funder, including the hierarchical structure of the organization within the name. In comparison, Scopus lists these different levels as separate organizations, resulting in the identification of three funders where there should only be one.

It is also important to note that the relationships between the organizations are not documented within the funding registry created by Web of Science. The hierarchical relationships between different organizations are available within the Crossref and GRID registries used by the other databases, allowing organizations to be aggregated into higher-level entities.

Conclusion

These types of discrepancies in funding acknowledgments are not rare. The standards and practices of what or whom to acknowledge vary depending on the field of study, with cultural factors also playing a role.

The above example illustrates certain nuances that one needs to be aware of when analyzing funding acknowledgment databases; each database captures and interprets the data differently and the exact nature of the acknowledgment is not always obvious. Moreover, it also shows that counting acknowledgments is not a straightforward task, with one paper resulting in either 1, 2, 3 or 4 acknowledgments depending on which bibliographic database you refer to.

It would be helpful if bibliographic data providers were more forthcoming on the decision processes that lie behind such results as shown above. How do they distinguish funding acknowledgments from other types of acknowledgments? Why do they include certain mentioned organizations in the text and ignore others? For some data providers, it is also not clear whether the connected funder is extracted from the text or derived from other sources, such as data provided by the publisher or funder. Here, too, transparency would be helpful for those using the data.

Finally, though one standardized format of writing acknowledgments could never fit every case, as mentioned in the previous post, using persistent identifiers and templates provided by funders can help to streamline the processing of this data.

Part 2 of the series on Funding Acknowledgements in Academic Publishing

Photo by Markus Winkler on Unsplash