Funding data (and the funding database)

Funding acknowledgements are a familiar component of scholarly publications. In most cases, authors use this segment of the publication to list any grants, programs or other forms of financial support that made the research possible. In some cases, authors also use this space to express gratitude for any personal support they have received.

For the uninitiated, these snippets of text may seem innocuous, but in fact they reveal the behind-the-scenes workings of scholarly publishing. They are useful for funders who have a vested interest in accurately tracking the research output of their grants. They are also of interest to readers who are interested in which funding entities are responsible for what research.

Within CWTS’ A-team, we have developed the structure of the so-called funding database, the aim of which is to create a taxonomy of funding organizations, while also enabling more research into this nascent field. This in-house database is based on Web of Science funding acknowledgment data, which itself is based on processing the acknowledgment text sections and extracting the relevant funder names and grants. After some CWTS preprocessing, these data can then be used to map the previously opaque support network which underlies the scholarly publishing space.

The new player in town: Dimensions

In addition to Web of Science, a new source of funding data has become available in recent years: Dimensions. Part of the ensemble of Digital Science products, Dimensions uses a comprehensive methodology to acquire funding data. According to its website, Dimensions employs a three-pronged approach to creating the connection between publications and grants:

1. Mining and extracting the funding acknowledgement section
2. Receiving information directly from funders
3. Extracting connections from PubMed and CrossRef

Furthermore, Dimensions identifies funding organizations using GRID, Digital Science’s immense global database of over 100,000 research-related organizations. The differing methodology combined with the huge scale of GRID make Dimensions a potentially interesting source of funding data, which could serve to complement or provide quality control on the already existing database.

Case study: the NWO

In order to compare the Dimensions funding data to our own, we examined the research output and related grant data of the largest funder in the Netherlands, NWO. As noted on the NWO website, the NWO-I (the institutes organization of the NWO) manages 9 national research institutes, and there are an additional 8 suborganizations under the NWO.

To properly compare the two databases, we need to make sure that we are making a fair comparison. Web of Science only began collecting (or at least making public) funding data from 2008 onwards, and has collected pre-2008 acknowledgments only marginally. Dimensions on the other hand has no such cut-off point in their collection. The comparison can be seen in Figure 1.

Figure 1: Count of funding acknowledgements in each database by year.
To make the comparison sensible, we limit our comparison to publications from 2008 and onwards. Additionally, we only include publications which are indexed in both databases, since many publications (and thus funding acknowledgements) are unique to each database, as shown in Figure 2. The databases are connected through stable identifiers, namely DOI and PubMedID.

Figure 2. Comparison between Web of Science and Dimensions from 2008 onwards (*Unique number of funding acknowledgements from each database, from 2008 onwards)
In general, when examining the funding database, one needs to take the level of granularity into consideration. In other words, a decision needs to be made beforehand about whether it is more interesting to know about the total publications of the NWO as a whole (Figure 3), or about the publications resulting from each suborganization (Figure 4). This decision depends on the aim of the research. For instance, the former case may be more useful when comparing the national research funders of different countries, while the latter may be more interesting when analyzing the funding outcomes of each individual suborganization.

Figure 3: The parent-level allocation, where the funding acknowledgements are aggregated at the parent organization level. In this sample, the NWO is marked as having funded 3 publications.
The parent allocation is possible due to the relations between organizations gathered on CWTS’ internal organization registry. These relations are maintained and updated by the A-Team.

Figure 4: The child-level allocation, in which funding is analyzed under the lens of the individual sub organizations. In this sample, both the CWI and NWO are marked as having funded 2 publications.
The amount of funding acknowledgements attributed to each of the suborganizations in the different databases is shown in the table below. In the ‘Aggregate Count’ column, only the main parent organization is considered (as in Figure 3). In the 'Detail Count’ column, organizations are counted if they appear in the original acknowledgment (in other words, the child institutes are here counted individually as funders). As such, we can see that the funding acknowledgements are more distributed using the ‘Detail Count’. We can also note that there are more funding acknowledgements attributed to NWO in the CWTS database than in Dimensions.

The discrepancies in total funding acknowledgements between the first two columns, CWTS’ two different counting systems, do beg the question: how should one attribute the credit for the funding: to the parent or the child institute? Additionally, how does one define the parent/child relationship? For instance, ZonMw (The Netherlands Organization for Health and Research and Development), and the NWO Domain Applied and Engineering Sciences do not contribute to the total count of the NWO because they are considered as independent funders in our database. This type of classification implies a different type of role and function for ZonMw compared to the other suborganizations of the NWO. This is an ontological issue, one which the A-Team deals with on a semi-regular basis. Ultimately, there is no definitive acknowledgment count; instead, the answer to the query depends on a case-by-case basis, based on what the researcher or funding agency is looking to find out. Our job is to make the database as capable as possible to deal with all such possible queries.

For a more detailed examination, we now look at two of the publications attributed to suborganizations in the Detail Count, but not featured in either of the other counts.

Here, the child institutes are directly attributed. In the first example, it would be impossible to attribute the publication to NWO funding without prior knowledge of the relationship between child and parent institute. Yet, should who gets credit really depend on the specific phrasings used in the acknowledgment section?

How to move forward: efforts at standardisation

Most likely, the difference in numbers above also stems from the databases’ different approaches in obtaining their data: one goes through the acknowledgment text while the other (also) gets its data directly from the funder itself. While we will continue to work hard to follow the funding streams and improve the accuracy of our data, funders, researchers and journals can also do their bit to improve the situation.

One recent development in acknowledgment practices has been for funders to stipulate in more precise terms how researchers need to acknowledge their funding. The NWO, for instance, gives the following template:

This publication is part of the project [name project] (with project number [insert project number] of the research programme [name research programme] which is (partly) financed by the Dutch Research Council (NWO).

Such templates might serve to shift back control to funders on which of their grants are acknowledged on which level. In the above case, if NWO wants funding credit to go to a specific institute, it can change the template. Moreover, it also urges researchers to include more detailed information like project names and numbers, which can ultimately help databases to more accurately track funding streams.

In order for such templates to work, researchers need to do their part by consistently following them. If a template does not exist, being accurate and complete goes a long way: include the official funder name and anything trackable like project and grant numbers.

Journals too have started to aid the process by providing dedicated funding fields in their application forms of their submission systems, as for instance the PLoS journals do. This way, financial support gets clearly distinguished from acknowledgments of gratitude (for instance).

These templates and alternative acknowledgement practices are steps towards answering the eternal question in funding acknowledgement research: to count or not to count.

Part 1 of the series on Funding Acknowledgements in Academic Publishing