"Welcome to the quackathon!". That is how we invited researchers from the CWTS to an afternoon workshop during our yearly first research retreat. But what on earth was that supposed to mean? The reference to a hackathon is fairly obvious. In a hackathon, programmers come together to jointly work on a computer problem and try to solve it during the meeting. We have programmers at our centre. But we have also researchers studying scientific research and its relation to society – both qualitatively and quantitatively.

In the previous months, while preparing our research retreat, we discussed the possibility of doing something around the theme of research methods. We did not want yet another abstract discussion about the respective merits and pitfalls of qualitative and quantitative research. Neither did we want a sterile exercise with clearly defined questions. What we aimed for was to let people actively collaborate on a relatively open research problem that had to be 'solved' at the end of the meeting. That is how the quackathon was born.

By the way: a quack is also an imposter who does things under the pretence of science – like the alchemist of earlier days. We distance ourselves fully from any negative connotations. But why not find out where a little methodological alchemy could lead us?

Organising the quackathon required some preparatory work. We needed a theme that was broad enough to speak to people with different backgrounds and interests, we had to find qualitative and quantitative research material to explore during the event, and we would have to provide some guidance on possible research questions.

Thematically, we decided to focus on displacements in science. Goal displacement springs to mind here (an organization might start with one goal but find itself pursuing another later on). But we could just as well speak of linguistic displacement in cases where it was once common to write in one language but where another language now dominates. Or of methodological displacements when there are shifts in the way certain scientific objects – say, 'oceans' or 'the economy' – are studied.

Preparing the material was an iterative process. We selected some qualitative material on economics on the one hand and ocean science on the other hand. In an interview setting, for instance, a policy-oriented economist reflected on the way politicians used economic knowledge and on the public engagement of economists. In another example, a marine biologist reflected on the place of impact factors and the distribution of prestige in his field. To complement the interview excerpts we collected bibliometric data of the two fields. We prepared some science maps, providing an overview of various characteristics, highlighting for example what topics were researched by a certain institution, or what topics were published in high-impact journals (see the visualization below for an example). We also just provided the tabular data, allowing for more quantitative analyses. And we went back to the qualitative data: should we perhaps add more to clarify the perspective of the interviewees?

Visualization of publications in economics with different citation scores

We split up all participants in four teams, each of which had an even mix of quantitative and qualitative scholars. We then asked participants: what can you do on the spot – discuss, code, analyse, count, calculate, model – with the provided material? How would you design a new research project in which qualitative and quantitative methods feed back into one another? What have you learned along the way in terms of your own limits and the insights you gained from others? Each team freely pursued their own interests and ideas. Some came up with ideas of how to use quantitative results to direct future interviews. Others constructed interactive visualizations or even wrote a blog post on the spot. There were even a few cases of digital ethnography taking off.

The interview excerpt with the marine biologists gave enough food for thought: Was taxonomy indeed on the demise? Was mathematised science more highly valued? The bibliometric data did not provide any definitive insights, and most teams quickly learned that some of the data was heavily biased by selective data gathering. Searching further, one team found online discussion forums about scientific career prospects in marine biology, where people were worried about ending up in a job where the of fish was all there was to it. And it was acknowledged that a more scientifically rewarding career required "someone who can write code, who is great with stats, or who is handy at signal processing." Though the evidence was (naturally) inconclusive it was possible to tentatively carve out relations between quantitative and qualitative findings (see the poster to the left).

The policy-oriented economist suggested in the interview that national knowledge about the economy was floundering. The bibliometric record showed that authors seemed to discuss their own countries more frequently. At the same time, the number of different countries mentioned only increased, suggesting a broader orientation, instead of a narrower focus on for example the US economy. Zimbabwe and Venezuela were for example frequently mentioned, possibly because of their economic predicaments.

Did the quackathon lead to transformative new insights? Did it integrate all of our approaches and are we now on a joint path towards studying science? No. However, nor do we consider this a failure of the exercise. Bridging the quantitative-qualitative divide is not a matter of resolving all our differences and finding that one unique pathway to enlightenment. Both approaches should remain viable and valid scientific methods of inquiry in their own right. However, we can enrich our understanding and sharpen our thinking by regularly exchanging our views, contrasting our evidence and confronting our own assumptions. Our first quackathon was successful in opening this discussion. And we are confident that quackathon 2.0 will play a part in keeping it going.