Academic institutions in the Netherlands and elsewhere are under pressure, with worries about the quality of scientific work and the validity of the knowledge it produces in an era of ‘post-truth’ (M'charek 2017; Sismondo 2017a, 2017b). Recent scandals over flagrant data fraud in Dutch scientific institutions, particularly in medicine and social psychology, have fuelled these concerns, and policy makers have sought to improve ‘research integrity’. This concept condenses a set of concerns about scientific credibility, whether in terms of outright fraud or plagiarism, or what has been termed ‘questionable research practices’ or sloppy science (Davies 2018; Davies and Lindvig 2021; John et al. 2012; KNAW 2012; Schimmack 2021). Sloppy science practices are more ambiguous and refer to more or less accepted ways of ‘cleaning up’ data in quantitative research.
Taking a fresh look at the ‘trouble with academia’, we present what Dutch academics from various disciplines have to say about research integrity (Felt 2017b), what they do to create and safeguard what they see as ‘good science’, and where they find this runs into problems. We use the notion of science in the broadest possible sense, to denote any academic discipline, and to show the concerns that trouble academics across disciplines and countries. The scientists’ understanding of this problematic, we will show, differs in vital ways from that of policy makers and integrity researchers and the measurements they developed and have implemented. In short, Dutch science policy understands scientific work in terms of individual excellence or misconduct and governs the sciences through funding competitions that are seen as a means to spur and shape excellent research, through agenda setting and stimulating financing through collaboration with industry (De Jong et al. 2021). The scientists in our study, as we will tease out in this article, understand scientific work as an evolving, collective practice in which educating young academics is also a priority. The scientists point at the institutional conditions for doing scientific work as jeopardising the integrity of research practices. These are hard for individual researchers to change. These different understandings of how scientific practices function, we argue, explain the unintended effects of policy and suggest very different ways to support scientific integrity.
Good science
The disciplines we chose for this analysis represent a broad variety of state-of-the-art work within Dutch universities, and a broad variety of ‘styles of knowing’ (Kwa 2011): mathematics, chemistry, anthropology, philosophy and medicine (including epidemiology and action research). Two of us (Sonja and Jonna) conducted seventeen interviews with members of these disciplines about what they consider ‘good science’ in their field, how they work to achieve good science themselves, the concerns they encounter, and how they try to counter these problems. The concerns we identified cut across all academic fields. In addition, we draw on our ethnographic observations and conversations among research groups in these disciplines, with participant observation being conducted for a period of at least three months for each discipline. Finally, we draw on cross-disciplinary deliberations we managed to organise in October 2020, between waves of coronavirus lockdowns, where we discussed our results and the scientists’ recommendations for policy.1
Asking about the good in ‘good science’ is a way to put normativity on the agenda in science studies. This is a different approach from that taken by earlier philosophers of science, who formulated normative criteria and prescriptions to demarcate knowledge. We see the question ‘What is good science?’ as an empirical one. We articulate what the scientists find good, or how they shape good science in their activities, and contrast this with other notions of good science, such as those enacted and proclaimed by policy. ‘Good’, here, is an intentionally open concept, rather than a predefined one. Our explanation of ‘good science’ is not, therefore, our judgement as researchers; it comes from our interlocutors’ judgements and ideals, as well as the notions of ‘good science’ inscribed in their research, publishing, and teaching practices. These ‘forms of the good’ (Thévenot 2001) may be ethical in nature but also come in different forms and from different registers, such as methodological goodness, good p-values, efficiency, or good working conditions (Boltanski and Thévenot 2006).
Crucially, such practices aimed at something good are not necessarily good in their effects. In our empirical ethics approach, normativity is seen as being shaped simultaneously with matters of fact. We conceptualise descriptions as re-scriptions (Harbers 2005; Pols 2008, 2015), or ways of representing situations that frame these situations in a particular way. The notion of re-scription signifies that descriptions are not neutral ‘de-scriptions’, nor are they normative prescriptions that tell people what to do. This way of studying normativity is developed in a branch of STS called care studies (see Harbers 2005; Mol 2010; Pols 2015, 2022, 2023; Thygesen and Moser 2010; Willems 2010; Willems and Pols 2010), and in the STS understanding that methods are creative (Law et al. 2011; M'charek 2017; Ruppert et al. 2013). Attending to the multiple enactments of a situation and what to do about it is crucial in an empirical ethics analysis (Mol 2002; Mol and Hardon 2021). We use this method to re-scribe research integrity, by asking what research integrity comes to mean when we ask academics how they achieve good science and where they encounter obstacles to achieve it.
The Dutch scientific landscape and the problem of integrity
In the Netherlands, concerns about research integrity reached a peak in 2012 with the spectacular fraud of Diederik Stapel (Abma 2013; Derksen 2021; Levelt et al. 2012; Wagenmakers and Borsboom 2013). Stapel was a much-celebrated professor of social psychology, with highly esteemed publications. However, it turned out that he got his research results by filling out impressive amounts of questionnaires himself. His all-too-neat conclusions and datasets alarmed some of his PhD students, who brought the fraud to light, and Stapel was fired. Concerns about integrity widened from worries about such obvious fraud to less-defined questions about ‘sloppy science’ or ‘questionable research practices’ (Horbach and Halffman 2017; SRC 2017). Dutch integrity researchers (Bouter et al. 2016) came up with a list of no less than sixty ways of how ‘sloppy’ scientists might misbehave, such as cleaning up data sets or presenting results a bit more declaratively than warranted.
Such concerns about integrity led to questions about the quality, validity and reproducibility of research that had been done in the past (Aarts et al. 2015; John et al. 2012; Munafò et al. 2017; Penders et al. 2019). These three distinct things – outright fraud, questionable research practices, and difficulties replicating research – together set the stage for new policy on ‘research integrity’ (Helgesson and Bülow 2023; Hiney 2015). In 2012, the Royal Netherlands Academy of Arts and Sciences (KNAW) published an advisory report that led to a series of interventions: a national board for scientific integrity and university committees were established, and the code of conduct was rewritten (KNAW et al. 2018). A research programme on integrity was established to explore and analyse scientific fraud and remedies for it by the Netherlands Organization for Health Research and Development (ZonMw) and the Dutch Research Council (NWO).2 Ethics programmes for improving research integrity were developed, and replication studies were funded. The project we report on here was also financed by this funding scheme.
One way to ensure responsible research practices, it was thought, was to develop policies that would raise better scientists (Bretag 2016). Courses in research ethics, for example, would train novice academics in the morals of good science. A second line of policy centred on data management (Bulger and Heitman 2007). Every new research project supported by the largest funders of scientific research, the Dutch Research Council (NWO) and the Netherlands Organization for Health Research and Development (ZonMw), now has to include a ‘data-management plan’ that adheres to the ‘FAIR’ principles: data should be findable, accessible, interoperable and re-usable. ‘Openness’, transparency, and data sharing are framed as the solution to integrity problems. Dutch universities now treasure a new type of employee: the ‘data steward’, who helps scientists to properly manage their data, ensuring the integrity of scientific research.
There is a lot to say about how this policy impacts different disciplines. Anthropologists in our study, for example, argued against making raw anthropological ‘data’ public, as their data are not systematically collected variables, but close-to-the-skin observations and privacy-sensitive narratives elicited from their interlocutors (Koning et al. 2019). A policy of ‘blind openness’ for sharing data would jeopardise interlocutors’ trust and anonymity, and removing the data from the context in which they were collected might lead to interpretation errors. Quantitatively oriented scientists in our study, too, were sceptical that published data sets could easily be re-used and saw the push toward open data as mainly a concern about fraud.
Our point in presenting this overview is to show that problems with research integrity have been attributed to either moral misconduct or the sloppy or opaque handling of data. In response, Dutch scientific policy has sought to make science ‘good’ by enforcing norms, educating junior scientists and establishing procedures for handling and checking data. Individual misconduct and a lack of procedures were framed as the problem with integrity. The structure of research funding, in which individual researchers compete against one another for grants to produce ‘excellent’ scientists, mirrors this link between individual scientists and research outcomes. Whether science is good or bad depends on the achievement or failure of individual researchers.
Science governance in the Netherlands
In the 1990s a new financial model was adopted in Dutch universities: instead of direct allocations from the government to universities, the government began funding research projects through organisations that regulated research funding through programmes and competition. The goal was to ensure that more funding went to studies that were relevant to society as well as to creative and innovative research being conducted outside research schools. Such schools were seen as rather static and conservative, and there were concerns about the long trajectories of ‘eternal students’. To remedy this, in the early 1990s, PhD fellowships began to be granted only to young researchers immediately after finishing their master's degree.
Innovation and relevance became prized values: science was seen to improve continually by facilitating new ways of thinking and new methods aimed at improving society. Competitions for research funding focused on individual excellence. Scientists began to need to demonstrate their excellence with strong CVs and seamless proposals. Increasingly larger parts of university budgets were transferred to national organisations that ran such competitions (such as NWO and ZonMw), which gave the government more influence over how research money was spent. Later, the government began soliciting private financing from industry, both to reduce costs for taxpayers and to increase the relevance of scientific research for society (Dehue 2010; Dumit 2012; Harkema 2017; Rossel 2004; Sismondo 2009, 2017b).
These shifts also fit into the Dutch government's efforts to organise public-sector work as a ‘regulated market’. In this framework, the government provides a limited budget; any public-sector entity requiring further funding must figure out how to pay for the rest of its activities through private money (Tjeenk Willink 2018). Like health care, the arts, the police, and education, academia was to function through a logic of markets and finance itself through contracts with other parties. And so the Netherlands saw a spectacular bankruptcy of a hospital, the impoverishment of public services, and a growing number of public servants – including academics who organised themselves in activist think tanks such as ‘WO in Actie’; ‘Science in Transition’; ‘Rethink Science’ – on strike.3
Dutch universities’ income depends on attracting students and ensuring that scholars bring in research funds. Working conditions have become increasingly characterised by flexible contracts; as of 2019, 41 per cent of Dutch academics did not have tenure but worked from one temporary contract to the next (Rathenau Institute 2020). At the same time, workloads have increased due to the inflow of more and more students taught by the same number of staff (e.g. Ministry of Education 2021) and because academics spend more and more time on writing research proposals for competitions that select very few (Minister of Education, Culture and Science 2020; NWO 2020; Rathenau Instituut 2023). Also, Dutch universities now increasingly attracted international students, who pay higher fees. Finally, scientists are increasingly expected to produce results that are deemed relevant to society and to work with industry. Funding is being shifted to the physical and natural sciences and away from the social sciences, humanities and medical research. Increasingly, goals for the sciences are formulated externally, whether by government, private industry or the public, rather than by university scientists themselves.
The good of science according to scientists
In December 2016, Sonja set out to interview a first set of scholars, mainly full professors from a variety of disciplines, in a pilot study for which we received a small grant. Given that we started a mere three weeks before the Christmas break, we did not have much hope that they would respond before the break. However, almost all of the professors we approached responded immediately and made time to be interviewed before they went on leave. It was clear that they had something to say.
Collective learning to understand the world
In speaking about their everyday practices, the academics articulated their scientific work as a collective effort to achieve ‘good science’. There was also a remarkable consensus when they answered the question of what keeps them interested in academic work: the urge to learn things about the world.4 They see this as a collective task of students, professors, and all academic ranks in between. A professor of theoretical physics responded:
My idea of a university is this: we talk to students and we inspire each other. But being profoundly interested in the nature of the world, and of reality, is key. Of course, people should be able to do some mathematics. It does not have to be a world-level top talent, but there should be some ease in working with mathematics and physics and chemistry. So, it's a combination of that [skills] with an absolute commitment to understand the world.
On the issue of ‘understanding the world’, the scientists voiced similar passions, if in different wordings; to them, the core of being a scientist is to make sense of the world and to understand the parts that are not yet understood. ‘I like to puzzle and I like to solve things’, said one interlocutor. ‘Academia must be interested in understanding for the sake of understanding’, said another.
Emphases differ, but there was a clearly expressed desire ‘to improve practice with my science’, be it by better tools, better patient care, better batteries, better molecules or better theory. All of the scientists talked about the value of their scientific work for the world, whether contributions to theoretical problems, climate problems or hands-on action research.
We learned a lot about inspiration, enthusiasm and the beauty of good scientific work. An epidemiologist pointed out: ‘To write [something] up in a mathematical model, has a sense of scientific beauty’. An important part of the beauty of science is that it builds on work that has been done before, picking up where other colleagues and students have stopped, and the excitement of learning new things. A chemistry professor explained:
Nothing is more beautiful than a PhD who comes running into the room saying, ‘I've got it!!’ That is really fan-tas-tic. It doesn't happen every day, unfortunately. And the other thing, which is not nice but you feel you are engaged, is when you speak with a PhD who, for the seventh time, did not get a certain chemical reaction. It is not nice, you feel bad, but it is really very interesting – how to say this, I'm looking for a good word – you feel very engaged. That is what I find important in my work. The fact that you live through that research together, and it does not work, and dammit your colleague in the US just did what you wanted to do! It feels bad. But it is meaningful. It is thrilling, maybe that is the word. I find that really very important in my work.
When scientific work starts lacking this ‘thrill’ of the work being about something that matters, the scientist's motivation may wane. Our interlocutors who have opportunities to work in industry at much higher wages stressed this intrinsic motivation for doing scientific work in academia. A chemistry professor told us, ‘The job should be fun. If you work in industry you have to realise that, because you are paid much less at the university. So if it is not fun, why should you do it?! You can do a job that pays less but is more fun. But if it is no fun, then it is ridiculous’. ‘Fun’ here points to the commitment to understanding the world, encapsulating the variety of motivations that scientists have to work at the university.
The university as a social and traditional practice
The meaning and beauty of scientific work lie not only in the understanding it may produce but also in the process of collective ‘doing’ of science itself. Our interlocutors made clear that they regard science as a social institution in which, in addition to building on the work of others, one develops knowledge through teaching and asking new questions. An assistant professor of chemistry explained:
Teaching is the reason why I am at the university here, and not in industry. You can do a lot of research in the industry. But here, the ‘getting wiser together’, I find that very inspiring. To see if you can make these students enthusiastic about science. And it's just beautiful if people can solve a problem. You have this substance, it does not dissolve…and suddenly it does! Or, ‘Wow, it has become a little white foam, fantastic!’ Small things. People are really happy with that. To try it, to see: can I put it together?
In both Dutch policy and in the management of universities, teaching and research are managed and financed separately. In the academic day-to-day, however, the scientists see students and young scientists as crucial motors of science, as they are the posers of new questions. They may challenge taken-for-granted routines. Teaching is a way to cultivate the pleasures of scientific work for them. A professor of Theory of Science told us about how her students stimulate her own thinking:
At the minimum, [students] have to learn how to think. Not to just reproduce the handbooks. For me that is also a great pleasure. Last night I taught there again, and these students, second and third years, they really got wings. They started to give presentations on what they had learned. It was really fantastic! So nice. And on complicated matters, things I haven't thought through myself. And about which we think together. Yes, that is fantastic.
As this quote suggests, the scientists see academia as a collective that evolves through different generations of academics. A professor of theoretical physics offered the example of Humboldtian University as a place where students’ contributions are thus valued:
There is the unity of research and teaching. I believe in this idea of the Humboldt University. There you have this community of professors and students who are there for each other and form sort of a coherent union. Where the professor is also a research partner of the students. The student isn't only someone who learns but someone who contributes to research by asking somewhat naïve, or very good, questions.
Conducting scientific work within academia, according to the scientists, is a social enterprise that is propelled by the desire to contribute to understanding the world better. Implicit here is the idea that scientific work is based on tradition and craftmanship. It builds on work done before, even if it deviates from it or innovates on it. It needs the spark of pleasure or ‘internal’ motivation as well as fertile grounds to cultivate new knowledge from what is already known and grown. For this, the collective work of learning from the tradition and asking new question is needed.
Scientific communities can, however, also be conservative, with set routines and a hesitancy to branch out past defined boundaries or well-worn lines of inquiry. An associate professor of epidemiology, also serving as head of department, spoke about how certain fields of research with wide readership garner more recognition than newer, cutting-edge research:
These [medical] journals have a much higher citation index because more people read them. If you publish about cancer or cardiovascular problems, these are large diseases, with so many doctors and people working on it, they read these journals. So, a publication in a journal that I like [naming a publication from a less established field], it has a citation score of 2.5. Meanwhile the cancer journals are 12 or 14, so that is many more points, just because you are in a different field. And these journals publish continuously. If they find some little thing, an association, a correlation, a small difference. It is not very exciting.
Indeed, in the publication culture of medicine, there is a steep hierarchy in citation scores, related to the number of people reading and quoting journal articles. This leads scientists to choose to publish in ‘safe’ journals with large audiences rather than in journals that seem more interesting, innovative, or important to them (Benedictus et al. 2016; Greenhalgh et al. 2014; Pearson 2021).
Originality, tradition, and not-yet-knowing
Formally, in the philosophy of science – and in Dutch scientific practices – there is much devotion to what philosopher of science Imre Lakatos (2015) calls the context of justification, or the phase in research that is about methods to back up the truth claim of the research results. Engaging in justification means arguing about the trustworthiness, reliability and validity of research findings. The idea in positivist science is that with stringent methods, facts may be distinguished from beliefs, valid results from errors. Good methods justify good science, by minimising researchers’ influence on outcomes (Daston and Gallison 2007) to make nature ‘speak for itself’ or ‘mirror nature’ (Rorty 1979). Lakatos contrasted justification with the context of discovery, or the context in which scientists get their ideas and develop hypotheses. Good ideas may come from dreams, a falling apple, or an overflowing bathtub. It is the justification that may turn these into facts.
In the scientists’ daily practices, however, justification was only one part of the work. Work in the context of discovery demanded much time and turned out to be about much more than ‘getting a good idea that can be tested’. The scientists cherish this space. It relates to the work of preparing the grounds before good methods may be applied: framing a hypothesis, developing a theoretical framework to interpret the results, reading up on a topic, or figuring out the precisely right concepts to think about a certain phenomenon. A main part of the work in the interpretive sciences entails framing ideas in one way or another to articulate what was not explicit before. The context of discovery, that may be better labelled as the context of creativity, is, however, important in all of the scientific fields we visited, as it relates to the comprehension and formulation of the phenomena under study (Despret 2006, 2015; Hirschauer 2006; Law 2004; Law and Savage 2011; Ruppert et al. 2013). A professor of theoretical physics explained:
What I typically do, if I want to enter a new field, I will try to give a course. I do some reading myself, but then I quickly go to people. Including students. […] And then [in the course] it becomes clear what the problem is. There is always a problem, but the problem is not always obvious, so by talking to people and reading, I try to get to see what the real problem in that area is, and what I need to do to do research in that. And then I start reading more, and I get back to these people, and at a certain point, I know what needs to be done.
The professor points at the space for ‘not yet knowing’ what the problem is, which is about investigating and engaging in discussions to articulate what cannot yet be put into words. It is the work of formulating that which is not yet known and what needs to be done to get to know it. He attributes making progress in his scientific field to conversation and thinking partners, including books and papers. Hence, we suggest extending Lakatos's work. His context of discovery is about the tacit and creative part of scientific work, where there are unknown phenomena to be grasped.5 Rather than a metaphor of ‘discovery’ or ‘mirroring nature’, working with this type of fluid, not-yet-describable knowledge is about ‘finding words’ for things not yet articulated. Instead of producing or transferring facts, scientific work involves juggling uncertainties, meticulously framing research problems, and interpreting findings. An assistant professor of organic chemistry contrasts allowing students to use calculators and other tools with insisting that they be able to understand their results:
These [digital lab journals that organise the data] calculate your tables for you, so these are mol masses, density, melting units, characteristics of substances, [all things] that you used to have to think about. How heavy is a molecule? […] Today, the lab journal calculates it all. But here the mistakes come in! Lab journals are great for keeping data, but not for the researcher to understand ‘What am I doing? Does it make sense?’ It is really the difference between doing sums in your head and with a calculator. […] I insist on this with students: ‘Well, you show me a spectrum, and you say it's good. But if I ask you why, you don't know. […] So [I tell them]: ‘Go back to the desk, print it out, work with it, so that you can tell me even when you are asleep what your molecule looks like’. You teach your students not to jump to conclusions. I really enjoy teaching that.
Finding words to articulate and understand problems is about practical routine work, what you are able to both understand and practice. Tacit knowledge needs to be made explicit, or you, the scientist – and your students – will never ‘get it’.
In this context of not-knowing, scientists’ interest in ‘negative results’ makes sense. ‘Negative results’ teach us things about the world, even if they are difficult to publish. A chemistry professor spoke about this:
Well, the problem is, a negative result is often difficult to publish. What we often do is take it along into publications with positive results. That you can show what does not work. We focus much on characterising mechanisms, on understanding the reactions and mechanisms. And for that, negative results are very useful, because they teach you a lot.
When defining scientific work as justification (testing hypotheses) – and financing it as such – a lot of this creative work becomes invisible. There are many things that cannot be neatly categorised as either ‘having a good idea’ (discovery), as testing hypotheses (justification) or as teaching.
These activities take a lot of the scientists’ time but are not directly financed and become invisible in a science-as-project-frame (see Table 1). They are undertaken alongside ongoing teaching and research obligations. The scientists in our study reported that they worked more than eight hours each day, and worked in the evenings, nights, weekends, and during days off. When working on a research contract, one is expected to follow the project plans – yet do these other things as well.
Invisible work in science
| Gathering knowledge through ‘reading up’ |
| Commenting on draft papers |
| Reading theses and being part of PhD committees |
| Writing applications and supporting colleagues in doing this |
| Sharing knowledge through interviews, public lectures, conferences, guest lectures, and engaging in public debates, advising NGOs, policy makers, and industry members |
| Caring for the department/research group |
| Peer-reviewing papers and research protocols |
Problems in scientific institutions: The erosion of everyday ethics
We described how the scientists understand their practices through what they cherished as ‘good science’. We will now look at their understandings through their descriptions of the problems they encountered. The first concern was with the ‘everyday ethics’ of scientific work (Banks et al. 2013; Pols 2023). Paul Brodwin 2013 used the term in the context of everyday mental health care. Everyday ethics thrives on traditions and motivations such as fun, beauty, thrill and inspiration. It has a pragmatics that is not rule-based or ‘added’ to the scientific work but forms an integral part of the work itself. There is not a handbook or set of guidelines to learn what should be done, but rather a culture where the orientation towards good science is lived and practiced on a daily basis. An example from a philosophy professor was as follows:
You have to ensure that there is sufficient time for internal debate…. Everybody has a lot of liberty…. But when we have a joint meeting, I expect everybody to be there…. You can only have an academic institute where people are working together if there is a space where they have discussions that are just driven by content. You want to get things right, you want to understand them better. You discuss the text and you are open about it…. [I want] that there is room for antagonistic statements, but in a friendly but clear way, and with a genuine interest in the topics. Students have to learn this attitude as well.
This ethos of a ‘good scientist’, then, cannot be separated from everyday work, from teaching students, talking with colleagues, and conducting – and understanding – experiments. This is the ‘everyday ethics’ that cannot be delegated to external committees; it emerges in the ongoing solving of frictions and making of decisions in conducting scientific work. The scientists ask: How should I approach this? How do I manoeuvre around the obstacles that I encounter? Why did the experiment ‘fail’, and what should I do about it? Was it a result of a lack of skill or is it evidence of facts that speak? How to decide on good cut-off points? How to phrase the nature of this problem?
The place of the unexpected
The concern with ‘what is not yet known’ and the struggle for the words, formulae, or experiments to articulate it make the process of scientific work unpredictable. Scientists have a certain respect for this unpredictability, which leaves some room for ‘out-of-the-box’ thinking and the possibility to pursue avenues that are not yet well articulated. A chemistry professor quotes the following example from the recent Dutch Nobel prize winner Ben Feringa:
I remember very well that first talk that one of Ben Feringa's PhDs gave. He talked about ‘molecular switches’, as part of ‘molecular motors’. At the end of his talk, many PhDs and professors said: ‘Pff, now, I don't know about that. What is he on about! It will be nothing’. And then: Nobel prize! And that sounds very romantic, and it does not happen like that often. Very often you say: that will lead nowhere, and it does not. Sure, all true. And then again: no [science funder] would have put money on ‘molecular switches’. I don't even know if Feringa knew himself [that it would be considered so significant]. But that is exactly what it became.
The quote about a Nobel prize-winning ‘discovery’ signals a tension between the unpredictability of scientific results and a traditional knowledge that is sceptical about innovations that use different terminology and methods. The importance of unpredictability was also put forward by anthropologists, scientists who make it their specialty to ask open questions to make space for interlocutors to ‘articulate problems’ in their own words and from their own perspectives. The researcher cannot know beforehand what they are concerned about. A professor of anthropology similarly spoke of the ‘co-creation’ of anthropological data (extensive in-depth interviews, observations) as the result of the collaboration between researcher and subjects.
The clash between originality and tradition
Many scientists experienced a tension with the limited possibilities to explore different routes and engage in original work. An epidemiologist recalled the following:
At a certain point I thought: I am not doing this anymore. I want to do beautiful studies in my own field, not just give others statistical advice. And, hup [just like that], my publications went down. But I got time to think. And I thought: ‘This is what I am going to develop. This has to be done. I am sick of it [giving advice]. I'm going to go for it’. I had some data, and I wanted to write about ways to analyse these. Very basic. One Sunday evening I sat at the kitchen table and started. And in the night the analysis was finished. It worked, and I thought: ‘I am going to write about this’. Alone, you see? On my own! […] I have 140 publications and I think there are maybe five or ten that I am a bit proud of. Two or three that I am really proud of. And let those be the ones nobody quotes.
Originality here is linked to individual scientists who are well aware of the limitations of their own field. Original work can pose questions at the very specialised crossroads of the different disciplines in which a particular scientist is trained. Interestingly, like the epidemiologist, the scientists who spoke about originality were not often applauded by their colleagues or their universities but encountered it as a difficulty. These research problems emerged not when doing something new within research traditions, but outside of them. The scientists described it as having to break with a set of routine understandings of what science is about and how it should be conducted well.
As a social enterprise and tradition-based practice of craftmanship, scientific work takes place in the interaction between conventions and new ideas. Conventions often concern good ways of justification, or ‘proper methods’. This relates to the traditions of a field. Too much originality might make it difficult to find an audience, as the epidemiologist indicated. Too much genius can be a handicap in science (Brenneis 2006; Lamont 2009).
Scientific conventions may lead to exclusion and even violence. One assistant professor described her cross-disciplinary work in a medical faculty, a place with strict ideas about proper methods that are modelled on epidemiological research. A kind of ‘dissident’ scientist, she developed new concepts aimed at innovating and improving patient care rather than doing epidemiological research. She created collaborations with scientists from the social sciences. She also involved social partners, including the municipality, rehabilitation clinics, paramedics, and so on. The methods she used were oriented towards action research in context, aiming to understand local processes and learn from these, rather than formulate generalising conclusions and recommendations. Her critique of evidence-based medicine points to a different appreciation of what makes ‘good science’:
For me the problem is the evidence-based medicine movement. My concern is that too much emphasis [on it] will lead to the impoverishment of care. You can imagine it as a leaky pipeline. […] At every stage in the process [of statistical research], a lot of [clinical] knowledge is leaking. And we articulate that knowledge. Objective knowledge is good, when it is possible to objectify. But there are so many other [research] practices needed in the care landscape. This is what drives me in my scientific work: I want to do something for patient care, also for professionals, to improve their practices.
The department this researcher worked in did not take her research interest lightly. Her (practice-oriented) publications did not help the department meet its output quota, her colleagues did not understand and approve of her methods, a senior professor did not want to sign her research applications anymore, and in the end, she was transferred to a different department. These painful confrontations affected her career, and she was not the only ‘dissident’ we encountered in our study. Her story shows that the blossoming of new ways of doing scientific work is not a mere matter of ‘opening up’ the academy. New ways of doing science need a protective environment. The researcher was adamant on this point:
You need protection. If my boss does not have the guts and the courage to support me, and it will be me against the evidenced-based medicine researchers, and then I am lost. In a way, you are a border violator. The procedures the rules, all these things – somehow you have trespassed that [by doing things differently]. And many people don't like that. So this protection is very, very important.
This scientist's story shows that originality, or departing from a scientific tradition, needs a social infrastructure. Colleagues who want to work together need a network of people with shared language, shared aspirations and safe spaces to meet to be able to develop their work. The ‘dissident’ spoke also about the importance of collaboration:
You [need to] compose a very informal, open, and dynamic research group. And that is why you can do this kind of things [cross-disciplinary work]…. I had a golden team, with people of different [disciplinary] backgrounds.… Behind closed doors we could talk about everything, also about the difficult things. Because the institution people did not like things being questioned. Then it became dangerous for us. So we had to rely on each other. You need a core team of people who you know will support the enterprise you are working on.
Openness, in the previous example, was not about ‘letting everybody in’, but rather entailed selectively closing the group to create the safety needed in a learning environment. Key values and starting points were shared, discussed, and developed. There needs to be a safe breeding ground for innovations to emerge. When results are finalised, they can be openly shared and discussed. But puzzling through possible formulations for problems and cultivating shared premises and values needs a space for ‘discovery’ where ‘everything can be discussed’ and the demands of the outside world can be negotiated.
In our research, then, everyday scientific work emerges as a balancing of traditions, skills, routines, and know-how with originality. Originality rarely takes the shape of a ‘eureka’ moment; it consists instead of hard and persistent work, in a context where one has to actively search for support from colleagues while fending off others whose critiques may sink the effort. Interdisciplinary work may take established forms, as is the case with epidemiology in medical research. But it may also consist of forging links with new scholars and trying out unfamiliar methods.
What is relevant research?
The ‘absolute commitment’ mentioned by the theoretical physics professor earlier may be overstating it, but a commitment to ‘get it right’ is needed to achieve good scientific results as well as good scientific practices. The scientists report a shift from internally defined understandings of scientific relevance towards goals set by external bodies. These differently formulated forms of relevance may be in tension. All scientists want to do relevant research, but what is considered relevant may vary greatly.
As the greatest influence on scientific relevance the scientists mentioned the competition for research money, which creates a need to invent what they called ‘sexy research topics’ (see KNAW 2020a). Many competitions require that research proposals be evaluated by multidisciplinary panels, which necessarily means non-specialists. An associate professor of chemistry stated the following:
I was asked [to evaluate] the last round in the competition, the interviews. It was a round of proposals for elementary particles physics. And I really don't know anything about that. Not more than what you can read in popular scientific journals. And I thought: Can I understand if the candidate talks sense and has a good idea? I thought that was very difficult. And still I had to judge it because I was part of this committee. I think the idea to broaden the disciplinary character of these committees was to make the procedure more objective. But it was rather that there was one expert in the committee who told the rest of us something about the proposed research. I could only listen to that. But is this more objective?
Michele Lamont's (2009) analysis of interdisciplinary panels also shows a tendency for the panellists to avoid disagreements. Some of our interlocutors argued that the Netherlands is too small to determine what is good research without a conflict of interests.6 What is perceived as relevant research, they said, tended towards ideas ‘that were easy to sell’ to a broader audience. A concern was that the competition for research money could lead to the hyping and fragmentation of scientific work, and the use of vague concepts because they were trendy. The scientists worried about long-term and fundamental research (see also KNAW 2019). Rather than being fed by ‘traditions’, scientific work was feared to be led by ‘fashions’. A mathematician lamented that concern: ‘Now everything is artificial intelligence, and before that it was all Big Data. But if you look more closely, we do the same thing for years, only we use different terms’. The scientists fear that research language is being eroded by the use of buzzwords.
As mentioned earlier, the scientists noticed an increasing influence on scientific research from government initiatives. A medical scientist saw conservative tendencies in this, complaining about ‘research within the norm’. A philosophy professor pointed to the consequences of this for winning research competitions: ‘Because there is one language that is spoken in the [funding] committee, it is difficult for different fields to say anything that may be heard’. An anthropology professor echoed these concerns, and highlighted the larger social consequences, as well: ‘Proposals are based on national consensus. But we want to study people who are not part of that. People whose problems are not recognised [as worthy of research] will be even less recognised’.
Industry's power to influence what is considered relevant was also considered a problem. The Dutch government strongly supports collaboration with industry, as a way to finance research. But the interests of industry and academia may differ. As a chemistry professor put it: ‘Sure, I do want to do nice fundamental research! And I have clear ideas about future use. But the industry wants stuff that earns them money the next day’. Scientists are concerned that industry will set the scientific agenda because they pay for research.
A last problem with how relevance is determined is the seeming randomness of who succeeds in a scientific career. An assistant professor of chemistry related how career pressure could impact what is relevant research:
Our first project is synthesising a new molecule. And I would like to test it in the biological system, because I designed this molecule with the idea that it can counteract infections. But I am advised to publish only about the synthesis, the making of that molecule. To have a paper out quickly. Whereas I had thought it would be really nice to see if this molecule actually works – that means you do the biological tests. But well, that easily takes another half year. But given my tenure track deadline [success in five years, or the researcher is out], I am advised to publish it already. And that is against my sense of integrity.
Career and individual success here clash with the desire to create scientific knowledge that is relevant to society.
Diversity in science
A sensitive issue is gender inequality in (Dutch) academia, as well as the representation of culturally minoritised populations7 and the position of young scientists. When understanding science as a competition between individual scientists, this would imply that policy should aim for guaranteeing the conditions for everyone to enter. However, some scientists blame the competitive culture as infecting everyday ethics that hinders equal opportunities for access, as did this chemistry professor:
There will be no women at the top. We don't have them. In our institute we have women on PhD level, postdoc level, we have some assistant professors, and no professors…. Other institutes might have a few more. And that has a lot to do with how competitive we are. The competitiveness is a male culture. We can only change that if we can keep competition outside [of our department]. We had three female professors, but they went elsewhere, because they didn't like the culture. We couldn't keep them.
Outdated regulations also added to the uneven conditions for women, and even some more recent ones were outright discriminatory. Another chemistry professor explained:
There are the rules for what you have to do if you want to get promoted from assistant to associate to full [professor]. And this changes rapidly, in the last years, nationally. Because in [name of city] a couple of women won a court case and so the universities are obliged to extend their contracts if they go on pregnancy leave. You see that [inequality] is repaired at a rapid pace in the faculties of exact science. It is a constant discussion but there is a lot of space. This is what I think. Maybe there are women who say: it is not enough. But compared to, say, six, seven years ago, it is a world of difference. Really a revolution, it goes really fast.
This concerns matters of access and practice, but there is also the question of the consequences of adding different perspectives and positionalities for the content of academic work (see, for example, classic feminist work on science: Haraway 2013; Fox Keller 1983). An important concern was about jobs for young academics, as many junior scholars are paid through research funding, or teaching contracts. Financing research through projects means that contracts are only ever temporary, and aspiring academics therefore must move from one project and university to another – or quit the job. This might seem attractive from the perspective of university managers: no people on the payroll forever. But when academic practice is understood as a practice that balances tradition and innovation, this is only in the short term. From this perspective, bad working conditions and a lack of diversity threaten the continuity, the content and the innovation of scientific knowledge.
Conclusion and discussion: What makes academia work?
The suggestions from policy for regulating the sciences turn out to be based on a quite different understanding of what academic scientific practice is and what its problems are. The ‘view from within’ provided by our analysis provides detailed understandings of academic practice, where policy interferes by enforcing a reality of excellent (or fraudulent) individual scientists in competition with one another and organised by ideas about academia functioning as a market (see also Brint 2020; Gumport 2019).
The scientists we studied understand and enact academic practice as a social practice that evolves by passing on what is known and simultaneously allowing space for doubting, not-knowing and challenging existing knowledge. New ideas emerge by thoughtfully departing from what is already known in a particular research tradition, or by questioning taken-for-granted assumptions. New science does not just ‘pop up’, fall from a tree or bud from an excellent brain, but takes place within a setting that is already filled with ideas that support a next step. Scientific innovation is thus not a matter of individual skills or genius, but of working with and from ideas that are already in circulation and adding to them, combining them, opposing them, or playing with them in practices of not-yet-knowing.
The scientists made clear that the sciences progress within communities that balance research within shared traditions with research that breaches the tradition. Science policy affects this precarious balancing of tradition and innovation. In the Dutch case, the competition for research money, where colleagues – within subfields and across disciplines – judge one another's work, favours uniform language and criteria, and does not stimulate border crossing between disciplines but rather ‘research to which nobody would object’. The competition stimulates safe choices for method, design and reputation that most colleagues from other disciplines can understand and subscribe to (see also KNAW 2020b). These uniformising tendencies also make it difficult for women and people from diverse cultural backgrounds to enter academia: the work has to be done on the terms that have already been established without them.8
Policy interferes with this social and collective learning practice by conceiving of the work of science as done by individual scientists, who may be excellent or sloppy or fraudulent. Policy aims to make the best science and scientists emerge by making them compete in designing the best research projects, and the project, as a form of organising research, has increasingly become a standard (see on ‘project-ness’: Law 1994; Felt 2017a). Our analysis shows that there are negative effects, particularly that the communities in which scientists work get torn apart. For example, the unity of teaching and research desired by scientists is broken up, dividing the labour of students and professors, and the potential critical insights of younger staff and students on the work of more senior scientists are lost. PhD students were the ones who brought Stapel's fraud to light (Levelt et al. 2012). The difficulty for young scientists to get a position has the same effect, as they move from one research project to another, and hence do not become part of a scientific community that works together on a daily basis over the long term. Sometimes competition enters even departments and classrooms, ruining the ‘fun’ search for collective learning and a better understanding of the world.
Severe competition and scarce research money threaten the everyday ethics of academic practice. The eroding of everyday ethics in science is amplified by the increasing external determination of relevance, for instance by industry, and the need for scientists to formulate projects that are attractive to a broad audience. Considerations about how to achieve good science are increasingly taken over by imperatives from outside.
This set of circumstances is far from the conditions desired by the scientists we spoke with, who were occupied with ‘things not yet known’ and the work of formulating research concepts, questions and directions. Scientific knowledge was understood to develop in more erratic and creative ways, rather than through defined research projects on justified roads that promise a specified end. They gave examples of how long it takes to formulate research questions well, choosing to publish in journals that may not be the highest ranked ones as a process of exploration, and the importance of gaining an understanding of what one is doing when making calculations, rather than relying on automated tools. They were frustrated with being discouraged from publishing ‘negative results’, which would be good for developing knowledge, but bad for evaluating scientific reputations, or impossible because of publication biases of journals. The financing of academia through funding competitions and increased teacher–student ratios made a lot of this work that relates to not (yet) knowing things invisible, hence adding to the work pressure in academia. The scientists cherish this work as it forms their creative drive of ‘figuring things out’. The importance and safety of individual scientists’ careers could, however, clash with their concern for scientific progress.
What does all this mean for the concern for scientific integrity? We found that the institutional conditions under which the scientists had to work forced them to consider choices, having to do with career, job security or reputation that had less integrity in terms of what they understood as doing good science. The scientists were very aware of these dilemmas and that these were the parameters for conducting any science at all. They might feel forced to opt for the alternative that does not support good science but does, say, save a colleague's career or the department's finances.
What is hopeful here, however, is the scientists’ dedication to achieve good science. Rather than quitting their jobs, they put in the long hours to work towards their scientific ideals, even if policy makes this difficult. Integrity problems, we conclude, are mainly problems with the integrity of institutions, of the local, national and international financing and organisation of academia and scientific work, rather than of malicious researchers or intentional or ignorant practices of sloppy research (see also Douglas-Jones and Wright 2017). The challenge is a complex one: to develop the management, financing and governance of the sciences in ways that support the quest for good science and education rather than undermine it. With the ‘Dutch case’ we hope to have shown the urgency of this quest worldwide.
Notes
We published our findings in a Dutch policy report as well (Jerak-Zuiderent et al. 2021).
For a critique on neoliberal models to organise the sciences, see Brint 2020 and Gumport 2019.
This was also the leading slogan in the protests on 6 April 2021. See: https://normaalacademischpeil.nl/.
Latour (1987) calls this: ‘science in the making’, where uncertainty rules, as opposed to ‘readymade science’, where results are stabilised.
Most research money goes to relatively few researchers: the so-called Matthew effect (Bol et al. 2018).
See also the post-colonial analysis of the injustice and resistance in present-day structures and methods that consolidate inequalities (e.g., Rothman 1987; Washington 2006).
The Netherlands is almost at the bottom of the list for the percentage of women professors. In Europe in 2016 only Belgium was worse. https://www.rathenau.nl/en/science-figures/personnel/women-science/share-female-professors-netherlands-and-eu-countries. Numbers are slowly rising: 25.7 per cent of professors were women in 2021, 24.3 per cent in 2020, up from 18.7 per cent in 2016. www.lnvh.nl/monitor2020/EN.html; www.lnvh.nl/a-3719/monitor-vrouwelijke-hoogleraren-2021; Rathenau Institute (2021).
References
Aarts, A. A., J. E. Anderson, C. J. Anderson, P. R. Attridge, A. Attwood, J. Axt, M. Babel, S. Bahnik, E. Baranski… Zuni, K. (2015), ‘Estimating the reproducibility of psychological science’, Science 349, no. 6251. https://doi.org/10.1126/science.aac4716.
Abma, R. (2013), De publicatiefabriek: over de betekenis van de affaire-Stapel [The Publishing Factory. On the significance of the Stapel affair] (Nijmegen: Van Tilt).
Banks, S., A. Armstrong, K. Carter, H. Graham, P. Hayward, A. Henry, T. Holland, C. Holmes, A. Lee, A. McNulty, N. Moore, N. Naylin, A. Stokoe and A. Strachan (2013), ‘Everyday ethics in community-based participatory research’, Contemporary Social Science, 8, no. 3: 263–277. https://doi.org/10.1080/21582041.2013.769618.
Benedictus, R., F. Miedema and M. Ferguson (2016), ‘Fewer numbers, better science’, Nature 538: 453–455. https://doi.org/10.1038/538453a.
Bol, T., M. De Vaan and A. Van de Rijt (2018), ‘The Matthew effect in science funding’, Proceedings of the National Academy of Sciences 115, no. 19, 4887–4890. https://doi.org/10.1073/pnas.1719557115.
Boltanski, L. and L Thévenot (2006), On Justification: Economies of Worth (Princeton, NJ: Princeton University Press).
Bouter L. M., J. Tijdink, N. Axelsen, B. C. Martinson and G. ter Riet (2016), ‘Ranking major and minor research misbehaviors: Results from a survey among participants of four world conferences on research integrity’, Research Integrity and Peer Review 1. https://doi.org/10.1186/s41073-016-0024-5.
Brenneis, D. (2006), ‘Reforming promise’, in A. Riles (ed), Documents: Artifacts of Modern Knowledge (Ann Arbor: University of Michigan Press), 41–69.
Bretag, T. (ed), (2016), Handbook of Academic Integrity (Singapore: Springer).
Brint, S. (2020), Two Cheers for Higher Education: Why American Universities Are Stronger Than Ever—And How to Meet the Challenges They Face (Princeton, NJ: Princeton University Press).
Brodwin, P. (2013), Everyday Ethics: Voices from the Front Line of Community Psychiatry (Berkeley: University of California Press).
Bulger, R. E. and E. Heitman (2007), ‘Expanding responsible conduct of research instruction across the university’, Academic Medicine 82, no. 9: 876–878. https://doi.org/10.1097/ACM.0b013e31812f7909.
Daston, L. and P. Galison (2007), Objectivity (Princeton, NJ: Princeton University Press).
Davies, S. R. (2018), ‘An ethics of the system: Talking to scientists about research integrity’, Science and Engineering Ethics 38, no. 1: 1–19. https://doi.org/10.1007/s11948-018-0064-y.
Davies, S. R. and K. Lindvig (2021), ‘Assembling research integrity: negotiating a policy object in scientific governance’, Critical Policy Studies 15, no. 4: 1–18. https://doi.org/10.1080/19460171.2021.1879660.
Dehue, T. (2010), De depressie-epidemie: over de plicht het lot in eigen hand te nemen [The Depression Epidemic: On the duty to take one's fate into one's own hands] (Netherlands: Atlas Contact).
De Jong, E.P.D., T. P. Franssen and S. Pinfield (2021), ‘Excellence’ in the research ecosystem: A literature review, RoRI Working Paper No.5. https://doi.org/10.6084/m9.figshare.16669834.
Derksen, M. (2021), ‘A menagerie of imposters and truth-tellers: Diederik Stapel and the crisis in psychology’, in S. Woolgar, E. Vogel, D. Moats and Claes-Fredrik Helgesson (eds), The Imposter as Social Theory: Thinking with Gatecrashers, Cheats and Charlatans (Bristol: Bristol University Press), 53–76.
Despret, V. (2006), ‘Sheep do have opinions’, in B. Latour and P. Weibel (eds), Making Things Public. Atmospheres of Democracy (Cambridge, MA: MIT Press), 360–370.
Despret, V. (2015), ‘Thinking like a rat’, Angelaki 20, no. 2: 121–134. https://doi.org/10.1080/0969725x.2015.1039849.
Douglas-Jones, R. and S. Wright (2017), Mapping the Integrity Landscape: Organisations, Policies, Concepts, CHEF Working Papers on University Reform 27 (Copenhagen: CHEF), https://pure.itu.dk/ws/portalfiles/portal/82382941/Working_Paper_27_Mapping_the_Integrity_Landscape.pdf.
Dumit, J. (2012), Drugs for Life: How Pharmaceutical Companies Define Our Health (Durham, NC: Duke University Press).
Felt, U. (2017a), ‘Under the shadow of time: Where indicators and academic values meet’, Engaging Science, Technology, and Society 3: 53–63. https://doi.org/10.17351/ests2017.109.
Felt, U. (2017b), ‘“Response-able Practices” or “New Bureaucracies of Virtue”: The challenges of making RRI Work in academic environments’, in L. Asveld, R. van Dam-Mieras, T. Swierstra, S. Lavrijssen, K. Linse and J. van den Hoven (eds), Responsible Innovation 3. A European Agenda? (Cham: Springer), 49–68.
Fox Keller, E. (1983), A Feeling for the Organism: The Life and Work of Barbara McClintock (WH Freeman: San Francisco).
Greenhalgh, T., J. Howick and N. Maskrey (2014), ‘Evidence based medicine: A movement in crisis?’, BMJ 348. https://doi.org/10.1136/bmj.g3725.
Gumport, P. J. (2019), Academic Fault Lines: The Rise of Industry Logic in Public Higher Education (Baltimore, MD: Johns Hopkins University Press).
Haraway, D. (2013), Simians, Cyborgs, And Women: The Reinvention of Nature (London: Routledge).
Harbers, H. (2005), ‘Epilogue: Political materials – material politics’, in H. Harbers (ed), Inside the Politics of Technology. Agency and Normativity in The Co-Production of Technology and Society (Amsterdam: Amsterdam University Press), 257–272.
Harkema, B. (2017), Publieke kennisorganisaties door de jaren heen [Public knowledge organisations over the years] Rathenau Instituut. https://www.rathenau.nl/nl/kennisgedreven-democratie/publieke-kennisorganisaties-door-de-jaren-heen (accessed 18 January 2021).
Helgesson, G. and W. Bülow (2023), ‘Research integrity and hidden value conflicts’, Journal of Academic Ethics 21, no. 1: 113–123. https://doi.org/10.1007/s10805-021-09442-0.
Hiney, M. (2015), Research Integrity: What it Means, Why it Is Important and How We Might Protect it (Brussels: Science Europe), https://www.scienceeurope.org/media/dnwbwaux/briefing_paper_research_integrity_web.pdf.
Hirschauer, S. (2006), ‘Puttings things into words. Ethnographic description and the silence of the social’, Human Studies 29, no. 4: 413–441. https://doi.org/10.1007/s10746-007-9041-1.
Horbach, S. P. J. M. and W. Halffman (2017), ‘Promoting virtue or punishing fraud: Mapping contrasts in the language of “scientific integrity”’, Science and Engineering Ethics 23, no. 6: 1461–1485. https://doi.org/10.1007/s11948-016-9858-y.
Jerak-Zuiderent, S., J. Brenninkmeijer, A. M'charek and J. Pols (2021), Goede wetenschap: een visie van binnenuit [Good science: A view from within] (Amsterdam, The Netherlands: AmsterdamUMC, UvA and ZonMw).
John, L. K., G. Loewenstein and D. Prelec (2012), ‘Measuring the prevalence of questionable research practices with incentives for truth telling’, Psychological Science 23, no. 5: 524–532. https://doi.org/10.1177/0956797611430953.
KNAW (2012), Zorgvuldig en integer omgaan met wetenschappelijke onderzoeksgegevens [Handling scientific research data carefully and with integrity] (Amsterdam: KNAW).
KNAW (2019), Evenwicht in het wetenschapssystem. De verhouding tussen ongebonden en strategisch onderzoek [Balance in the science system – The relationship between unfettered and strategic research] (Amsterdam: KNAW).
KNAW (2020a), Academische vrijheid in Nederland. Een begripsanalyse en richtsnoer [Academic Freedom in the Netherlands: A Conceptual Analysis and Guideline] (Amsterdam: KNAW).
KNAW (2020b), Het rolling-grantfonds. Kloppend hart voor ongebonden onderzoek [The Rolling grant fund—Beating heart for unfettered research] (Amsterdam: KNAW).
KNAW, NFU, NWO, TO2-federatie, Vereniging Hogescholen, VSNU (2018), Nederlandse gedragscode wetenschappelijke integriteit [Netherlands Code of Conduct for Research Integrity] (Amsterdam: KNAW).
Koning de, M., B. Meyer, A. Moors and P. Pels (2019), ‘Guidelines for anthropological research: Data management, ethics, and integrity’, Ethnography 20, no. 2: 170–174. https://doi.org/10.1177/1466138119843.
Kwa, C. (2011), Styles of Knowing: A New History of Science from Ancient Times to the Present (Pittsburgh, PA: University of Pittsburgh Press).
Lakatos, I. (2015), Proofs and Refutations: The Logic of Mathematical Discovery (Cambridge: Cambridge University Press).
Lamont, M. (2009), How Professors Think: Inside the Curious World of Academic Judgment (Cambridge, MA: Harvard University Press).
Latour, B. (1987), Science in Action. How to Follow Scientists and Engineers through Society (Milton Keynes: Open University Press).
Law, J. (1994), Organizing Modernity (Oxford: Blackwell).
Law, J. (2004), After Method: Mess in Social Science Research (London: Routledge).
Law, J., E. Rupert and M. Savage (2011), The Double Social Life of Methods, Cresc working paper, no. 95 (Milton Keynes: Cresc Open University), http://w.heterogeneities.net/publications/Law2010DoubleSocialLifeofMethod5.pdf.
Levelt, W. J. M., P. Drenth and E. Noort (eds) (2012), Flawed Science: The Fraudulent Research Practices of Social Psychologist Diederik Stapel (Tilburg: Commissioned by the Tilburg University, University of Amsterdam and the University of Groningen), https://www.tilburguniversity.edu/sites/default/files/download/Final%20report%20Flawed%20Science_2.pdf.
M'charek, A. (2017), ‘From a politics of difference to a politics of sameness, and back!’, EASST Review 36, no. 2: 8–9, https://easst.net/easst-review/36-2/from-a-politics-of-difference-to-a-politics-of-sameness-and-back/.
Minister of Education, Culture & Science (2020), ‘Letter to parliament’, https://zoek.officielebekendmakingen.nl/kst-31288-834.html.
Ministry of Education (2021), Toereikendheid, doelmatigheid en kostentoerekening in het mbo, hbo en wo&o [Research into the sufficiency of the microbudget and cost-effectiveness in Dutch education], Rijksoverheid.nl.
Mol, A. (2002), The Body Multiple: Ontology in Medical Practice (Durham, NC: Duke University Press).
Mol, A. (2010), ‘Care and its values. Good food in the nursing home’, in A. Mol, I. Moser and J. Pols (eds), Care in Practice: On Tinkering in Clinics, Homes and Farms (Bielefeld: Transcript Verlag), 215–234.
Mol, A. and A Hardon (2020), ‘What COVID-19 may teach us about interdisciplinarity’, BMJ Global Health 2020, no. 5: e004375. https://doi.org/10.1136/bmjgh-2020-004375.
Munafò, R., B. Nosek, D. V. M. Bishop, K. S. Button, C. D. Chambers, N. Percie du Sert, U. Simonsohn, E.-J. Wagenmakers, J. J. Ware and J. P. A. Ioannidis (2017), ‘A manifesto for reproducible science’, Nature Human Behaviour 1, no. 1: 1–9. https://doi.org/10.1038/s41562-016-0021.
NWO (Dutch Research Council) (2020), ‘Jaarverslag NOW’, https://www.nwo.nl/jaarverslag (accessed 15 February 2021)
Pearson, H. (2021), ‘How COVID broke the evidence pipeline. The pandemic stress-tested the way the world produces evidence — and revealed all the flaws’, Nature 593: 182–185. https://doi.org/10.1038/d41586-021-01246-x.
Penders, B., J. B. Holbrooke and S. de Rijcke (2019), ‘Rinse and repeat: Understanding the value of replication across different ways of knowing’, Publications 7, no. 3: 52–66. https://doi.org/10.3390/publications7030052.
Pols, J. (2008), ‘Which empirical research, whose ethics? Articulating ideals in long-term mental health care’, in G. Widdershoven, J. McMillan, T. Hope, and L. van der Scheer (eds), Empirical Ethics in Psychiatry (Oxford: Oxford University Press), 51–68.
Pols, J. (2015), ‘Towards an empirical ethics in care: Relations with technologies in health care’, Medicine, Health Care and Philosophy 18: 81–90.
Pols, J. (2022), ‘Care as an attempt to do something good: Studying normativity in practices of health care and research’, in L. Cubellis and R. Lester (eds), Traces of Care: Discernment and the Work of Recognition (Durham, NC: Duke University Press).
Pols, J. (2023), Reinventing the Good Life. An Empirical Contribution to the Philosophy of Care (London: UCL Press).
Rathenau Instituut (2020), Tijdelijke contracten bij universiteiten in perspectief. [Temporary contracts at universities in perspective], https://www.rathenau.nl/nl/wetenschap-cijfers/wetenschappers/personeel-aan-de-universiteiten-en-umcs/tijdelijke-contracten-bij (accessed 15 February 2021).
Rathenau Instituut (2023), ‘Aanvraagdruk bij NWO’ [Application pressure at NWO], https://www.rathenau.nl/nl/wetenschap-cijfers/werking-van-de-wetenschap/excellentie/aanvraagdruk-bij-nwo (Accessed 11 February 2024).
Rorty, R. (1979), Philosophy and the Mirror of Nature (Princeton, NJ: Princeton University Press).
Rossel, P. (2004), ‘Implementing outward-bound academic entrepreneurship in the human sciences’, Acta Academica 36, no. 3: 111–139.
Rothman, D. J. (1987), ‘Ethics and human experimentation’, New England Journal of Medicine 317, no. 19: 1195–1199.
Ruppert, E., J. Law and M. Savage (2013), ‘Reassembling social science methods: The challenge of digital devices’, Theory, Culture & Society 30, no. 4: 22–46. https://doi.org/10.1177/0263276413484.
Sismondo, S. (2009), ‘Ghosts in the machine: Publication planning in the medical sciences’, Social Studies of Science 39, no. 2: 171–198. https://doi.org/10.1177/0306312708101047.
Sismondo, S. (2017a), ‘Hegemony of knowledge and pharmaceutical industry strategy’, in D. Ho (ed.), Philosophical Issues in Pharmaceutics. Development, Dispensing, and Use (Dordrecht; Springer), 47–63.
Sismondo, S. (2017b), ‘Not a very slippery slope: A reply to Fuller’, EASST Review 36, no. 2, https://easst.net/easst-review/36-2/not-a-very-slippery-slope-a-reply-to-fuller/.
SRC (Swedish Research Council) (2017), Good Research Practices (Stockholm: Swedish Research Council).
Thévenot, L. (2001), ‘Pragmatic regimes governing the engagement with the world’, in T.R. Schatzki, K. Knorr-Cetina and E. von Savigny (eds), The Practice Turn in Contemporary Theory (London: Routledge), 56–73.
Thygesen, H. and I. Moser (2010), ‘Technology and good dementia care: An argument for an ethics-in-practice approach’, in M Schillmeier and M Domènech (eds), New Technologies and Emerging Spaces of Care (Farnham: Ashgate), 129–147.
Tjeenk Willink, H. (2018), Groter denken, kleiner doen [Think bigger, act smaller] (Amsterdam: Prometheus).
VSNU, NFU, KNAW, NWO en ZonMw (2019), ‘Ruimte voor ieders talent; naar een nieuwe balans in het erkennen en waarderen van wetenschappers’ [Space for everyone's talents: towards a new balance in Recognition and Rewards for scientists], https://www.nwo.nl/sites/nwo/files/documents/2019-Erkennen-en-Waarderen-Position-Paper_NL.pdf.
Wagenmakers, E. J. and D. Borsboom (2013), ‘Derailed: The rise and fall of Diederik Stapel’, APS Observer 26, no. 1, https://www.psychologicalscience.org/observer/derailed-the-rise-and-fall-of-diederik-stapel.
Washington, H. A. (2006), The Dark History of Medical Experimentation on Black Americans from Colonial Times to the Present (New York: Doubleday, Anchor Books).
Willems, D. L. (2010), ‘Varieties of goodness in high-tech home care’, in A. Mol, I. Moser and J. Pols (eds), Care in Practice: On Tinkering in Clinics, Homes and Farms (Bielefeld: Transcript Verlag), 215–234.
Willems, D. and J. Pols (2010), ‘Goodness! The empirical turn in health care ethics’, Medische Antropologie 23, no. 1: 161–170, https://api.semanticscholar.org/CorpusID:147106420.