On a cloudy October afternoon in 2015, I set off on a drive to my partner’s home. I was living apart from her while finishing my PhD, as many early-career researchers (ECRs) do. The drive through New Brunswick, Canada — from Saint John to Moncton — was long and boring, but I made a point of visiting every weekend.
I don’t remember much until I pulled up to a toll gate in the next province, nearly two hours past Moncton — I had driven for three hours and didn’t remember any of it.
I was suffering from an acute bout of imposter-syndrome-driven depression and anxiety, which in total lasted for about four weeks. Absent-mindedness and an inability to focus on much of anything were near the less-severe end of my symptom list.
An on-campus mental-health professional helped me to realize that my depression was driven, at least in part, by repeated manuscript rejection and harsh reviewer comments related to an experiment I had conducted earlier in my PhD. In an effort to address this, my supervisor hired an undergraduate student to help me replicate the experiment. When the student confirmed my previous results, this, along with a wonderful support system outside work, turned my well-being around. For me, replicable scientific results and mental health were linked.
Successful replication of the results of a previous experiment, in that instance, was able to alleviate my depression. But what might have happened to me had I been unable to reproduce my findings? And what might be happening to other ECRs who are unable to reproduce their own or others’ results? It is no secret that many scientific disciplines have low rates of reproducibility, often dubbed the ‘reproducibility crisis’. It is also apparent that ECRs — who are often on the front line of cutting-edge scientific inquiry — have an extremely high prevalence of mental illness1,2.
This association remained in my thoughts when I arrived in Trondheim, Norway, to begin a new postdoc in 2018, because my supervisor, Fredrik Jutfelt, frequently mentioned that he was often unable to reproduce clear results found in the literature. With this in mind, we wanted to know how common it might be for an inability to reproduce results to affect the mental well-being of ECRs. We conducted a Twitter poll among scientists and found that 40% of 53 respondents said irreproducibility played at least some part in mental-health problems or feelings of imposter syndrome during their graduate studies.
Although there are limitations to anonymous online surveys such as this, I know from my own experience that failing to reproduce previously published results can cause extreme levels of stress, because students often interpret an inability to replicate experiments as a reflection of their own ability.
So what can we do about this problem? For me, knowing that a failure to replicate results is not a reflection of my ability as a researcher was crucial. If I had known that failing to replicate findings did not mean that I was incompetent, it might have helped me to avoid that dark period of depression during my PhD — or at least to overcome it more quickly. In fact, not being able to replicate previous results is more than OK — it is common, and often correct. Initial studies might have poor methodologies, a lack of transparency and a wealth of biases3. Having understanding supervisors throughout my career — people who recognize irreproducibility as a healthy part of science — has helped to shape the way I think about reproducibility.
An acknowledgement from the culture of science that negative results aren’t such a bad thing would have helped me. I feel that direct training on how to effectively recognize and produce transparent and reproducible research (from experimental design through to publication) would have helped to alleviate my stress and improve my mental well-being. In my experience, such training is not common but would be worth developing. Discussions focused on the importance of publishing negative and null results would also be of benefit.
It is of the utmost importance that we address reproducibility if we are to eliminate the proliferation of false positives in scientific discovery. Science must also improve the dismal mental well-being of its workforce. I’m not aware of any studies that investigate the direct effect of reproducibility on mental health but would welcome one: I suspect I’m not alone in my experience of one issue affecting another. Issues in reproducibility must be addressed, not just for the well-being of science as a process, but for the well-being of scientists as people.