In a hilarious instance of meta-irony, a study published last week claims that more than half of social science studies are bogus. The study, conducted by the Reproducibility Project, a research accountability group at the University of Virginia, examined 100 studies published in three leading peer-reviewed journals and found that 60 of them did not match the author’s conclusions.
This study highlights the importance of reproducibility, a defining factor of scientific inquiry: If the results cannot be reproduced, they are false.
“Reproducibility is extremely important, because if it isn’t reproducible, you can’t use it,” UT psychology professor Bill Swann said. “I reproduce my own studies in the paper. They aren’t direct replications, but they are conceptual replications.”
Conceptual replications are when the more important aspects of the study are replicated in order to ensure the results are conceptually correct. Swann said he believes the best scientists replicate their own studies themselves, such as double-checking your work before you turn it in.
The Reproducibility Project’s study comes at a critical juncture in the world of social science, where increasing media attention coupled with a growing pool of unemployed Ph.D. students can lead to overstating the statistical significance of findings, or worse, fudging the data in order to get published.
“There will always be bad apples,” Swann said. “But the majority of us are just trying to do good science.”
The implications of the Reproducibility Project’s work are not all bad. However, it seems as though the work is indicative of a new trend of hypersensitivity. With a sharply rising number of journal retractions and retroactively falsified findings, Swann points to an increase in awareness on the matter.
“I think there’s a lot more sensitivity and skepticism because of data fakers getting exposed,” Swann said. “Communication is so much better, so if you suspect someone of doing something, they’re an email away. Ultimately, you’ll get more moderating and more replication, and eventually retraction will die down.”
The key point is scientists need to reproduce studies. It does not matter whether the project is replicated by the original author or done by a group — just that it is done. The problem is that reproduction is not flashy, nor is it novel. It only reiterates what is already known when most scientists are looking for what is unknown.
“It’s not sexy, but it’s important,” said Swann.
And it’s important that the public clamors for reproduced studies so that we can be sure our scientific knowledge is steadily encroaching on the truth, rather than flashy falsehoods.
David Bordelon is a philosophy sophomore. He is a guest columnist.