The lost art of scientific verification

Jacob Schmidt

A recent study demonstrated that eating chocolate can make you lose weight. Dr. Johannes Bohannon, the research director of the Institute of Diet and Health and author of the study, showed eating 48 grams of chocolate per day could increase weight-loss results by 10%. The ground-breaking research made the front page of many top print and online newspapers, blogs and magazines around the world. So, why was the study retracted only a few days after publication?

Because chocolate does not make you lose weight. In fact, The Institute of Diet and Health is fake, and the whole study was a scheme designed by journalist John Bohannon to expose just how easily bad science can hack both our media and reading public.

Intentional scientific fraud is rare, but sloppy science is virulent and elusive within media publication. Even honest researchers and scrupulous publishers struggle to abstain from p-hacking, confirmation bias, the file drawer problem and other lingering, malignant tendencies. Since fact-checking takes massive energy, readers willfully cede to the media their right — their obligation — to critical analysis. It is often easier to trust the legitimacy of published sources than invest in our own ability to distinguish true from false, although this habit can prove faulty.

“In many journalism outlets there is no one dedicated to covering these important and often complex topics. With fewer of these specialists, the reporting falls to general assignment reporters who may not be as prepared to cover the weight of scientific evidence,” Kris Wilson, a senior lecturer in UT’s School of Journalism, said. “Reporters who cover science full time have a much better understanding of the science and can then translate that more clearly and accurately to their audiences.”

An old Russian maxim commonly attributed to President Ronald Reagan urges us to “trust, but verify” our sources of information. While there is nothing inherently wrong with trusting the news, we begin to have serious issues when we indiscriminately outsource the verification process to it. That is when we get rumors about candy-fueled weight loss circulating on the front pages of Europe’s largest newspapers.

Sadly, the problem traces much deeper. Our grade schools are struggling to teach good analytical and experimental practices, according to Arturo De Lozanne, molecular biologist and member of UT’s Academy of Distinguished Teachers.

“I put a good statistical understanding at the top of the [education system’s priorities], next to sex education,” De Lozanne said. “There’s an incredible lack of general science skills in the lower grades.”

In general, science education helps us develop the open-minded skepticism of conscious news consumers, which saves us from the tragedy of chocolate scandals. In the case of the phony chocolate study’s media attention, De Lozanne says, “it wasn’t just Vanity Fair [that picked up the story] — it was everybody. It spread like wildfire.”

While not all gossip surrounding science is categorically unfounded, there’s a marked difference between interesting reporting and negligence.

“It was clear the media only spent about three seconds reading the study before reporting it because it’s hot, it’s cool,” De Lozanne said. “Nobody stopped and said, ‘Wait, what? Really? Can we talk to the scientists?’”

For the betterment of society, we need to hold both researchers and media accountable for their publications. But improvement starts at home: don’t trust outright what you hear, read, or see without verifying the data behind the claim yourself. Read primary sources.

Schmidt is a physics sophomore from Austin. Follow him on Twitter at @heyjakers.