The response to my TEDx Quran talk in the week since the video of it was released has been magnificent, and the comments here on the AT, on YouTube, and on many other sites, have been immensely moving, humbling, thought-provoking, and inspiring, sometimes all at the same time.
All of which means that I haven’t had much time for my usual reading. So when I finally curled up late last night with last week’s New Yorker, I thought it would just be a relaxing break. Instead, Jonah Lehrer’s densely argued article on scientific research called “The Truth Wears Off” had me sitting straight up in alert attention, and his conclusion left me kind of breathless:
We like to pretend that experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true. When the experiments are done, we still have to choose what to believe.
This article is dynamite. I mean that almost literally, since Lehrer explodes the carefully nurtured image of scientific research as the ultimate arbiter of fact. It demonstrates what he calls “the slipperiness of empiricism.” And it challenges the faith in science – I use the word “faith” advisedly here – of “new atheists” like Sam Harris, Chris Hitchens, Richard Dawkins, and Daniel Dennett (a foursome I think of H2D2 for short).
Lehrer focuses on what he calls “the decline effect” – the way much research across a wide range of fields, including psychology, ecology, biology, and medicine, fails the most basic scientific test, which is replicability. If other researchers can produce the same results, then the research is valid; if they cannot, it has to be assumed flawed. And many, in some fields even most, cannot. Results that initially seem ground-breaking often fade with each replication of the research, and sometimes even turn negative.
“It’s as if our facts were losing their truth,” says Lehrer. As one somewhat depressed biologist told him:
We cannot escape the troubling conclusion that some – perhaps many – cherished generalities are at best exaggerated in their significance, and at worst a collective illusion nurtured by strong a-priori beliefs.
Citing the example of wildly divergent results for the efficacy of acupuncture in the Far East as opposed to Western Europe, for instance, Lehrer writes:
This wide discrepancy suggests that scientists find ways to confirm their preferred hypothesis, disregarding what they don’t want to see. Our beliefs are a form of blindness.
It gets downright scary when he talks to a Stanford epidemiologist who looked at the 49 most cited clinical-research studies in three major medical journals. “Of those that had been subject to replication, 41% had either been directly contradicted or had their effect sizes significantly downgraded.” It gets even worse when the research is in a “fashionable” field like genetic differences in disease risk for men and women. “Out of 432 claims, only a single one was consistently replicable.”
The Stanford epidemiologist suspects “significance chasing,” where “scientists are so eager to pass the magical test of statistical significance that they start playing around with the numbers, trying to find anything that seems worthy.” They don’t do this deliberately, but unconsciously. “Knowing” that something has to be true, they find ways to make it seem so.
“The decline effect” may in fact be a decline of illusion, says Lehrer. The problem being that it’s very hard to let go of illusions:
Although many scientific ideas generate conflicting results and suffer from falling effect sizes, they continue to get cited in textbooks and drive standard medical practice. Why? Because these ideas seem true. Because they make sense. Because we can’t bear to let them go.
Because, in short, scientists are as human as the rest of us. And the very idea of truth — that absolute ideal of veracity — is as slippery and troubling as ever.
This reminds me of my personal fasination (and horror) when I went from a idealist English Major to working in a bank marketing department… Given the assignment to research and write a report on the numbers presented on various accounts… I found I could make three or four different conclusions with the same numbers… ie: I had the power to make the numbers say what I wanted them to say! It scared the innocense out of me.
Thought provoking article. As I have alluded to in my writing, “faith” or “religion” can infect philosophy and science, not just religious institutions. As I see it, it’s not that “truth” is a problem, as much as it’s human bias that is a problem. It’s usually unconsciously applied, as you say, but so easy for people to get on a bandwagon that’s based on one source saying, “new research suggests…” before the full gamut of research has been completed.
We tend to reinforce what we already believe, whatever side we are on. Publishers still get textbooks on evolutionary theory wrong (e.g. Haeckel’s drawings of embryos repudiated as “falsified” by Stephen J. Gould, et al, can still be found), as do Creationists who blindly follow selective “evidence” for a young earth despite the preponderance of more replicable tests.
I recommend Francis Wheen’s “How Mumbo-Jumbo Conquered the World: A Short History of Modern Delusions” as a thoughtful exploration of this sort of anti-scientific relativism.
Thanks for your TED talk earlier this fall! We’ve just posted a piece on it (http://insideislam.wisc.edu/index.php/archives/6109) and hope that you can spread the word on the initiative we’re doing here at the University of Wisconsin to spread voices such as yours. We look forward to your upcoming book!
Re: “They (scientists) don’t do this deliberately, but unconsciously. “Knowing” that something has to be true, they find ways to make it seem so.”
> This is an atrocious article. Mumbo jumbo.
This is exactly opposite to what is involved with proper science. Scientists welcome repeated examination to disprove an assertion. This is how an assertion is either made stronger, or is rightly abandoned because of a more compelling explanation.
Your point being the point of Lehrer’s article, no? Plus the basic epistemological question: what is “proper” science?
I find it striking that we are still trying to kill empiricism/logical positivism 60 years after it was put to death unequivically by the new philosophers of science (Quine-Popper-Davidson-Feyerabend-Kuhn et al). Not to mention the critical realists who point out how it all comes down to how language and culture mediates between the empirical reality and the perceiver.
Unfortunately most scientists have a very narrow field definition of what constitutes adequate criteria for their epistemic enterprise (they are mostly philosophical illiterates), and usually depend on their experimental procedures for that criteria. This is scientism, or what I call the technologization of science (a very pragmatic orientation). H2D2 fit into this category quite clearly.
Real science is about discovery, which depends on a type of creativity that unconditions the mediated reality. It is based on conjecture, not on instrumental “proofs” that come later.
There are no truths, only vested interest where human relations are concerned.
Remember the astronemer who made drawing of canals on Mars? Maybe his mind supplied him to match his intentions of wanting to find intelligent life on another planet.
Our minds create what we want to believe, subjectively & objectively.
We see what our mind wants to see… We remember what our mind is interested in… Objectivity is always colored by our subjectivity… all what we do is a mere approximation to reality… wallahu alam…
Below is a relevant and highly recommended read: