Another problem is that press releases issued by universities, colleges,
and various other institutions are frequently announcing scientific
research in ways that include exaggerations, unwarranted claims or
outright falsehoods. A
scientific paper reached the following conclusions, indicating a huge
hype and exaggeration crisis both among the authors of scientific
papers and the media that reports on such papers:
Another huge problem involves what is called the Replication Crisis. This is the fact that a very large fraction of scientific research results are never replicated. The problem was highlighted in a widely cited 2005 paper by John Ioannidis entitled, “Why Most Published Research Studies Are False.”
Although physics is often regarded as a more “hard” and reliable form of science, there is still tons of wobbly speculation in the world of cosmology and theoretical physics. An example was the recent paper by three cosmologists claiming to have found evidence of something called “Hawking points” in the cosmic background radiation, which they interpreted as supporting their cyclical theory of the universe that almost no one else but them believes in. A cosmologist studying the cosmic background radiation for the faintest traces of something he wants to believe in may be compared to someone who checks his toast with a magnifying glass every day, and eventually reports something that he thinks looks a little like the face of Jesus.
Thirty-four
percent of academic studies and 48% of media articles used language
that reviewers considered too strong for their strength of causal
inference....Fifty-eight percent of media articles were found to have
inaccurately reported the question, results, intervention, or
population of the academic study.
Another huge problem involves what is called the Replication Crisis. This is the fact that a very large fraction of scientific research results are never replicated. The problem was highlighted in a widely cited 2005 paper by John Ioannidis entitled, “Why Most Published Research Studies Are False.”
Although physics is often regarded as a more “hard” and reliable form of science, there is still tons of wobbly speculation in the world of cosmology and theoretical physics. An example was the recent paper by three cosmologists claiming to have found evidence of something called “Hawking points” in the cosmic background radiation, which they interpreted as supporting their cyclical theory of the universe that almost no one else but them believes in. A cosmologist studying the cosmic background radiation for the faintest traces of something he wants to believe in may be compared to someone who checks his toast with a magnifying glass every day, and eventually reports something that he thinks looks a little like the face of Jesus.
What
rule can you use to distinguish between solid well-established
science on the one hand and hype and speculation on the other hand?
Can we simply use the rule of “trust something if you read it one
of the top science publications like Science or Nature or Scientific
American, but maybe be skeptical if you read about it in a
publication or web site of lesser stature?” No, this principle does
not at all work. Nowadays the most respected science
publications often contain quite a few misleading headlines proclaiming as
discoveries dubious research results that do not at all qualify as
discoveries. For example, in the leading science journals, we very often find neuroscience experiments done with too few test animals, such as only 7 (15 test animals per study group is the minimum for a moderately reliable result).
It
is also not at all true that you can rely on the truth of a science
headline that you read in the New York Times, since the writers at
this publication almost never show signs of critically scrutinizing
dubious claims by scientists and university press releases. Nor is
it true that you can count on a research result that is directly
stated in the title of a scientific paper. Scientists frequently give
their papers dubious titles announcing results they have not proven.
Nor is it true that you can count on a result announced by a
distinguished college or university such as MIT. Nowadays the press
offices of colleges and universities are notorious for the dubious
hype of their press releases, and this problem is not at all confined
to less prestigious academic institutions. In this post and in the series of posts I
have labeled “overblown hype” you will find many examples to back
up the claims I have made in this paragraph.
Can
we perhaps distinguish between solid science and unproven speculation
by following the principle “trust in things that most scientists
believe”? No, because of some of the reasons discussed in this
post and this post. Unfortunately, communities of experts can become ideological
enclaves, and in such enclaves it is all too easy for a majority to
reach an opinion that is not well established, once that opinion
becomes “all the rage” in that community.
There
is actually no reliable process in place for determining what
doctrines are believed in by a majority of scientists. It is quite
unreliable to try to gauge the opinion of scientists by analyzing
scientific papers, because a scientific paper may repeat standard
shibboleths to increase its chance of getting published, and it's
hard to tell how much the authors believe in such customary
utterances. An opinion poll of scientists is a more reliable way of
measuring their opinions. But most such polls suffer from defects,
such as offering too little choice, and not offering an option of “I
don't know” or “I'm uncertain about this.” Some opinion polls
of scientists also require them to publicly assert their opinions to their superiors, which is not a reliable way of measuring
private opinion.
The
most reliable way to measure opinions on a topic is a secret ballot.
But there is no process in place for measuring the opinions of
scientists through a secret ballot. Furthermore, common opinions
regarding a consensus of scientists are based entirely on impressions
got from US and European scientists. A true global measure of
scientific opinion (including all the scientists in India and China)
might have many surprises. In light of all these difficulties, it is
not a particularly reliable or useful guideline to try to distinguish
between strong science and less reliable science claims by using
common opinions about which things most scientists believe in or
don't believe in.
But
I can think of one simple “rule of thumb” principle that is
pretty good for distinguishing between solid topnotch science on the
one hand and weaker claims on the other hand. The principle is what I
call “Nobel's Razor.” This is simply the principle: if some
science claim has won a Nobel Prize, regard it as topnotch “Grade
A” science, but if no one has ever won a Nobel Prize for
establishing the claim, regard it as something less than topnotch,
well-established science.
The
Nobel Prize committees award annual prizes in physics, chemistry, and
medicine and physiology. Over the years, the Nobel Prize committees
have been extremely good about awarding prizes only to very solid
scientific results (with a handful of exceptions). The Nobel Prize committees “wait for the dust
to settle,” almost always avoiding giving a prize to any research
result until its solidity has been established over a period of
several years. For example, Penzias and Wilson discovered the cosmic
background radiation in the mid-1960's, but had to wait until 1978
before getting their richly deserved Nobel Prize in Physics.
There
are some interesting examples of things that are claimed to be
examples of established science, but which have never won any Nobel
prizes. One such example is that no one has ever won a Nobel prize
for any work establishing Darwin's theory of evolution by natural
selection, nor any work helping to establish Neo-Darwinism. This is a great
embarrassment to Darwin enthusiasts. It is true that Darwin died
before the Nobel prizes were established. But we may ask: if
Darwinism is really a topnotch scientific result, why has there been no
Nobel Prize for any type of research work done to establish such a theory?
Other
interesting examples of widely-repeated claims by scientists that
have no Nobel prizes in their favor are the opinion that the human
mind is a product of the brain, and the opinion that human memories
are stored in our brains. No one has ever won a Nobel prize for
research helping to establish such ideas. The link here shows the
scientists who won the Nobel prize in medicine and physiology, and
what research they did to win the prize. None of the prizes are for
work involving memory, consciousness, or the relationship of the
brain and the mind.
You
can see here a list of all people who have won Nobel prizes in
physics. No one has ever won for any research on dark
matter, dark energy, the multiverse, or the “cosmic inflation”
claim that the universe underwent an instant of exponential expansion
(not to be confused with the more general theory of the Big Bang).
Do
all these omissions mean that this “Nobel's Razor” principle is
not a good way of distinguishing between topnotch well-proven science on the one hand and lesser claims that are not very well
proven? No, such omissions help to establish the solidity of such a
“Nobel's Razor” rule-of-thumb, and to help show that the Nobel
committees have been excellent about only giving awards to results
that are well established by observations or experiments. When people press you to believe in some science claim that is not topnotch science demonstrated by observations or experiments, you can ask such persons, "Why should I believe in that when no one ever won a Nobel Prize for establishing it?" Such a question will not be an effective reply to assertions about global warming, seeing that the IPCC committee was awarded a Nobel Peace Prize.
Postscript: Despite it being a good "rule of thumb" to trust science that got a Nobel Prize, in some cases the Nobel Prize committee awards science Nobel Prizes it should not have awarded. A notable case (the case of Christian Anfinsen) is discussed in my post here, which notes misstatements in one year's press release for a Nobel Prize.