Scientists Have a Hundred Ways To Conjure Up Phantasms That Don't Exist

Well, that was exciting while it lasted. 

Phosphine is a gas mainly produced on Earth by living things. On September 14, 2020, a scientific paper entitled "Phosphine gas in the cloud decks of Venus" claimed to have detected the "apparent presence" of phosphine in the atmosphere of Venus. Following their long-standing tendency to hype like crazy any story that may serve as clickbait to produce more page views and advertising revenue, a host of web sites began proclaiming that life or a sign of life had been discovered at Venus.  

I may have been the first one in the blogosphere to make a substantive criticism of this claim, since I published on the next day a post entitled "No, They Haven't Detected Life at Venus," in which I cited several reasons for doubting the idea that the paper had provided any evidence of life at Venus.  I said on September 15 that an alternate explanation was that "an error in interpretation could have occurred in spectral data that is hard-to-interpret because of overlapping signals from a variety of different gases in the atmosphere of Venus," and that such a possibility was "not very unlikely." I also pointed out the substantial chance that merely geological processes could have produced phosphine, and pointed out the lack of any plausible scenario for life on Venus, given the incredibly hostile conditions both on its hot-enough-to-melt-lead surface and its clouds (having almost no water but lots of sulfuric acid vapor). 

Days after my post, we began seeing on some sites such as www.liveScience.com and the National Geographic web site some articles (such as this one) questioning whether any sign of life had really been discovered on Venus.  On October 19, 2020 there appeared a scientific paper with the title "Re-analysis of the 267-GHz ALMA observations of Venus: No statistically significant detection of phosphine."  The paper doesn't merely question whether evidence of life on Venus has been found. The paper tells us that no robust evidence of phosphine has been discovered in the atmosphere of Venus. 

The five authors of the scientific paper provide a critique of the  September 14 paper claiming evidence for phosphine in the atmosphere of Venus,  saying that the paper used a dubious statistical technique that "leads to spurious results." The five authors state that the data actually provides "no statistical evidence for phosphine in the atmosphere of Venus." 

If the five authors are right, then we have here another example of what goes on so very often: scientists conjuring up a phantasm by using dubious methods that lead them to suggest the existence of something that does not really exist.  What we should never forget is that when you're a scientist, there are always a hundred ways for you to get some illusory evidence that is just a false alarm.  This doesn't require or typically involve any outright deception. It merely requires for a scientist to use some method that isn't quite right. 

One extremely common way for a scientist to conjure up a phantasm is to use too small a sample size or too small a study group size, which tends to result in false alarms. Scientists know how they can avoid this sin: by doing a sample size calculation to determine the minimum study group size needed to produce a moderately persuasive result, and to only use study groups with such a size. But a large fraction of scientific studies (particularly animal neuroscience studies) fail to include such a calculation, and fail to have adequate study group sizes. 

Another way for a scientist to conjure up a phantasm is to prune or filter his data until the desired thing seems to appear. It might be that when considering his full data set, there will seem to be no evidence of some thing (call it X) that the scientist wants the data to show evidence for.  But the scientist can prune the data at its beginning or end, until some evidence of X seems to show up. For example, if there were 4 weeks of data collection, the scientist can just get rid of the week 1 data or the week 4 data, or maybe both weeks of data. Or, the scientist can apply some "data filter" which gets rids of certain data points, until some evidence of X seems to show up.  The decision to apply such a data filter can often be rationalized in various ways, to make it sound like some "quality filter" excluding "bad data" or "outlier data."

Another way for a scientist to conjure up a phantasm is to collect data in a biased way that will maximize the chance that the desired result will appear. I will give a hypothetical example. Let us imagine that you are a scientist who wants to show that rainy days in New York City cause a higher chance of 300-point drops in a stock market indicator such as the Dow Jones Industrial Average.  You might begin recording daily stock market results on a rainy day in which there was a 300-point drop in the stock market.  You might then continue to record daily results, and conveniently end your data collection on a rainy day in which there was more than a 300-point drop in the stock market.  Given such convenient start and stop points of your data collection, you may well be able to write up a "statistically significant" correlation between rainy days in New York and 300-point drops in the stock market. But if you had resolved beforehand to start collecting data on some day 14 days in the future, and continue collecting data for exactly 100 days, then the desired result would probably not show up. 

Once so-called "raw data" has been collected, there are 1001 ways to "massage" the data before it is analyzed, some of which may make sense and some of which are dubious. A scientist can produce all kinds of rationalizations for particular data exclusions and data inclusions and data averagings and "data smoothings" and "weighted averagings" that may have been used, which can have a huge effect on whether some illusory phantasm shows up. 

Moving from the topic of data collection to data analysis, there are innumerable ways in which a scientist can conjure up phantasms by some kind of data analysis that isn't quite right. Don't be reassured when some science paper claims that it uses some kind of "standard software" for data analysis. There is almost always no such thing as a "standard analysis" of data. There are standard software tools used for data analysis, but such tools can be used in a million valid ways, and a million dubious ways. An example is Microsoft Excel, the leading spreadsheet program.  There are a million bad ways to use it, as well as a million good ways. 

A science paper may try to reassure you that it used some standard software for doing some type of data analysis (such as measuring brain scan data or trying to measure "freezing behavior" in mice).  But there are always countless different ways to use such software, some good and some bad.  Every software program has program settings or startup options or menu options that allow you to customize how the program is used. Software programmers usually ask "how can I give the user the freedom to do exactly what he wants," and almost never ask "how I can make it so that there's no way to use the software in a stupid way." 

Another way a scientist can conjure up phantasms is by failing to do a preregistered study that announces an experiment will be testing one very specific hypothesis, and going on a kind of "fishing expedition" within his analytic activity. For example, let's imagine the scientist does brain scans looking for some correlation between some behavior (or some aspect of thinking) and activity in some brain region. By failing to limit himself to checking one small specific part of the brain corresponding to a previously declared hypothesis, and giving himself the freedom to check any of 100 small parts of the brain, he will have a good chance of finding some tiny region that weakly correlates with the behavior or aspect of mentality.  That's simply because given 100 parameters that show random variations, and the freedom to check any of them, it's easy to get something that looks like a slight correlation, even if only chance and not causation is involved.  The name sometimes given for this procedural sin is HARKing, which stands for Hypothesizing After Results are Known. 

Then there's the matter of what the scientists wishes to call attention to in writing up the paper.  Examining exactly the same data, one scientist may choose to make some little data anomaly get special treatment by referring to it in the paper title, while another scientist may "bury" such a data anomaly by mentioning it only in a footnote or in the paper's "supplemental information."  

Or a scientist can conjure up some phantasm by simply having the paper announce something was found that was not actually found. This is a surprisingly common sin. A scientific paper found that 34% of scientific papers "used language that reviewers considered too strong for their strength of causal inference."  Such unfounded claims often occur in the title of scientific papers, which often proclaim some result that is not justified by anything in the paper. 

A scientist may conjure up a phantasm by carefully cherry-picking some examples supporting some speculative idea, while conveniently failing to mention other examples that argue against such an idea. I have recently read two scientific papers that mentioned some evidence for some speculative idea, and conveniently failed to mention the largest studies done on the topic, which presented data against such an idea. 

A scientist may conjure up some phantasm by using some loaded term that isn't appropriate. For example, he may refer to some brain region being "activated" when some mental event occurs, even though the brain region is actually constantly active, and is merely showing something like a 1% greater-than-average activity, something that will randomly occur in different brain regions when nothing special is occuring.  Or the scientist may refer to a "gene regulatory network," rather giving you the idea that genes act like some committee members making decisions. Genes are mere mindless low-level chemicals.  Or the scientist may claim that some behavior is caused by "hard-wiring" in the brain, even though no one has ever found any brain structure (other than reflexes) that determine behavior. 

The things mentioned above are generally not clear-cut deceptions. But it does sometimes happen that scientists conjure up a phantasm by simply using language that is just plain false.  For example, many a scientist has spoken of DNA as being a blueprint or recipe for making a human. Containing only low-level chemical information and no instructions for making anatomy, DNA is no such thing, and bears no resemblence to either a blueprint or a program or a recipe for making a human or any other organism.  "DNA as a blueprint for organisms" is one of the most notorious phantasms of scientist literature. 

Scientists have a hundred ways to conjure up illusory phantasms, and once a phantasm has been conjured up, there are many tricks by which the phantasm may be made to seem like something real, such as the use of complex charts and thick jargon which make the problematic presentation seem very scientific. All in all, we may say that the power of scientists to give you an impression of the reality of something illusory is comparable to the similar power of Hollywood's CGI special effects wizards. 

Postscript: Now we have two additional papers saying that there is no phosphine in the atmosphere of Venus. One paper by a single author states, "There is thus no significant evidence for phosphine absorption in the JCMT Venus spectra." Another paper with many co-authors is entitled, "No phosphine in the atmosphere of Venus." 

The case of the "phosphine at Venus" error might give you the impression that scientist errors are quickly corrected. That sometimes happens, but other scientist errors may become enshrined and persist for decades or centuries.  A particularly notorious example is the case of the fraudulent or erroneous embryonic drawings made by biologist Ernst Haeckel.  The drawings were made to try to give evidence for the idea that early stages of the human form resemble animals from which humans were originally descended.  For well over 130 years, the bogus drawings were some of the chief evidence cited for the doctrine of common descent, the idea that all organisms have a common ancestor. You may still find these bogus drawings in evolutionary textbooks at your local library. 

Pareidolia Can't Explain the Latest Shocking Mars Strangeness

The Mars rovers such as Opportunity and Curiosity have photographed an astonishing variety of anomalous-looking things on Mars. They are discussed in this series of blog posts. You can find more examples by doing a Google image search for “Mars anomaly.”

Two recent examples are the "Mars mouse" shown at the top of this original NASA photo, and the Mars dome shown at the top of this original NASA photo. The composite photo below shows the "mouse" and "dome."

In September 2015 one of the Mars robotic probes photographed what looks like a long spoon, one that rather seemed to be levitating above the ground. Now a newly examined photo from Mars shows another “long spoon.” Below are the photos side by side. The original NASA photos can be found here and here. 

 

When such anomalies occur, skeptics are quick to trot out what they call a “scientific explanation”: pareidolia, the supposed tendency of the mind or brain to interpret random pieces of matter as being something familiar, like a face or a spoon.

Is this actually a scientific explanation? No, it isn't. Evoking pareidolia is just another way of saying something is a coincidence. Whenever you say that something is coincidental, you never are giving an explanation; you are instead really indicating that you have no explanation.

Imagine the case of someone who photographs a cloud that looks very much like a human figure. There are two ways in which a skeptic might respond: he might claim that it was just a coincidental arrangement of matter, or he might say that it was pareidolia that the cloud is being compared to a human. But both responses are basically the same thing.

Both responses involve two assumptions:

1. There was merely a coincidental random arrangement of matter.
2. The mind interpreted this random arrangement of matter as something meaningful.

If I merely say that it was just coincidence, I am explicitly stating the first of these two, and leaving the second as an unstated, implicit assumption. If I merely say that it was pareidolia, I am explicitly stating the second of these two assumptions, and leaving the first assumption as an unstated, implicit assumption. But it's the same thinking in both cases. So appealing to pareidolia really is just saying “It was a coincidence.” And you never explain something by saying it was a coincidence. When you say, “It was a coincidence,” you are really saying, “I have no explanation.”

Consider these cases:

• You are gambling with a poker dealer who deals himself five consecutive royal flushes in spades (the highest possible poker hand). When you ask him to explain this astonishing stroke of luck, he says, “It was just a coincidence.”
• You have a son who comes home five consecutive days with a suitcase of money, which the son claims are casino winnings. When asked to explain how he could have been so lucky on five days in a row, the son says, “It was just a coincidence.”
• You find out about the scientific fact that the electric charge on each electron in the universe is precisely the same (to twenty decimal places) as the electric charge on each proton in the universe, the only difference being that the sign is opposite. When asked to explain this, a scientist says, “It's just a coincidence.”

Now, are any of these statements actual explanations? No, they are non-explanations. The person appealing to coincidence is always someone saying the equivalent of, “I have no explanation.” An explanation of a physical thing occurs when you mention some causative factor that explains the characteristics of the thing. You do no such when you evoke pareidolia.

Of course, it sounds more appealing to say, “The scientific explanation is pareidolia,” than to say, “The explanation is that I have no explanation.” But when you evoke pareidolia, you are simply saying it was a coincidence; and by saying that you are giving away that you have no explanation. Pareidolia is a non-explanation masquerading as an explanation. 

If scientists were to produce an account of how Mars might have a tendency to produce geological objects looking like long wooden spoons, that would be an actual explanation for the startling pair of spoon-like objects. But no such account seems possible, given that no desert on Earth tends to produce such objects naturally.