False Alarms Are Bad for Firemen, But Can Be Great for Scientists

Misstatements by the mainstream press about the search for chemical precursors of life in space have included the following:

• An article on the web site of Air and Space magazine with the phony title "Fingerprints of Martian Life," one merely reporting some observation of biologically irrelevant molecules. Not even the simplest molecular components of life (amino acids) have ever been discovered on Mars. 
• Another article of the major newspaper The Independent with the misleading title "Best evidence yet for alien life on Saturn's moon found by scientists." one not reporting any evidence for life because it merely reported molecules with a molecular weight of 200, which is a molecular weight 50 times smaller than a typical molecular weight of a protein molecule.
• A story on the scitechdaily.com site entitled "Astrophysicists Identify 'Significant Reservoirs' of Organic Molecules Necessary To Form the Basis of Life."  The molecules discussed were biologically irrelevant molecules found only in the tiniest trace amounts, not in "reservoirs."  
• A NASA announcement that a gigantic "reservoir" of water had been discovered in space, a statement extremely misleading because the water was scattered over so many hundreds of cubic light-years that the density of water was not even the water density in the air above the Sahara Desert. 
• A June 2023 press release from the University of Illinois claims a detection (by 30+ scientists) of polycyclic aromatic hydrocarbon (PAH) chemicals in distant space, and incorrectly claims that such chemicals are "considered the basic building blocks for the earliest forms of life."  None of the PAH chemicals are such things. 
• At the major website www.salon.com, we had a very recent article claiming a "recent study in the journal Nature revealed quite significant evidence for life on the distant moon" Enceladus. No such thing was found. The study merely reported the existence of phosphorus, one of the elements used in living things. Claiming that phosphorus is "evidence for life" is as silly as claiming that trees (something that can be converted into paper) are evidence for book-writing. The amount of purposeful work that must be done to make even the simplest living thing from mere elements such as carbon, oxygen, hydrogen and phosphorus is comparable to the amount of purposeful worked needed to make a 200-page well-written book from a tree.
• This week the hype-heavy scitechdaily.com site has a breathless headline of "Foundation of All Known Life: Webb Telescope Makes First Detection of Crucial Carbon Molecule." The molecule in question is a biologically irrelevant methyl cation molecule that is in no sense whatsoever a foundation of any kind of life. 

Also recently we had a press release from EurekAlert!, a source that  often recycles misleading or dubious press releases from various institutions and universities. The press release was entitled "An amino acid essential for life is found in interstellar space." The press release refers to a paper "A search for tryptophan in the gas of the IC 348 star cluster of the Perseus molecular cloud." In the paper the lone author of the paper (Susana Iglesias-Groth) makes no confident claim to have found tryptophan in interstellar space; and the title refers to a search, not a finding. She merely claims to have got some spectrum readings that she claims are compatible with tryptophan. Spectrum readings from very distant space are very often subject to multiple different interpretations. The Perseus molecular cloud is 1000 light-years away, and trying to use spectrum readings to detect a molecule existing only in trace amounts is a dicey business with a large chance of error. 

The Perseus molecular cloud (credit: NASA/JPL)

Interpreting spectroscopic readings can be as subjective as interpreting the ambiguous Rorschach ink blots once used by psychologists for psychological testing. Using a much closer target (Venus), a team of scientists previously announced spectroscopic readings they claimed were evidence of the gas phosphine. But later there followed four papers saying that there is no phosphine in the atmosphere of Venus. One paper by a single author states, "There is thus no significant evidence for phosphine absorption in the JCMT Venus spectra." Another paper with many co-authors is entitled, "No phosphine in the atmosphere of Venus." A third paper states there is "no statistical evidence for phosphine in the atmosphere of Venus."  Another paper says, "These findings, along with the recent papers by Encrenaz et al. (2020), Snellen et al. (2020), Lincowski et al. (2020), and Villanueva et al. (2020) undermine the reported detection of PH3 [phosphine] by Greaves et al. (2020a,b) and its possible biogenic origin."  A news account of this paper says, "The team concluded that what the scientists probably saw was just sulfur dioxide, which is a common gas around Venus and would not indicate the possible presence of life."

The paper "A search for tryptophan in the gas of the IC 348 star cluster of the Perseus molecular cloud" is now behind a paywall, but when I read it before it was paywall-blocked, I could find no plain assertion of a detection. The author merely claimed to have seen some sort of something that she claimed was compatible with tryptophan, and noted that future investigations could determine whether tryptophan really existed at the investigated location. The abstract is not behind a paywall, and that abstract makes no claim to have detected tryptophan, but merely a claim to have searched for it, and to have found something that could be tryptophan. The abstract estimates a particular sparse amount using the phrase "assuming that the detected emission lines are due to tryptophan," as if the author was not sure such an identification is correct. 

The author (Susana Iglesias-Groth) published in 2021 a preprint on this topic, "A search for mid-IR bands of amino acids in the Perseus Molecular Cloud," which can be read here. In that preprint, referring to spectral lines and using the royal "we" she stated, "We tentatively identify 14 lines for tryptophan, 12 lines for tyrosine, 7 lines for phenylalanine and 5 lines for isoleucine and glycine, respectively."  This kind of tentative identification business sounds like very iffy guesswork, rather like trying to guess what type of people are passing by your basement window when you can only see shadows cast upon a wall. The "Data" section mentions "image cleaning to remove bad pixels," and some averaging to produce a "combined spectrum," and we may wonder whether artifacts may have arisen from such debatable manipulation. Figure 4 of the preprint looks unconvincing. We see a bunch of wavy lines, looking rather like brain waves lines or lines of a graph plotting ups and downs of a Dow Jones stock; and above some of the up-blips in the line (up-blips which have varying shapes) we see "Trp," which looks like a guess that the blip was caused by tryptophan. It all sounds terribly tentative and subjective. Interstellar clouds such as the Perseus molecular cloud are hard to reliably analyze, because there is so much signal contamination from a variety of chemical sources. 

It seems rather that people have long been claiming to see interstellar tryptophan that was not actually there. A 1984 paper noted that "Hoyle, Wickramasinghe and their associates have examined interstellar (IS) absorption features in the ultraviolet, visible and infrared" and claimed to see a variety of things including  tryptophan. The paper states, "We conclude that the identifications claimed by Hoyle, Wickramasinghe and their colleagues are unwarranted."

Similarly, in the 2006 paper here we read about an apparent false alarm regarding the detection of the amino acid glycine in interstellar space:

"The early searches for glycine were all negative, but two years ago Kuan et al. (2003) reported detection of a number of glycine lines, some 27 in several astronomical sources. Unfortunately, this claim has not been confirmed. The amount of glycine claimed by Kuan et al. is in conflict with previously published upper limits (e.g. Combes et al. 1996; Ceccarelli et al. 2000), and glycine lines which should have appeared were not found. In a detailed analysis of the evidence, Snyder et al. (2005) recently concluded that few, if any, of the lines attributed by Kuan et al. to interstellar glycine were actually from that molecule. The spectroscopic data on which the claim of Kuan et al. was based have not been published or made available to other workers, and there is now a fairly wide consensus among radio astronomers and laboratory spectroscopists that glycine has not yet been found in space."

A more recent 2022 paper tells us this: "The simplest amino acid, glycine (NH2CH2COOH), has been searched for a long time in the interstellar medium, but all surveys of glycine have failed." 

We do not yet have any robust or replicated evidence for the existence of tryptophan or any other amino acid in interstellar space. The recent press release announcing a discovery of tryptophan in interstellar space seems like another case in the science press of premature overconfident boasting, and people claiming to have seen something when the evidence was all blurry and similar to a tiny hazy blip seen on the far horizon, something that could have been a hundred different things.  Very many or most of the interesting-sounding announcements you read these days in the science news are false alarms, and in fields such as neuroscience, false alarms are strongly incentivized. 

Spectrographic analysis of deep interstellar space seems to be a kind of "wild, wild West" in which there are few or no standards to minimize false alarms. In many cases it seems like subjective squinting at tiny lines, rather like the work of some palm reader looking through a magnifying glass to study tiny lines on someone's palm. In the 2021 preprint "A search for mid-IR bands of amino acids in the Perseus Molecular Cloud" by Susana Iglesias-Groth, published in 2021, which can be read here, we have no mention of statistical significance, no mention of an effect size, no mention of a p-value, no mention of blinding, and seemingly no mention of any observational standard that was met. Such subjective one-person classification guesses mean little until they are replicated by multiple independent observers. 

Consider how unreliable is Case 1 compared to Case 2 below:

Case 1: Scientist gets money to search for chemical X in deep space. Scientist examines thousands of little spectroscopic lines looking for something he can say looks like chemical X. He knows that if he doesn't report finding such a thing, he won't get his paper published, because of publication bias which favors positive results. Scientist then reports seeing some little lines that he thinks looks like chemical X. 

Case 2: Scientist X is given some spectroscopic readings provided by Scientist Y, who says, "Tell me what this looks like to you." Scientist X has no idea what answer is expected, or what Scientist Y was looking for. Scientist X identifies the reading as chemical Z, and Scientist Y, "Yes, that is what I was looking for, and what I thought it was." 

Case 1 is not compelling at all, but Case 2 is much better evidence. 

False alarms are bad for firemen, but can be great for scientists. False alarms can mean more fame for a scientist, more grant money, and more of the cherished paper citations. A scientist can pay the rent for years or decades by milking false alarms. And false alarms can be great for the person trying to show evidence for some untrue dogma that is close to his heart, or the person trying to show that he and his colleagues are making progress when they are mired in the mud, and going in the wrong direction. 

One of the ways in which scientists have milked false alarms is by making dubious reports of an association between a gene and some trait. A 2012 paper tells us that most of the reports of genes claimed to be associated with general intelligence are probably false positives. In their paper "Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature," Denes Szucs and John P. A. Ioannidis state "False report probability is likely to exceed 50% for the whole literature," and "In light of our findings, the recently reported low replication success in psychology is realistic, and worse performance may be expected for cognitive neuroscience." 

I predict with only medium confidence that the recent claim about tryptophan in interstellar space will not be well-replicated.  I also predict with medium conference that in the next few years some scientist will capture global headlines by claiming he saw "chemical signatures of life" at some other planet, by analyzing ambiguous hard-to-interpret spectroscopic data from the James Webb telescope. But it will probably be a false alarm. It will probably be another case of some scientist looking at little lines that could be caused by many different things, and making an interpretation of this hard-to-interpret reading that will allow him to say he saw what scientists have been hoping to see for so long. A Harvard astronomer (Avi Loeb) has shown that making a headline-grabbing dramatic-sounding subjective interpretation of some ambiguous marginal observational data can be very profitable. His "interstellar spacecraft" speculations about a strange object named ‘Oumuamua resulted in a lucrative book deal. 

After trying to suggest that another space oddity (the CNEOS 2014-01-08 meteor) was a crashed interstellar spacecraft, Loeb has recently finished his million-dollar oceanic expedition looking for what he hoped would be remnants of a crashed extraterrestrial spaceship (an idea that is implausible for reasons I discuss here).  The expedition seems like a bust, as I predicted it would be.  All Loeb has to report finding are tiny scarcely visible specks and tiny metal scraps. We can imagine what derisive laughter would come if some person without Loeb's following were to try to pass off such measly things as evidence of extraterrestrials. We are apparently expected to think the makers of the ET spaceship were so incompetent that they let their ship get instantly blown up upon entering Earth's atmosphere, so that only the tiniest smithereens remained. Meanwhile two other scientists recently published a paper saying that the simplest explanation for the CNEOS 2014-01-08 meteor is that it was not from some other solar system, and that its speed was simply overestimated.