White House Releases a Long Report on Scientific Integrity, One Blind to Its Main Threats

The White House Office of Science and Technology has just released a long 53-page report on the topic of protecting scientific integrity. The report is entitled "Protecting the Integrity of Government Science."  Nowadays in the United States "government science"  means almost the same as "American science," because such a large fraction of research depends on grants from the Federal government. 

The report uses the term "scientific integrity" very many times. But the authors of the report seem to be unaware of the main things undermining scientific integrity these days.  The report focuses mainly on threats to scientific activity such as government interference in scientific activity, such as when an administration or federal agency might try to pressure scientists into reporting something more agreeable to the administration or agency.  The authors of the report make little or no mention of the main things threatening scientific integrity these days. 

Below is a list of the main things threatening scientific integrity these days:

(1) There has arose in scientific journals a tendency called publication bias, a habit of not publishing negative experimental papers reporting only a null result. 

(2) There has arose in academia an incentive system and scoring system in which scientists are judged numerically by the number of scientific papers they have written and the number of citations such papers have received. 

(3) Under such a system there is an incentive for scientists to  produce low-quality papers or low-value papers in high numbers, for the sake of increasing the count of papers they have published, rather than producing papers of much higher quality in smaller numbers. 

(4) Under such a reward system there is also a strong incentive for scientists to run experiments following poor design standards and using inadequate sample sizes, because such easier-to-produce and poorly designed experiments will be more likely to create false alarms that will be reported as some positive result, thereby increasing the chance that the resulting scientific paper will be published, and not be rejected because of publication bias that excludes null results.  

(5) Under such a reward system there is also a strong incentive for scientists to not use proper blinding protocols, and to interpret and analyze experimental data in a biased way, to maximize the chance that some positive result can be reported, rather than a null result (which may cause the resulting paper not to be published because of publication bias). 

(6) Under such a reward system (in which citation counts are a key metric under which scientists are judged) there is also a strong incentive for scientists to exaggerate or misstate their experimental findings or analytic findings, claiming that they showed some important result that was not actually found by the research.  

(7) Wishing to create an aura of research success that increases their institutional prestige, universities and colleges have a strong incentive to write press releases that exaggerate or misstate the research results of scientists at their institutions, making minor or unimportant research sound like some very important result. 

(8) Not wishing to have any scandal that might decrease their institutional prestige, universities and colleges have a strong incentive to not investigate or penalize scientists at their institution who engage in fraud or poor research practices or misstatements about their research. 

(9) Wishing to create additional web traffic that results in more revenue because of online ads that generate revenue proportional to the number of page visitors, science web sites and other web sites have a strong incentive to produce hype-filled misleading pages that inaccurately summarize scientific research, making dubious or unimportant research sound like some important result.  

(10) These incentives are producing exactly the results we should expect them to produce. A large fraction of scientists are producing mainly low-quality or unimportant papers, as if they were more interested in their paper count than in the quality of their papers. In some fields such as experimental neuroscience, very poor research practices are more the norm than the exception, with inadequate sample sizes, a lack of a needed sample size calculation, and nonexistent or inadequate blinding protocols seeming to occur in the majority of experiments.  Very large numbers of scientists are making inaccurate claims in the abstracts or titles of their papers, claiming the research shows things it did not actually show. With great regularity colleges and universities are producing press releases that make inaccurate or exaggerated claims about some research result at their institution that is being announced.  Science news sites and science magazines habitually make unwarranted hype-filled claims about scientific research.  There is a huge replication crisis that has been documented by scientists.  Attempts to replicate experimental results typically show that far fewer than 50% of reported experimental results can be successfully reproduced.  Research surveys of experimental scientists (such as this one and this one) show that a large fraction of them either confess to poor research practices or suspect very many of their colleagues of such conduct.   

(11) In addition to all these problems that have nothing to do with belief traditions among scientists, there are a host of scientific integrity problems resulting from belief traditions that have arisen in scientific communities, cases in which scientists are socially pressured to support or conform to far-from-proven theories that have become popular within scientific communities.  Such theories include the dogma of abiogenesis (that the first living thing arose accidentally), the dogma of common descent (that all species evolved from a common ancestor), the dogma that memories are stored in brains (despite no one ever finding a stored memory in a brain), the dogma that all mental phenomena are caused by brains, the dogma of dark matter, the dogma of dark energy, the dogma of primordial cosmic inflation, the dogma of the nonexistence of spooky psychic phenomena, the dogma that genetically modifying food is safe, the dogma that pesticides are relatively safe (pushed by sites such as www.realclearscience.com), the dogma that gene-splicing is not risky, the doctrine that COVID-19 had a purely natural origin, and many others. When some unproven theory gains ascendancy in a scientific community, and becomes a belief tradition in that community, its members will tend to interpret all observations in a way that conforms to such a theory, rejecting all observations inconsistent with such a theory.  Dogma-driven interpretations of observational results threaten objective scientific inquiry in many ways.   

(12) Because of some of these problems,  a significant fraction of the billions of dollars that the US government spends each year on scientific research is wasted, going to unworthy projects or poorly designed experiments or poorly conducted research.  Moreover, there is a gigantic amount of misinformation being spread about federally funded projects, misinformation such as claims that experiments or projects showed things they did not show. 

The recent White House report on scientific integrity ("Protecting the Integrity of Government Science") seems to be almost entirely blind to such problems.  The authors show no real signs of understanding the problems listed above. Instead of focusing on such problems, which are the main threats these days to scientific integrity, the report focuses on things of lesser significance such as rare cases when some administration attempts to interfere with scientific analysis.  Any report such as this should have said very much about the very widely discussed problem known as the replication crisis, the fact that most scientific research does not seem to be reproducible, and that attempts to reproduce experimental results are failing most of the time. But the 53-page report does not even use the words "replicate" or "replication" or "reproduce" or "reproducible" or "reproducibility."  The report has lots of glittering generalities, but fails to deliver an effective algorithm for how to beef up scientific integrity.  The report has no mention of sample sizes or blinding protocols, two of the main things that should come up in a thorough discussion of scientific integrity. Inadequate sample sizes and a lack of thorough blinding protocols are two of the chief current threats to the integrity of experimental science.  

The report seems to have been carefully worded so that no specific mention would be made of any ongoing shortfall of today's scientists.  It is as if the authors were terrified of offending anyone in the science community or academic community or journalism community.  As a result, the report is a kind of bowl of bland bureaucratic mush, rather than the kind of sharp, incisive thing that might really help beef up scientific integrity.

What might an effective White House report on scientific integrity look like? It might be one that thoroughly documented all the problems threatening the integrity of scientific research.  It might be one that proposed specific measures to substantially reduce such problems.  The report might include a proposal for enforcing quality standards on federally funded research.  Rather than vague bland mush, the report might have included specific recommendations rather like this:

(1) No federal funding of any research project that has its published results available only behind a paywall. Taxpayers should be able to easily read all the research they paid for, except for classified research. 

(2) Some mechanism guaranteeing the publication of all federally funded observational results (including null results). 

(3) Some mechanism (such as public weekly research logs) depriving federally funded researchers from cherry-picking the experimental results that will be published, something that often occurs so that only results conforming to prevailing belief traditions appears. 

(4) No federal funding of any experimental research project unless it first published a detailed research plan precisely describing how data will be gathered and analyzed, and also followed such a plan.  This would mean no more federal funding of "fishing expedition" projects which allow researchers to slice and dice data a hundred ways until they find some result they were hoping to get, and which allow researchers to "torture the data until it makes the desired confession." 

(5) No federal funding of any experimental research project that did not declare before data was gathered a thorough blinding protocol that will be followed, and also adhered to such a protocol.

(6) No federal funding of any experimental research project that did not use at least 15 subjects for each of its study groups. 

(7) A mechanism by which federal money granted for research could be "ungranted," with a refund demanded whenever the standards above were not met. 

If such standards existed for granting federal funding, the US government could stop wasting so many billions on poorly designed experiments producing unreliable or unreproducible results. 

An example of the timidity of the report is the fact that it brings up the topic of what is the definition of scientific integrity, but timidly fails to even define such a thing. We have on page 3 of the report a shaded box labeled "Box 1.1 Defining Scientific Integrity," and we have in that box some vacuous "go around in circles" language:

 

"The 2021 Presidential Memorandum does not define the term 'scientific integrity.' Rather it reaffirms and builds on the 2009 Presidential Memorandum and 2010 OSTP Memorandum, which establish principles and guidance, respectively, for protecting scientific integrity, without explicitly defining the term. The Task Force has taken a similar approach, focusing its initial efforts on assessing agency scientific integrity policies against the principles and guidelines articulated in the memoranda and identifying practices for improving policies and their implementation as called for in the 2021 Presidential Memorandum. The Task Force notes that some, but not all, agencies provide definitions of scientific integrity in their scientific integrity policies. These definitions vary across agencies and would benefit from greater harmonization. The Task Force intends to produce a definition of scientific integrity for adoption by Federal agencies as it develops a framework for assessing scientific integrity policies. The definition will be informed by the insight gained in preparing this report."

This is classic bureaucratic "use lots of words to say nothing" talk.

Replication Study Show the Massive Rot Within Experimental Biology

A recent scientific paper gave the results of a large project designed to test how well cancer studies replicate. Entitled "Reproducibility in Cancer Biology: Challenges for assessing replicability in preclinical cancer biology," the paper is a shocking portrait of a massive degree of malfunction within the world of experimental biology. 

The authors attempted to replicate 193 experiments from 53 widely-cited cancer research papers. The authors were shocked to find that not a single one of the 193 papers gave a methods description sufficient for the authors to reproduce the experiment without asking for more information from the scientists who ran the experiment.  They state, "None of the 193 experiments were described in sufficient detail in the original paper to enable us to design protocols to repeat the experiments, so we had to seek clarifications from the original authors."

Upon asking for additional information from the scientists who ran the experiments,  the authors found that while 41% of the scientists were very helpful in providing information, 9% of the scientists were only minimally helpful, and 32% of the scientists were not helpful at all or did not respond to requests for information.  Such a result suggests that a large fraction of all cancer experiments (a quarter or more) are either junk science procedures that scientists are ashamed to discuss, or fraudulent experiments that scientists refuse to talk about any further, because of a fear of their fraud being discovered.  

Imagine if you were a scientist who had pulled some shenanigans or skulduggery when doing an experiment.  Years later, you get an email from someone saying, "I am trying to replicate your experiment, but your paper has not given me enough information -- can you please answer this list of questions?"  What would you do? In such a case you would probably just not answer the email. The last thing you would want is for someone to discover the sleazy shortcuts you had used, or to discover that you had fudged the results.  Conversely, if you did the experiment using best-practices methods, and proceeded in an entirely honest and commendable manner, you would not be troubled by such an email, and would probably answer it in a helpful way. 

The paper authors tried to reproduce the cancer studies in a way that produced a statistical power of at least .80 (which can be roughly described as a pretty good likelihood that the result was not a false alarm).  They often found that the studies they were trying to reproduce typically used too small a sample size to reach such a standard. We read this: "As an illustration, the average sample size of animal experiments in the replication protocols (average = 30; SD = 16; median = 26; IQR = 18–41) were 25% higher than the sample size of the original experiments (average = 24; SD = 14; median = 22; IQR = 16–30)."  

This is quite interesting from the standpoint of neuroscience experimental research. In neuroscience experiments, the great majority of scientific experiments use way-too-small sample sizes. Most experimental neuroscience papers use a sample size smaller than 15 for some of the study groups.  I have often cited this "15 subjects per study group" as a minimal quality standard that neuroscientists typically fail to meet in their experiments.  But according to the figures quoted above, it seems the quality shortfall is even greater than I have described. The paper suggests that sample sizes should be an average of not just 15, but 30.  If that is correct, then the failure of neuroscientists to use adequate sample sizes in their experiments is far greater than I have suggested. 

In the paragraph below the authors discusss some of the rot they have discovered within experimental biology:

"The present evidence suggests that we should be concerned. As reported in Errington et al., 2021b, replication efforts frequently produced evidence that was weaker or inconsistent with original studies. These results corroborate similar efforts by pharmaceutical companies to replicate findings in cancer biology (Begley and Ellis, 2012; Prinz et al., 2011), efforts by a non-profit biotech to replicate findings of potential drugs in a mouse model of amyotrophic lateral sclerosis (Perrin, 2014), and systematic replication efforts in other disciplines (Camerer et al., 2016; Camerer et al., 2018; Cova et al., 2018; Ebersole et al., 2016; Ebersole et al., 2019; Klein et al., 2014; Klein et al., 2018; Open Science Collaboration, 2015; Steward et al., 2012). Moreover, the evidence for self-corrective processes in the scientific literature is underwhelming: extremely few replication studies are published (Makel et al., 2012; Makel and Plucker, 2014); preclinical findings are often advanced to clinical trials before they have been verified and replicated by other laboratories (Chalmers et al., 2014; Drucker, 2016; Ramirez et al., 2017); and many papers continue to be cited even after they have been retracted (Budd et al., 1999;Lu et al., 2013; Madlock-Brown and Eichmann, 2015; Pfeifer and Snodgrass, 1990)...Fundamentally, the problem with practical barriers to assessing replicability and reproducibility is that it increases uncertainty in the credibility of scientific claims. Are we building on solid foundations? Do we know what we think we know?"

A separate paper by the same authors ("Investigating the replicability of preclinical cancer biology") gives results on what degree of success was achieved in trying to reproduce the selected experiments.  Getting little or no help from such a large fraction of the scientists, and finding the original papers failing to give enough information for replication, the authors were only able to re-run 50 of the 193 experiments they had originally chosen.  Of those 50, only 46% were successfully replicated in the sense of producing results like those reported in the original paper.  

So after setting the goal of replicating 193 experiments, and doing their best to replicate all 193 whenever possible, the authors were only able to successfully replicate about 23 of the experiments.  That's a pitiful replication rate of only about 12%. The authors report this: "One method compared effect sizes: for positive effects, the median effect size in the replications was 85% smaller than the median effect size in the original experiments, and 92% of replication effect sizes were smaller than the original." What this suggests is that the effects reported in experimental biology papers tend to be massively overstated. 

What all these numbers give us is a vivid portrait of massive decay, rot, malfunction and arrogance within experimental biology. Another scientific study surveying animal researchers (discussed here) gives similar results. Very clearly, junk experimental results are being produced to a massive degree by experimenters very often guilty of Questionable Research Practices. From the facts that such a large fraction of the experimenters refuse to respond to questions from those attempting to reproduce their experiments, and the fact that only a small fraction of the studies can be successfully replicated, we may assume that either a very large amount of fraud or a massive degree of incompetent activity is occurring within experimental biology -- probably both. Therefore, a good general principle to follow is: assume that any novel experimental biology result you read about in the science news is bogus or junk science, unless the result has been very well  replicated, with many other experimenters getting the same result.  (Vaccine results have been massively replicated, because when millions of people have taken a vaccine without harm, that is equivalent to massive replication.)

I have long discussed the poor practices and shabby standards of experimental neuroscience. When poor research practices occur in neuroscience, the damage is mainly intellectual.  Junk neuroscience experiments cause people to wrongly think that scientists are on the right track in their assumptions about minds and brains, which is not true. They are very much on the wrong track, betting the farm on false assumptions.  But at least such misleading junk science experiments don't lead to physical human suffering.  It's a different situation if so many cancer research studies are unreliable.  We can only guess how great is the physical toll to human beings when so many cancer studies are not reliable.  

Yesterday a jury found Elizabeth Holmes guilty of wire fraud. Her company Theranos had bilked investors out of countless millions, long making grand biology-related promises but producing only feeble results. There is many an Elizabeth Holmes (male and female) in the world of experimental biology.  They victimize not wealthy investors but the federal government. Every year the US government doles out billions for scientific research, and a large fraction of this goes to fund junk experimental science that cannot be replicated because poor experimental procedures were used, or because the scientists were trying to prove something that is untrue.  

You can see endless cases of wasted money by using the National Science Foundation's query tool.  Below is an example, searching for grants given on the topic of synapses:

https://www.nsf.gov/awardsearch/simpleSearchResult?queryText=synapses

Very often when you click on the rows of your search results, and very carefully analyze both the original research proposal and the resulting scientific papers published, you will find that a grant proposal was submitted promising some grand result, but that the scientific papers produced were merely junk science papers describing experiments using Questionable Research Practices such as a lack of a blinding protocol or way-too-small sample sizes.  The paper titles and the paper abstracts often claim to have found something not actually shown by the research.  

The US government has a whole big agency (the IRS) dedicated to tracking down and punishing people who file false tax returns. But the government seems to have no agency dedicated to tracking down and penalizing "sham, scam, thank you Sam" researchers who bilk Uncle Sam out of millions by getting lavish government research grants and then producing junk experimental results incapable of being successfully replicated.  We may presume that in the labs they sometimes whisper that Uncle Sam is an easy mark. 

 In an unsparing essay entitled "The Intellectual and Moral Decline in Academic Research," PhD Edward Archer states the following:

"Universities and federal funding agencies lack accountability and often ignore fraud and misconduct. There are numerous examples in which universities refused to hold their faculty accountable until elected officials intervened, and even when found guilty, faculty researchers continued to receive tens of millions of taxpayers’ dollars. Those facts are an open secret: When anonymously surveyed, over 14 percent of researchers report that their colleagues commit fraud and 72 percent report other questionable practices....Retractions, misconduct, and harassment are only part of the decline. Incompetence is another....The widespread inability of publicly funded researchers to generate valid, reproducible findings is a testament to the failure of universities to properly train scientists and instill intellectual and methodologic rigor. That failure means taxpayers are being misled by results that are non-reproducible or demonstrably false."

Researcher Survey Reveals the Sick State of Animal Cognition Research

Claims that neuroscientists make about brains, minds and memory are mainly based on experiments with animals. This is largely because of the moral restrictions against doing brain experiments on human subjects.  With a rat you can do something like open up its brain and scan some of its cells to look for signs of brain changes caused by learning, or you can do something like remove part of the rat's brain to see whether that affects learning. But you can't do those kind of experiments with a human without engaging in behavior that would be widely condemned. 

You do not need any survey of animal cognition researchers to know that the field of animal cognition research is in a sick state.  You can do that by making a critical analysis of the papers published by such researchers. You will find a very large prevalance of Questionable Research Practices and other serious problems.  Among the problems are these:

(1) Researchers routinely do experiments with insufficient sample sizes, very often using study group sizes smaller than 15 subjects.  !5 subjects per study group is the minimum needed for a moderately reliable result, and any study using fewer than 15 subjects in any of its study groups will have a very high chance of producing a false alarm. 

(2) Researchers routinely fail to show evidence that their experiments followed a blinding protocol designed to reduce experimenter bias under which a researcher will "see whatever he wants to see." Usually no mention will be made of of any blinding procedure, and the word "blind" or "blinding" will not even appear in the paper describing the research.  In the minority of cases in which some mention is made of blinding, the mention will be some half-hearted mention of some fragmentary effort at a blinding protocol. Almost never will we read a discussion of how a detailed plan was made to implement a thorough blinding protocol, and a discussion of how such a plan was followed. 

(3) The overwhelming majority of animal cognition research papers will fail to show any evidence that a detailed hypothesis was chosen before an experiment began, and that the scientific paper reported on whether the pre-selected hypothesis succeeded or failed. The overwhelming majority of such papers will not be pre-registered papers in which the authors chose (before gather data) a hypothesis to be tested, how data would be gathered, and how data would be analyzed. Instead the great majority of papers will give the impression of using "fishing expedition" techniques in which data is gathered, and then the researchers were free to "slice and dice" the data in innumerable ways, trying different data analysis methods until there turned up what could be called some marginal evidence for any hypothesis that the researchers might have dreamed up after gathering data. 

(4) In some cases bad experimental methods have become a tradition. For example, the reigning tradition among animal cognition researchers is to try to measure fear in rodents by making estimates of "freezing behavior."  Such a subjective and unreliable method of judging fear is far less reliable than measuring heart rate, which reliably undergoes a very sharp spike when rodents are afraid. 

(5) Researchers very often make claims in the title or abstracts of their papers that are not justified by any research described in their papers.  They sometimes confess to doing this. At a blog entitled "Survival Blog for Scientists" and subtitled "How to Become a Leading Scientist," a blog that tells us  "contributors are scientists in various stages of their career," we have an explanation of why so many science papers have inaccurate titles:

"Scientists need citations for their papers....If the content of your paper is a dull, solid investigation and your title announces this heavy reading, it is clear you will not reach your citation target, as your department head will tell you in your evaluation interview. So to survive – and to impress editors and reviewers of high-impact journals,  you will have to hype up your title. And embellish your abstract. And perhaps deliberately confuse the reader about the content."

The European Journal of Neuroscience published an editorial entitled "Getting published: how to write a successful neuroscience paper." The editorial emphasized that the title and abstract of a neuroscientist paper need to be "enticing," and suggested the use of the active voice, using the example of saying that neurons signal something about memory. We can only guess at how many neuroscience papers have been given misleading and inaccurate titles because neuroscientists are being advised to use "enticing" titles for their papers, and urged to use the active voice in referring to mindless and passive chemicals and cells. 

Recently there was published a paper which gives us a different way of detecting the sick state of animal cognition research. Entitled "The hidden side of animal cognition research: Scientists’ attitudes toward bias, replicability and scientific practice," the paper was a survey of scientists doing animal cognition research. 210 researchers filled out the survey.  Collectively their answers are an indictment of the dysfunctional state of animal cognition research. 

When asked about bias in their experiments, nearly 80% of researchers confessed that they found themselves often or sometimes hoping for some particular result in their study.  When we have such a level of bias and a failure of most animal cognition research papers to follow a blinding protocol to reduce bias, we have basically a sure-fire recipe for unreliable results caused by unmitigated experimenter bias.  When asked whether the results and theories in their area of animal cognition research are strongly affected by the biases of researchers, more of the experimenters agreed than disagreed.  When asked whether the results and theories in other areas of animal cognition research are strongly affected by the biases of researchers, far more of the experimenters agreed than disagreed, with 46% agreeing, and only 15% disagreeing. 

When asked about overstating claims in papers (making claims in a paper that are not justified by the research), 7.7% of researchers confessed to making stronger claims than warranted.  Remembering the fact that only a tiny fraction of people confess to wrongdoing they have done, we should assume that the actual number of animal cognition researchers making stronger claims than warranted is many times higher than this 7.7%.  Indeed, when asked about the practices of other researchers in animal cognition,  56% of the respondents said that other researchers in animal cognition made stronger claims than warranted.   

When asked about what percent of their experimental studies have been published or will be published, the average response was 80%. This suggests there is a very large problem that studies producing null results will tend not to be published.  So, for example, if a researcher produces a result that does not support prevailing neuroscientist dogmas about brains storing memories, such a study will simply end up in a file drawer without being published. 

Researchers in animal cognition confessed to a rather low confidence in the statistical analysis in their papers.  When asked whether they "somewhat agree" or "strongly agree" that the statistical analysis in their own papers is valid, 42.9% said that they merely "somewhat agree" rather than "strongly agree."  Since this is a self-confession question and since we would expect that only a small fraction of the researchers who doubt their own statistical analysis would confess to having doubt, we may assume that it is actually the great majority of animal cognition researchers who lack strong confidence in their statistical analysis.  There are strong reasons for suspecting that extremely dubious statistical analysis is more the rule rather than the exception in animal cognition research. One respondent stated, "The majority of animal cognition researchers have a very sparse statistical education," and that this can be "a huge potential for errors."

Why is there so much arcane statistical analysis in neuroscience papers? It's because typically experimenters get results providing no good evidence for the incorrect dogmas they are trying to support. Then our researchers very often decide to keep playing around with statistical analysis until the data seems to provide some faint whisper sounding a bit like the desired effect.  There's an old expression: if you torture the data sufficiently, it will confess to anything. 

When asked how often Questionable Research Practices occur in their own research,  27% of animal cognition researchers confessed that such QRP practices occur "sometimes," "often" or "always."  Since this is a self-confession question in which we would expect that only a small fraction of the researchers engaging in Questionable Research Practices would confess to doing so, we should assume that the actual percentage of such researchers engaging in Questionable Research Practices is far higher than 50% (an assumption very much warranted from reading the papers of such researchers).  When asked about how often Questionable Research Practices occur in the research of other animal cognition researchers, 52.7% of the respondents said that such practices occur "sometimes," and 28.9% of the respondents said that such practices occur "often" or "always." 

When asked about replication, 73% of the respondents agreed that some areas of animal cognition research would experience a replication crisis if attempts to replicate most of its studies were attempted.  This amounts to a confession of large-scale unreliability in animal cognition research. 

How would things work if experimental science was being done properly? For one thing, there would be some central repository in which scientists could (without any restriction) do things such as (1) publish a research protocol for an upcoming experiment, which would amount to a public promise to test one particular hypothesis, and to gather and analyze data in a particular way, and (2) concisely report the results (null or not) of particular experiments, regardless of whether any paper on such a result got published.  But the animal cognition survey research paper states that no such repository even exists. We read this:

"While there is currently no central repository or systematic method for study registration (c.f. https://clinicaltrials.gov/ for medical trials), research groups could seek to publicly archive all studies they conduct, which would allow other researchers to assess the strength of evidence not just from individual studies, but in relation to the entire research programme they have come from."

So our science community does not even have one of the most basic tools it should have to do research in a competent way. How lame is that? 

What the very illuminating "hidden side of animal cognition research" study reveals is that animal cognition research is in a sick and dysfunctional state.  We should remember this every single time a scientist claims that memories are stored in brains and that brains produce minds.  Such claims are mostly based on animal cognition research, which is a diseased and dysfunctional branch of research that is a not a reliable pillar for anything.  There is no robust evidence for any of the main claims of neuroscientists.  We have no good evidence from either animals or humans that brains store memories or that brains are the source of human mental effects such as self-awareness,  thinking, instantaneous memory recall, imagination, understanding and self-hood. To the contrary, low-level research on brains very frequently provides us with strong reasons for rejecting all such claims, revealing the brain and its synapses as too slow, unreliable, noisy and unstable to be the source of the human mind and its abilities. 

Replication Crisis Is Only Part of the Scientific Academia Dysfunction

It is widely recognized among scientists that there is a problem called the “replication crisis.” This is the problem that a large fraction of research studies cannot be replicated. The problem was highlighted in a widely cited 2005 paper by John Ioannidis entitled, “Why Most Published Research Findings Are False.” A scientist named C. Glenn Begley and his colleagues tried to reproduce 53 published studies called “ground-breaking.” He asked the scientists who wrote the papers to help, by providing the exact materials to publish the results. Begley and his colleagues were only able to reproduce 6 of the 53 experiments. In 2011 Bayer reported similar results. They tried to reproduce 67 medical studies, and were only able to reproduce them 25 percent of the time.

The replication crisis is real, but it is only part of the malfunction in scientific academia. There are other very serious problems. Here are some of them:

• There is a great deal of overconfidence and hubris among many scientists, who often claim to know things they do not at all know.
• Speaking in triumphal tones, many scientists claim that they, their colleagues, or their predecessors have accomplished things that were not actually accomplished.
• Many scientists inaccurately describe as “science” or "fact" truth claims or speculations that have not been proven by experiments or observations, claims that are merely philosophical ideas or simply dogmas, stories or speculations that became popular among scientists.

In the past and future posts of this blog, you will find discussions of many examples of such things. But for now, let's look at some recent examples.

One example is a recent headline from the web site of the publication New Scientist. The headline is: “Kepler finds 219 new exoplanets and 10 are rocky and Earth-like.” But the story doesn't actually discuss the discovery of Earth-like planets; it merely discusses the discovery of Earth-sized planets. The discovery of an Earth-like planet would be the discovery of a planet with life. No such thing has taken place.

Another recent example was a FermiLab press release claiming that a new “dark matter map” had been created. But (as discussed here) the map in question was not actually a dark matter map, but a map of mass (something that might be any combination of dark matter and ordinary matter). The technique used to create the map was gravitational lensing, an effect produced by any type of matter, whether dark matter or regular matter. Claiming to have created a map of dark matter based on gravitational lensing is like using an infrared sensor to make a map of human body heat signatures in New York City, and then claiming that you have created a map of Chinese people in New York City. Of course, such a technique cannot distinguish between Chinese and non-Chinese people. By announcing a “dark matter map,” FermiLab was guilty of creating a completely false impression that dark matter had been directly observed.

Another recent example of scientists claiming to have accomplished things they have not accomplished is to be found in this press release from the Australian National University, which was headlined, “ANU-led study solves mystery of how first animals appeared on Earth.” The press release was picked up by science-reporting sites such as ScienceDaily.com, which had an article with the title, “Mystery of how first animals appeared on Earth solved.”

A person hearing such a claim may think immediately of the mystery of what is called the Cambrian Explosion. When we examine the fossil record, we don't see fossils appearing in larger and larger sizes, at an even rate of progression between 3 billion years ago and 100 million years ago. Instead, we see very little fossil evidence of life prior to the Cambrian era about 540 million years ago. But during the Cambrian era there is a sudden surge of fossils in the fossil record. This sudden blossoming of life during the Cambrian era is known as the Cambrian Explosion.

The largest classification category used for living things is the phylum. Astonishingly, every major phylum of animal dates from the time of the Cambrian era about 540 million years ago, or shortly before. Referring to the Cambrian Era ending about 485 million years ago, a scientific web site says, “By the end of the period, every major animal phylum was firmly established, and life after the Cambrian was radically different from what had gone before.”  

This situation is a severe problem for orthodox Darwinism. From Darwinist assumptions, we would expect that the animal phyla would have gradually appeared over the past billion years, with the number of phyla slowly increasing as time passed. But the fossil record shows no such thing. Instead there was a kind of a biological “Big Bang” in which all the major animal phyla appeared rather suddenly. Explaining this problem has been a long-standing problem.

What explanation do these Australian National University scientists offer? Below are some excerpts from the press release:

"We crushed these rocks to powder and extracted molecules of ancient organisms from them," said Dr Brocks from the ANU Research School of Earth Sciences. "These molecules tell us that it really became interesting 650 million years ago. It was a revolution of ecosystems, it was the rise of algae." Dr Brocks said the rise of algae triggered one of the most profound ecological revolutions in Earth's history, without which humans and other animals would not exist....

Dr Brocks said the extremely high levels of nutrients in the ocean, and cooling of global temperatures to more hospitable levels, created the perfect conditions for the rapid spread of algae. It was the transition from oceans being dominated by bacteria to a world inhabited by more complex life, he said. "These large and nutritious organisms at the base of the food web provided the burst of energy required for the evolution of complex ecosystems, where increasingly large and complex animals, including humans, could thrive on Earth," Dr Brocks said.

So this is Dr. Brocks' explanation: there suddenly appeared all of the major animal phyla on planet Earth, the first large animals, because there was some algae available for eating. This is, of course, not an actual explanation. It's like trying to explain 40 different types of monsters rising up out of the ground in your backyard by saying that you were having a barbecue, and monsters like barbecued food.

Philosophers distinguish between two types of conditions: necessary conditions and sufficient conditions. A necessary condition for something is a condition that is necessary for that thing to occur or appear, but which does not guarantee that such a thing will occur or appear. A sufficient condition for a thing is a condition which, if satisfied, guarantees that such a thing will occur. You do not explain a thing by merely mentioning a necessary condition of that thing. For example, you would not explain the not-observed appearance of a snowman in your back yard, by pointing out that it's very cold, and snowmen only appear when it is very cold.

The existence of algae in the ocean might be a necessary condition for the existence of large complex animals, because that algae might be at the bottom of a food chain used by the animals. But the existence of algae in no sense explains the appearance of large complex animals that the Earth has never seen before. The existence of algae is at best a necessary condition for the Cambrian Explosion, and is not a sufficient condition. 

The Australian National University press release is therefore guilty of a preposterous misstatement by announcing that this is research that has solved the mystery of how the first animals appeared. It is no sense correct that Dr. Brocks and his colleagues have solved the mystery of how the first animals appeared on Earth, nor have they contributed even 1 per cent towards solving such a mystery. Most people already presumed long ago that algae existed before the first animals, and the previous existence of algae does not explain the existence of such animals, which are many times more complex than algae. The mystery of the Cambrian Explosion remains unsolved. A Cambridge University professor, responding to Brock's claims, says he has gotten things backwards, and that the explosion of algae did not drive the rise of animals.

The example we see in this case is not very uncommon. It is sadly true that when reading today's science magazines, one has to be very careful to separate the gold from the dross, the fact from the tribal folklore. Scientists have many great accomplishments to be proud of, so why do many of them seem to be claiming to know things they don't, or claiming that they or their predecessors accomplished things they didn't? It's like some person with $200 million in the bank telling people he's a billionaire. 

 Science mags are grab-bags mixing the sound and the shaky 

Postscript: A 2015 study found a huge surge in the use of pretentious words in biomedical scientific papers, suggesting that hype is becoming ever-more-common.  The study noted this:

The absolute frequency of positive words increased from 2.0% (1974-80) to 17.5% (2014), a relative increase of 880% over four decades. All 25 individual positive words contributed to the increase, particularly the words “robust,” “novel,” “innovative,” and “unprecedented,” which increased in relative frequency up to 15,000%.

This is only in scientific papers themselves; there's a whole other layer of hype going on in university press releases and the press coverage of scientific activities.  Call it the "hype crisis."   

Biomedical Blunders Blow Billions

In the new book Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hopes, and Wastes Billions by Richard Harris, we are told this: “Misleading animal studies have led to billions of dollars worth of wasted effort and dead ends in the search for drugs.” There is a rather acidic visual on the front cover of the Rigor Mortis book. We see a toe tag tied around the letter “I” in the title, like the toe tag they tie around the toes of corpses. In the “Name” slot of the toe tag, we see: “Biomedical Research.”

That visual is an exaggeration, since biomedical research isn't dead. But judging from the book, there are serious problems in the field. It seems that a very large fraction of research studies cannot be replicated. The problem was highlighted in a widely cited 2005 paper by John Ioannidis entitled, “Why Most Published Research Studies Are False.” A scientist named C. Glenn Begley and his colleagues tried to reproduce 53 published studies called “ground-breaking.” He asked the scientists who wrote the papers to help, by providing the exact materials to publish the results. Begley and his colleagues were only able to reproduce 6 of the 53 experiments.

In 2011 Bayer reported similar results. They tried to reproduce 67 medical studies, and were only able to reproduce them 25 percent of the time. On page 14 of the book by Harris, we are told that one expert estimates that 28 billion dollars a year is spent on untrustworthy papers.

Part of the problem is a culture that provides high rewards for splashy results that can be called “ground-breaking,” but which makes it rather hard for a biologist to get a paper published if the paper reports a failure to replicate a previous study. Another part of the problem is insufficient attention to methodology and precise mathematics. One expert quoted on page 172 says that in the current culture of biomedical research, it “pays to be first” but “it doesn't necessarily pay to be right.” The expert laments, “It actually pays to be sloppy and just cut corners and get there first,” noting that this is “really wrong.”

On page 96 we learn about a problem with misidentified cell lines, in which experiments are done assuming some series of cells are from one type of organism when they are actually from some other type of organism. We read:

A 2007 study estimated that between 18 and 36 percent of all cell experiments use misidentified cell lines. That adds up to tens of thousands of studies, costing billions of dollars....Sometimes, even the species isn't correct. Nelson-Rees found a “mongoose” cell line was actually human and determined that two “hamster” cell lines were from marmosets and humans, respectively. “Have the Marx Brothers taken over the cell-culture labs?” Roland Nardone asked in a 2008 paper bemoaning this state of affairs.

On page 203 of the Harris book, an expert laments that “we haven't trained a lot of our biologists to think mathematically or to understand or analyze data.” On the same page we are told that there are no standards in whole genome sequencing, that there are no standards in searching for mutations in genomes, and that in searching for mutations in genomes, “Nobody does it the same way.”

A Pro Publica article is entitled, “When Evidence Says No, But Doctors Say Yes.” Apparently there is a problem of some doctors recommending procedures that aren't backed up by evidence. Below is a quote from the article:

In a 2013 study, a dozen doctors from around the country examined all 363 articles published in The New England Journal of Medicine over a decade — 2001 through 2010 — that tested a current clinical practice, from the use of antibiotics to treat people with persistent Lyme disease symptoms (didn’t help) to the use of specialized sponges for preventing infections in patients having colorectal surgery (caused more infections). Their results, published in the Mayo Clinic Proceedings, found 146 studies that proved or strongly suggested that a current standard practice either had no benefit at all or was inferior to the practice it replaced; 138 articles supported the efficacy of an existing practice, and the remaining 79 were deemed inconclusive.

Another huge problem in contemporary medical practice involves doctors who invest in fantastically expensive equipment, and who then give advice that may be biased by their desire to pay off the cost of such a machine (or profit from its use).

It seems that there is a huge amount of unnecessary medical treatment being done. A New Yorker article by a doctor states the following:

In just a single year, the researchers reported, twenty-five to forty-two per cent of Medicare patients received at least one of the twenty-six useless tests and treatments....The Institute of Medicine issued a report stating that waste accounted for thirty per cent of health-care spending, or some seven hundred and fifty billion dollars a year, which was more than our nation’s entire budget for K-12 education....Millions of people are receiving drugs that aren’t helping them, operations that aren’t going to make them better, and scans and tests that do nothing beneficial for them, and often cause harm.

If you are asked to take some expensive test or undergo some expensive medical procedure, the following are good questions to ask your doctor:

(1) Is the course of treatment or testing you are recommending considered a standard practice or "best practice" for patients with my set of circumstances? 

(2) If you were teaching a room full of medical students, would you recommend this exact treatment or testing for someone with my set of circumstances?

Look for a firm, confident answer of "Yes," rather than a weaker answer such as "Doctors often do this."

During earlier times, people had complete faith in words spoken by anyone wearing the black outfit of the priest. Today we have somehow been socially conditioned to regard anyone with a white coat as a totally reliable source of information. Perhaps both forms of “color confidence” involved too much uncritical trust.

Postscript:  An article in the Guardian states the following:

More than 70% of the researchers (pdf), who took part in a recent study published in Nature have tried and failed to replicate another scientist’s experiment. Another study found that at least 50% of life science research cannot be replicated. 

A Nature article states this: "Although 52% of those surveyed agree that there is a significant ‘crisis’ of reproducibility, less than 31% think that failure to reproduce published results means that the result is probably wrong, and most say that they still trust the published literature." This smells like scientists having too much overconfident faith in their fellow scientists.