Exhibit B Suggesting Scientists Don't Know How a Brain Could Retrieve a Memory

In a 2019 post “Exhibit A Suggesting Scientists Don't Know How a Brain Could Retrieve a Memory,” I took a close look at 68 “expert answers” given on one page of an “expert answers” site, a page with the topic of "how are memories retrieved in the brain?" I argued  that none of the experts had a coherent and convincing answer to the question “how are memories retrieved in the brain?” I maintain that answering such a question convincingly will always be impossible, because human memories are not stored in brains, and nothing in the human brain bears any resemblance to either  a device for retrieving factual information learned during human experience or a device for storing memories for years. In particular, there is not any thing in the human brain that can explain how a human brain can instantly retrieve detailed information learned long ago about about some obscure person, place or event. Since the brain lacks any addressing system, any indexing system, and any position notation system, it should be absolutely impossible for a brain to instantly recall obscure information, such as we see happening on the long-running television quiz show Jeopardy. For example, if someone asks you (for the first time ever in your life) to name three Russian composers, and you instantly answer “Tchaikovsky, Borodin, and Rimsky-Korsakov,” you are doing something absolutely inexplicable in terms of brain activity.

Now I will give a kind of “Exhibit B” suggesting that scientists don't know how a brain could retrieve a memory: a 2019 paper entitled “The neurobiological foundation of memory retrieval.” When we get beyond the hype and unwarranted braggadocio of this paper, we find that it fails to convincingly portray any such foundation at all.

A great deal of the paper is involved with trying to persuade us that experimental studies have made great progress in identifying memory storage sites (called engrams). The authors state, “In the last decade, enormous progress has been made in identifying and manipulating engrams in rodents.” This statement is not at all correct. A few scattered studies have claimed to identify and manipulate such alleged engrams, but such studies have failed to provide any convincing evidence that such engrams really exist. The studies typically suffer from several of the following methodological sins:

Sin #1: assuming or acting as if a memory is stored in some exact speck-sized spot of a brain without any adequate basis for such a “shot in the dark” assumption.

Sin #2: either a lack of a blinding protocol, or no detailed discussion of how an effective technique for blinding was achieved.

Sin #3: inadequate sample sizes, and a failure to do a sample size calculation to determine how large a sample size to test with.

Sin #4: a high occurrence of low statistical significance near the minimum of .05, along with a frequent hiding of such unimpressive results, burying them outside of the main text of a paper rather than placing them in the abstract of the paper.

Sin #5: using presumptuous or loaded language in the paper, such as referring in the paper to the non-movement of an animal as “freezing” and referring to some supposedly "preferentially activated" cell as an "engram cell."

Sin #6: failing to mention or test alternate explanations for the non-movement of an animal (called “freezing”), explanations that have nothing to do with memory recall.

Sin #7: a dependency on arbitrarily analyzed brain scans or an uncorroborated judgment of "freezing behavior" which is not a reliable way of measuring fear.

I fully discuss all of these methodological problems in my post “The Seven Sins of Memory Engram Experiments,” and I give very many examples of how the papers cited as evidence for engrams in rodents are guilty of such procedural sins. So when the authors of the paper “The neurobiological foundation of memory retrieval” assert that "enormous progress has been made in identifying and manipulating engrams in rodents,” they do not speak correctly at all. There still exists no robust well-replicated evidence that any such thing as an engram (a neural site of stored learned information) exists in any animal. 

The authors present a lengthy, credulous and uncritical review of weak neuroscience studies that have attempted to find evidence for memory engrams (neural storage sites for memories). Their review repeatedly fails to subject such studies to an appropriate level of scrutiny. The authors  trumpet weak and poorly replicated studies as evidence for the memory engrams that they  want to believe in. We hear no mention of the very many problems in such studies, such as the fact that they typically use unreliable bias-prone techniques for judging the degree of fear in rodents (subjective judgments about "freezing behavior") rather than reliable objective techniques such as heart-rate measurement (the heart rate of a rat dramatically surges when the rat is afraid). 

In the section entitled “Retrieval as neuronal reinstatement,” we have the main part of the authors' ideas about how memory retrieval might work in a brain. Get beyond the dense layers of jargon, digressions and circumlocutions, and we find very little of substance. Their basic idea is that “natural retrieval cues reactivate neural ensembles active at encoding.” “Encoding” is a jargon term used by neuroscientists to describe some process that allegedly occurs when learned information is translated into neural states or synapse states. Despite the fact that the term “encoding” has been constantly used in scientific papers, we have neither any good evidence that such encoding occurs (in the sense of knowledge being translated into neural or synapse states), nor any coherent theory as to how it possibly could occur (there being an ocean of difficulties in the idea that human experience or conceptual knowledge could ever be translated into neural states). We merely have evidence that human beings remember things.

Neuroscientists so often use the term “encoding” that one way to interpret the word is to simply use it as a synonym for learning or memory acquisition. So using that interpretation, we can regard “natural retrieval cues reactivate neural ensembles active at encoding” as simply meaning “when you recall something, your brain reactivates some part of the brain that you used in learning the thing or experiencing the thing recalled.”

When we consider how a brain works, and the fact that all parts of it are constantly active, we can realize that such an explanation for memory retrieval is vacuous or untenable. All neurons in the human brain are constantly firing. Each neuron fires multiple times per minute. So we cannot at all explain a memory recollection as being a case where some tiny part of the brain was “activated,” as if that tiny part was the only part active. All neurons are constantly active.

Brain scanning studies contradict the claim that some little part of the brain (where some memory might be stored) is activated to a higher degree during memory recall.  Excluding the visual cortex that may be to used to kind of visually enhance some memory that was retrieved, such studies show that when humans recall things, there is no brain area that has even a 1% greater activation than any other brain area. Here are some specific numbers from particular studies:

• This brain scan study was entitled “Working Memory Retrieval: Contributions of the Left Prefrontal Cortex, the Left Posterior Parietal Cortex, and the Hippocampus.” Figure 4 and Figure 5 of the study shows that none of the memory retrievals produced more than a .3 percent signal change, so they all involved signal changes of less than 1 part in 333.
• In this study, brain scans were done during recognition activities, looking for signs of increased brain activity in the hippocampus, a region of the brain often described as some center of brain memory involvement. But the percent signal change is never more than .2 percent, that is, never more than 1 part in 500.
• The paper here is entitled, “Functional-anatomic correlates of remembering and knowing.” It shows a graph showing a percent signal change in the brain during memory retrieval that is no greater than .3 percent, less than 1 part in 300.
• The paper here is entitled “The neural correlates of specific versus general autobiographical memory construction and elaboration.” It shows various graphs showing a percent signal change in the brain during memory retrieval that is no greater than .07 percent, less than 1 part in 1000.
• The paper here is entitled “Neural correlates of true memory, false memory, and deception." It shows various graphs showing a percent signal change during memory retrieval that is no greater than .4 percent, 1 part in 250.
• This paper did a review of 12 other brain scanning studies pertaining to the neural correlates of recollection. Figure 3 of the paper shows an average signal change for different parts of the brain of only about .4 percent, 1 part in 250.
• This paper was entitled “Neural correlates of emotional memories: a review of evidence from brain imaging studies.” We learn from Figure 2 that none of the percent signal changes were greater than .4 percent,  1 part in 250.
• This study was entitled “Sex Differences in the Neural Correlates of Specific and General Autobiographical Memory.” Figure 2 shows that none of the differences in brain activity (for men or women) involved a percent signal change of more than .3 percent or 1 part in 333.

So it simply is not true that when you recall something, there is some substantially greater activation of some region of your brain where the memory is stored.  The claim that "natural retrieval cues reactivate neural ensembles active at encoding" basically means merely "your brain uses the information that it stored somewhere," but such an idea doesn't explain how a human brain supposedly storing very many thousands or millions of learned items of information could ever instantly find just the right neurons to use to cause you to instantly recall just the right piece of information when you are asked a specific question such as "What jobs did Ulysses Grant have?" 

There are many seemingly insurmountable problems that would have to be tackled by any theory of neural memory retrieval. The first is what I call the navigation problem. This is the problem that if a memory were to be stored on some exact tiny spot on the brain, it would seem that there would be no way for a brain to instantly find just that little spot. For that to occur would be like someone instantly finding a needle in a mountain-sized haystack, or like someone instantly finding just the right book in a vast library in which books were shelved in random positions. Neurons are not addressable, and have no neuron numbers or neuron addresses. So, for example, we cannot imagine that the brain instantly finds your memory image of Marilyn Monroe (when you hear her name) because the brain knows that such information is stored at neural location #239355235.  There are no such "neural addresses" in the brain. 

Then there is also the fact that the brain seems to have nothing like a read mechanism by which some small group of neurons are given special attention. The hard disk of a computer has a read/write head, but there's nothing like that in the brain. 

Then there is the fact that if memory information were encoded into neural states, the brain would have to decode that encoded information; but such a decoding would seem to require time that would prevent instantaneous recall. When cells do vastly simpler decoding involved in decoding DNA information, it takes cells many seconds or minutes. We would expect that any decoding of encoded information stored in a brain would take many seconds or minutes, preventing any such thing as instantaneous recall of rarely-remembered data items. In addition, we have not the slightest idea of how human learned information (with so many diverse forms) could either be translated or encoded into neural states, or decoded back into thoughts once such translated or encoded knowledge was decoded.  There exist hundreds of genes for the relatively simple job of decoding the genetic information in DNA. If human learned information and experiences (with so many diverse forms) were to be translated into neural or synapse states, so that learned information could be stored in a brain, there would need to be many hundreds or thousands of genes and proteins devoted to so complex a task. But no such genes and proteins seem to exist, and no one has proven that any gene or protein is dedicated to the task of memory encoding or decoding. 

None of these problems are addressed by the paper "The neurobiological foundation of memory retrieval."  The authors simply ignore the whole speed problem of explaining instant memory recall.  Their paper makes no mention of such a thing, and doesn't use words such as "speed" or "quick" or "fast" or "instant" or "instantaneous."  The authors also ignore the issue of how a brain could decode (during memory retrieval) encoded information stored in a brain. Their paper does not use the words "decode," "decoding" or "translate."  The paper merely refers in passing to some research they claim has "potentially interesting translational implications," but give no details to clarify such a claim.  Nor does the paper have any discussion of some theory of a read mechanism that could be used to read memories from brains. Searching for the word "read" in the paper produces no relevant sentences. 

Any real theory of a neural retrieval of memories would have to also be a theory of the storage and encoding of such memories. There can be no understanding of how some memories could be read from neurons or synapses or decoded unless you had an understanding of how such memories were stored and encoded in neurons or synapses.  But the paper "The neurobiological foundation of memory retrieval" gives no theory of how a brain could store learned information. The paper does make quite a few uses of the word "encoding," but simply uses that as a synonym for "learning" or "memory acquisition" without doing anything to explain how learned information could be translated into neural states. 

So the paper claiming to elucidate a "neurobiological foundation of memory retrieval" fails to discuss in any substantive way any of the main things that would need to be explained by an actual theory or understanding of how a brain could retrieve a memory: (1) how a brain could instantly find just the right tiny engram where a memory was stored in it; (2) how a brain could read information stored in it; (3) how a brain could perform the miracle of instantly decoding such learned information that had been encoded in neural states or synapse states, acting 1000 times faster than cells do when they decode DNA information; (4) what miracle of translation would have allowed information so diverse to ever have been encoded as neural states or synapse states in the first place. The paper is additional evidence that our scientists have no actual understanding of how a brain could instantly retrieve a memory. There does not exist any such thing as a "neurobiological foundation of memory retrieval." Humans and animals remember things, but neither scanning their brains during memory activity nor rat experiments provide any insight as to how instantaneous recall of specific learned items (or any recall at all of such items) can occur. 

The lack of any real understanding on this matter is almost admitted by the paper in question, which states at its end, "Our understanding of the neurobiological underpinnings of retrieval remains rudimentary." That is not how it would be in the year 2020 (70 years after the discovery of DNA) if human brains actually performed memory retrieval.  In a brain that stored and retrieved memories, there would have been signs of its memory storage and retrieval mechanism discoverable around 1950; and around the same time we discovered the readable microscopic encoded information in DNA, around 1950, we would have discovered readable encoded memory information in brains (something which still has not been found).  Instead of finding any evidence for proteins dedicated to encoding memories,  which would have to exist in massive numbers if a brain stored memories, what was found was that the proteins in synapses (the alleged storage place of memories) have lifetimes 1000 times shorter than the maximum age of human memories. 

The Seven Sins of “Memory Engram” Experiments

There are some very good reasons for thinking that long-term memories cannot be stored in brains, which include:

• the impossibility of credibly explaining how the instantaneous recall of some obscure and rarely accessed piece of information could occur as a neural effect, in a brain that is without any indexing system and subject to a variety of severe signal slowing effects;
  
• the impossibility of explaining how reliable accurate recall could occur in a brain subject to many types of severe noise effects;
• the short lifetimes of proteins in synapses, the place where scientists most often claim our memories are stored;
• the lack of any credible theory explaining how memories could be translated into neural states;
• the complete failure to ever find any brain cells containing any encoded information in neurons or synapses other than the genetic information in DNA;
• the lack of any known read or write mechanism in a brain.

But scientists occasionally produce research papers trying to persuade us that memories are stored in a brain, in cells that are called "engram cells." In this post, I will discuss why such papers are not good examples of experimental science, and do not provide any real evidence that a memory was stored in a brain. I will discuss seven problems that we often see in such science papers. The "sins" I refer to are merely methodological sins rather than moral sins. 

Sin #1: assuming or acting as if a memory is stored in some exact speck-sized spot of a brain without any adequate basis for such a “shot in the dark” assumption.

Scientists never have a good basis for believing that a particular memory is stored in some exact tiny spot of the brain. But a memory experiment will often involve some assumption that a memory is stored in one exact spot of the brain (such as some exact spot of a cubic millimeter in width). For example, an experimental study may reach some conclusion (based on inadequate evidence) about a memory being stored in some exact tiny spot of the brain, and then attempt to reactivate that memory by electrically or optogenetically stimulating that exact tiny spot.

The type of reasoning that is used to justify such a “shot in the dark” assumption is invariably dubious. For example, an experiment may observe parts of a brain of an animal that is acquiring some memory, and look for some area that is “preferentially activated.” But such a technique is as unreliable as reading tea leaves. When brains are examined during learning activities, brain regions (outside of the visual cortex) do not actually show more than a half of 1% signal variation. There is never any strong signal allowing anyone to be able to say with even a 25% likelihood that some exact tiny part of the brain is where a memory is stored. If a scientist picks some tiny spot of the brain based on “preferential activation” criteria, it is very likely that he has not picked the correct location of a memory, even under the assumption that memories are stored in brains. Series of brains scans do not show that some particular tiny spot of the brain tends to repeatedly activate to a greater degree when some particular memory is recalled. 

Sin #2: Either a lack of a blinding protocol, or no detailed discussion of how an effective technique for blinding was achieved.

Randomization and blinding techniques are a very important scientific technique for avoiding experimenter bias. For example, what is called the “gold standard” in experimental drug studies is a type of study called a double-blind, randomized experiment. In such a study, both the doctors or scientific staff handing out pills and the subjects taking the pills do not know whether the pills are the medicine being tested or a placebo with no effect.

If similar randomization and blinding techniques are not used in a memory experiment, there will be a high chance of experimenter bias. For example, let's suppose a scientist looks for memory behavior effects in two groups of animals, the first being a control group having no stimulus designed to affect memory, and the second group having a stimulus designed to affect memory. If the scientist knows which group is which when analyzing the behavior of the animals, he will be more likely to judge the animal's behavior in a biased way, so that the desired result is recorded.

A memory experiment can be very carefully designed to achieve this blind randomization ideal that minimizes the chance of experimenter bias. But such a thing is usually not done in memory experiments purporting to show evidence of a brain storage of memories. Scientists working for drug trials are very good about carefully designing experiments to meet the ideal of blind randomization, because they know the FDA will review their work very carefully, rejecting the drug for approval if the best experimental techniques were not used. But neuroscientists have no such incentive for experimental rigor.

Even in studies where some mention is made of a blinding protocol, there is very rarely any discussion of how an effective protocol was achieved. When dealing with small groups of animals, it is all too easy for a blinding protocol to be ineffective and worthless. For example, let us suppose there is one group of 10 mice that have something done to their brains, and some other control group that has no such done thing. Both may be subjected to a stimulus, and their “freezing behavior” may be judged. The scientists judging such a thing may be supposedly “blind” to which experimental group is being tested. But if a scientist is able to recognize any physical characteristic of one of the mice, he may actually know which group the mouse belongs to. So it is very easy for a supposed blinding protocol to be ineffective and worthless. What is needed to have confidence in such studies is not a mere mention of a blinding protocol, but a detailed discussion of exactly how an effective blinding protocol was achieved. We almost never get such a thing in memory experiments. The minority of them that refer to a blinding protocol almost never discuss in detail how an effective blinding protocol was achieved, one that really prevented scientists from knowing something that might have biased their judgments. 

For an experiment that judges "freezing behavior" in rodents, an effective blinding protocol would be one in which such freezing was judged by a person who never previously saw the rodents being tested. Such a protocol would guarantee that there would be no recognition of whether the animals were in an experimental group or a control group. But in "memory engram" papers we never read that such a thing was done.  To achieve an effective blinding protocol, it is not enough to use automated software for judging freezing, for such software can achieve biased results if it is run by an experimenter who knows whether or not an animal was in a control group. 

Sin #3: inadequate sample sizes, and a failure to do a sample size calculation to determine how large a sample size to test with.

Under ideal practice, as part of designing an experiment a scientist is supposed to perform what is called a sample size calculation. This is a calculation that is supposed to show how many subjects to use per study group to provide adequate evidence for the hypothesis being tested. Sample size calculations are included in rigorous experiments such as experimental drug trials.

The PLOS paper here reported that only one of the 410 memory-related neuroscience papers it studied had such a calculation. The PLOS paper reported that in order to achieve a moderately convincing statistical power of .80, an experiment typically needs to have 15 animals per group; but only 12% of the experiments had that many animals per group. Referring to statistical power (a measure of how likely a result is to be real and not a false alarm), the PLOS paper states, “no correlation was observed between textual descriptions of results and power.” In plain English, that means that there's a whole lot of BS flying around when scientists describe their memory experiments, and that countless cases of very weak evidence have been described by scientists as if they were strong evidence.

The paper above seems to suggest that 15 animals per study group is needed.  But In her post “Why Most Published Neuroscience Findings Are False,” Kelly Zalocusky PhD calculates (using Ioannidis’s data) that the median effect size of neuroscience studies is about .51. She then states the following, talking about statistical power:

"To get a power of 0.2, with an effect size of 0.51, the sample size needs to be 12 per group. This fits well with my intuition of sample sizes in (behavioral) neuroscience, and might actually be a little generous. To bump our power up to 0.5, we would need an n of 31 per group. A power of 0.8 would require 60 per group."

So the number of animals per study group for a moderately convincing result (one with a statistical power of .80) is more than 15 (according to one source), and something like 60, according to another source.  But the vast majority of "memory engram" papers do not even use 15 animals per study group.

Sin #4: a high occurrence of low statistical significance near the minimum of .05, along with a frequent hiding of such unimpressive results, burying them outside of the main text of a paper rather than placing them in the abstract of the paper.

Another measure of how robust a research finding is the statistical significance reported in the paper. Memory research papers often have marginal statistical significance close to .05.

Nowadays you can publish a science paper claiming a discovery if you are able to report a statistical significance of only .05. But it has been argued by 72 experts that such a standard is way too loose, and that things should be changed so that a discovery can only be claimed if a statistical significance of .005 is reached, which is a level ten times harder to achieve.

It should be noted that it is a big misconception that when you have a result with a statistical significance (or P-value) of .05, this means there is a probability of only .05 that the result was a false alarm and that the null hypothesis is true. This paper calls such an idea “the most pervasive and pernicious of the many misconceptions about the P value.” 

When memory-related scientific papers report unimpressive results having a statistical significance such as only .03, they often make it hard for people to see this unimpressive number. An example is the recent paper “Artificially Enhancing and Suppressing Hippocampus-Mediated Memories.”  Three of the four statistical significance levels reported were only .03, but this was not reported in the summary of the paper, and was buried in hard-to-find places in the text.

Sin #5: using presumptuous or loaded language in the paper, such as referring in the paper to the non-movement of an animal as “freezing” and referring to some supposedly "preferentially activated" cell as an "engram cell." 

Papers claiming to find evidence of memory engrams are often guilty of using presumptuous language that presupposes what they are attempting to prove. For example,  the non-movement of a rodent in an experiment is referred to by the loaded term "freezing," which suggests an animal freezing in fear, even though we have no idea whether the non-movement actually corresponds to fear.  Also, some cell that is guessed to be a site of memory storage (because of some alleged "preferential activation" that is typically no more than a fraction of 1 percent) is referred to repeatedly in the papers as an "engram cell,"  which means a memory-storage cell, even though nothing has been done to establish that the cell actually stores a memory. 

We can imagine a psychology study using similar loaded language.  The study might make hidden camera observations of people waiting at a bus stop.  Whenever the people made unpleasant expressions, such expressions would be labeled in the study as "homicidal thoughts."  The people who had slightly more of these unpleasant expressions would be categorized as "murderers."   The study might say, "We identified two murderers at the bus stop from their increased display of homicidal expressions." Of course, such ridiculously loaded, presumptuous language has no place in a scientific paper.  It is almost as bad for "memory engram" papers to be referring so casually to "engram cells" and "freezing" when neither fear nor memory storage at a specific cell has been demonstrated.  We can only wonder whether the authors of such papers were thinking something like, "If we use the phrase engram cells as much as we can, maybe people will believe we found some evidence for engram cells." 

Sin #6: failing to mention or test alternate explanations for the non-movement of an animal (called “freezing”), explanations that have nothing to do with memory recall.

A large fraction of all "memory engram" papers hinge on judgments that some rodent engaged in increased "freezing behavior,"  perhaps while some imagined "engram cells" were electrically or optogenetically stimulated. A science paper says that it is possible to induce freezing in rodents by stimulating a wide variety of regions. It says, "It is possible to induce freezing by activating a variety of brain areas and projections, including the hippocampus (Liu et al., 2012), lateral, basal and central amygdala (Ciocchi et al., 2010); Johansen et al., 2010; Gore et al., 2015a), periaqueductal gray (Tovote et al., 2016), motor and primary sensory cortices (Kass et al., 2013), prefrontal projections (Rajasethupathy et al., 2015) and retrosplenial cortex (Cowansage et al., 2014).” 

But we are not informed of such a reality in quite a few papers claiming to supply evidence for an engram. In such studies typically a rodent will be trained to fear some stimulus. Then some part of the rodent's brain will be stimulated when the stimulus is not present. If the rodent is nonmoving (described as "freezing") more often than a rodent whose brain is not being stimulated, this is hailed as evidence that the fearful memory is being recalled by stimulating some part of the brain.  But it is no such thing. For we have no idea whether the increased freezing or non-movement is being produced merely by the brain stimulation, without any fear memory, as so often occurs when different parts of the brain are stimulated.

If a scientist thinks that some tiny part of a brain stores a memory, there is an easy way to test whether there is something special about that part of the brain. The scientists could do the "stimulate cells and test fear" kind of test on multiple parts of the brain, only one of which was the area where the scientist thought the memory was stored. The results could then be compared, to see whether stimulating the imagined "engram cells" produced a higher level of freezing than stimulating other random cells in the brain. Such a test is rarely done. 

Sin #7: a dependency on arbitrarily analyzed brain scans or an uncorroborated judgment of "freezing behavior" which is not a reliable way of measuring fear.

A crucial element of a typical "memory engram" science paper is a judgment of what degree of "freezing behavior" a rodent displayed.  The papers typically equate non-movement with fear coming from recall of a painful stimulus. This doesn't make much sense. Many times in my life I saw a house mouse that caused me or someone else to shreik, and I never once saw a mouse freeze. Instead, they seem invariably to flee rather than to freeze. So what sense does it make to assume that the degree of non-movement ("freezing") of a rodent should be interpreted as a measurement of fear?  Moreover, judgments of the degree of "freezing behavior" in mice are too subjective and unreliable. 

Fear causes a sudden increase in heart rate in rodents, so measuring a rodent's heart rate is a simple and reliable way of corroborating a manual judgment that a rodent has engaged in increased "freezing behavior." A scientific study showed that heart rates of rodents dramatically shoot up instantly from 500 beats per minute to 700 beats per minute when the rodent is subjected to the fear-inducing stimuli of an air puff or a platform shaking. But rodent heart rate measurements seem to be never used in "memory engram" experiments. Why are the researchers relying on unreliable judgments of "freezing behavior" rather than a far-more-reliable measurement of heart rate, when determining whether fear is produced by recall? In this sense, it's as if the researchers wanted to follow a technique that would give them the highest chance of getting their papers published, rather than using a technique that would give them the most reliable answer as to whether a mouse is feeling fear. 

Another crucial element of many "memory engram" science papers is analysis of brain scans.  But there are 1001 ways to analyze the data from a particular brain scan.  Such flexibility almost allows a researcher to find whatever "preferential activation" result he is hoping to find.  

Page 68 of this paper discusses how brain scan analysis involves all kinds of arbitrary steps:

"The time series of voxel changes may be motion-corrected, coregistered, transformed to match a prototypical brain, resampled, detrended, normalized, smoothed, trimmed (temporally or spatially)...Furthermore, each of these steps can be done in a number of ways, each with many free parameters that experimenters set, often arbitrarily....The wholebrain analysis is often the first step in defining a region of interest in which the analyses may include exploration of time courses, voxelwise correlations, classification using support vector machines or other machine learning methods, across-subject correlations, and so on. Any one of these analyses requires making crucial decisions that determine the soundness of the conclusions."

The problem is that there is no standard way of doing such things. Each study arbitrarily uses some particular technique, and it is usually true that the results would have been much different if some other brain scan analysis technique had been used. 

Examples of Such Shortcomings

Let us look at a recent paper that claimed evidence for memory engrams. The paper stated, “Several studies have identified engram cells for different memories in many brain regions including the hippocampus (Liu et al., 2012; Ohkawa et al., 2015; Roy et al., 2016), amygdala (Han et al., 2009; Redondo et al., 2014), retrosplenial cortex (Cowansage et al., 2014), and prefrontal cortex (Kitamura et al., 2017).” But the close examination below will show that none of these studies are robust evidence for memory engrams in the brain. 

Let's take a look at some of these studies. The Kitamura study claimed to have “identified engram cells” in the prefrontal cortex is the study “Engrams and circuits crucial for systems consolidation of a memory.”  In Figure 1 (containing multiple graphs), we learn that the number of animals used in different study groups or experimental activities were 10, 10, 8, 10, 10, 12, 8, and 8, for an average of 9.5. In Figure 3 (also containing multiple subgraphs), we have even smaller numbers. The numbers of animals mentioned in that figure are 4, 4, 5, 5, 5, 10, 8, 5, 6, 5 and 5. None of these numbers are anything like what would be needed for a moderately convincing result, which would be a minimum of 15 animals per study group. So the study is very guilty of Sin #3. The study is also guilty of Sin #2, because no detailed description is given of an effective blinding protocol. The study is also guilty of Sin #4, because Figure 3 lists two statistical significance values of “< 0.05” which is the least impressive result you can get published nowadays. Studies reaching a statistical significance of less than 0.01 will always report such a result as “< 0.01” rather than “<0.05.”  The study is also guilty of Sin #7, because it relies on judgments of freezing behavior of rodents, which were not corroborated by something such as heart rate measurements. 

The Liu study claimed to have “identified engram cells” in the hippocampus of the brain is the study “Optogenetic stimulation of a hippocampal engram activates fear memory recall.” We see in Figure 3 that inadequate sample sizes were used. The number of animals listed in that figure (during different parts of the experiments) are 12, 12, 12, 5, and 6, for an average of 9.4. That is not anything like what would be needed for a moderately convincing result, which would be a minimum of 15 animals per study group. So the study is  guilty of Sin #3. The study is also guilty of Sin #7. The experiment relied crucially on judgments of fear produced by manual assessments of freezing behavior, which were not corroborated by any other technique such as heart-rate measurement. The study does not describe in detail any effective blinding protocol, so it is also guilty of Sin #2. The study is also guilty of Sin #6. The study involved stimulating certain cells in the brains of mice, with something called optogenetic stimulation. The authors have assumed that when mice freeze after stimulation, that this is a sign that they are recalling some fear memory stored in the part of the brain being stimulated. What the authors neglect to tell us is that stimulation of quite a few regions of a rodent brain will produce freezing behavior. So there is actually no reason for assuming that a fear memory is being recalled when the stimulation occurs. 

The Ohkawa study claimed to have “ identified engram cells” in the hippocampus of the brain is the study “Artificial Association of Pre-stored Information to Generate a Qualitatively New Memory.” In Figure 3 we learn that the animal study groups had a size of about 10 or 12, and in Figure 4 we learn that the animal study groups used were as small as 6 or 8 animals. So the study is guilty of Sin #3. Because the paper used a “zap their brains and look for freezing” approach, without discussing or testing alternate explanations for freezing behavior having nothing to do with memory, the Ohkawa study is also guilty of Sin #6. Judgment of fear is crucial to the experimental results, and it was done purely by judging "freezing behavior," without measurement of heart rate.  So the study is also guilty of Sin #7. This particular study has a few skimpy phrases which claims to have used a blinding protocol: “Freezing counting experiments were conducted double blind to experimental group.” But no detailed discussion is made of how an effective blinding protocol was achieved, so the study is also guilty of Sin #2.

The Roy study claimed to have “identified engram cells” in the hippocampus of the brain is the study "Memory retrieval by activating engram cells in mouse models of early Alzheimer’s disease."  Looking at Figure 1, we see that the study groups used sometimes consisted of only 3 or 4 animals, which is a joke from any kind of statistical power standpoint. Looking at Figure 3, we see the same type of problem. The text mentions study groups of only "3 mice per group," "4 mice per group," and "9 mice per group,"  and "10 mice per group."   So the study is guilty of Sin #3. Although a blinding protocol is mentioned in the skimpiest language,  no detailed discussion is made of how an effective blinding protocol was achieved, so the study is also guilty of Sin #2.  Some of the results reported have a statistical significance of only "<.05," so the study is guilty of Sin #4. 

The Han study (also available here) claimed to have “identified engram cells” in the amygdala is the study "Selective Erasure of a Fear Memory." In Figure 1 we see a larger-than average sample size was used for two groups (17 and 24), but that a way-too-small sample size of only 4 was used for the corresponding control group. You need a sufficiently high number of animals in all study groups, including the control group, for a reliable result.  The same figure tells us that in another experiment the number of animals in the study group were only 5 or 6, which is way too small. Figure 3 tells us that in other experiments only 8 or 9 mice were used, and Figure 4 tells us that in other experiments only 5 or 6 mice were used. So this paper is guilty of Sin #3. No mention is made in the paper of any blinding protocol, so this paper is guilty of Sin #2. Figure 4 refers to two results with a borderline statistical significance of only "< 0.05," so this paper is also guilty of Sin #4.  The paper relies heavily on judgments of fear in rodents, but these were uncorroborated judgments based on "freezing behavior," without any measure of heart rate to corroborate such judgments. So the paper is also guilty of Sin #7. 

The Redondo study claimed to have “identified engram cells” in the amygdala is the study "Bidirectional switch of the valence associated with a hippocampal contextual memory engram."  We see 5 or 6 results reported with a borderline statistical significance of only "< 0.05," so this paper is  guilty of Sin #4. No detailed description is given of how an effective blinding protocol was achieved, and only the skimpiest mention is made of blinding, so this paper is guilty of Sin #2.  The study used only "freezing behavior" to try to measure fear, without corroborating such a thing by measuring heart rates.  So the paper was guilty of Sin #7.  The study involved stimulating certain cells in the brains of mice, with something called optogenetic stimulation. The authors have assumed that when mice freeze after stimulation, that this is a sign that they are recalling some fear memory stored in the part the brain being stimulated. What the authors neglect to tell us is that stimulation of quite a few regions of a rodent brain will produce freezing behavior. So there is actually no reason for assuming that a fear memory is being recalled when the stimulation occurs.  So the study is also guilty of Sin #6. 

The Cowansage study claimed to have “identified engram cells” in the retrosplinial cortex of the brain is the study "Direct Reactivation of a Coherent Neocortical Memory of Context." Figure 2 tells us that only 12 mice were used for one experiment. Figure 4 tells us that only 3 and 5 animals were used for other experiments. So this paper is guilty of Sin #3. No detailed description is given of how an effective blinding protocol was achieved, and only the skimpiest mention is made of blinding, so this paper is guilty of Sin #2.    It's a paper using the same old "zap rodent brains and look for some freezing behavior" methodology, without explaining why such results can occur for reasons having nothing to do with memory recall. So the study is guilty of Sin #6. Some of the results reported have a statistical significance of only "<.05," so the study is guilty of Sin #4. 

So I have examined each of the papers that were claimed as evidence for memory traces or engrams in the brain. Serious problems have been found in every one of them.  Not a single one of the studies made a detailed description of how an effective blinding protocol was executed. All of the studies were guilty of Sin #7.  Not a single one of the studies makes a claim to have followed some standardized method of brain scan analysis. Whenever there are brain scans we can say that the experiments merely chose one of 101 possible ways to analyze brain scan data. Not a single one of the studies has corroborated "freezing behavior" judgments by measuring heart rates of rodents to determine whether the animals suddenly became afraid. But all of the studies had a depenency on either brain scanning, uncorroborated freezing behavior judgments, or both. The studies all used sample sizes far too low to get a reliable result (although one of them used a decent sample size to get part of its results). 

The papers I have discussed are full of problems, and do not provide robust evidence for any storage of memories in animal brains. There is no robust evidence that memories are stored in the brains of any animal, and no robust evidence that any such thing as an "engram cell" exists. 

The latest press report of a "memory wonder" produced by scientists is a claim that scientists implanted memories in the brains of songbirds. For example, The Scientist magazine has an article entitled, "Researchers Implant Memories in Zebra Finch Brains."  If you read the scientific paper in the journal Science, you will find that one of the crucial study groups used consisted of only seven birds, which is less that half of the fifteen animals per study group that is recommended for a moderately convincing result. The relevant scientific study is hidden behind a paywall of the journal Science.  But by reading the article in The Scientist, we can get enough information to have the strongest suspicion that the headline is an unjustified brag. 

Of course, the scientists didn't actually implant musical notes into the brains of birds.  Nothing of the sort could ever occur, because no one has the slightest idea of how learned or episodic information could ever be represented as neural states. The scientists merely gave little bursts of energy into the brains of some birds. The scientists claimed that the birds who got shorter bursts of energy tended to sing shorter songs. "When these finches grew up, they sang adult courtship songs that corresponded to the duration of light they’d received," the story tells us.  Of course, it would be not very improbable that such a mere "duration similarity" would occur by chance.  

It is very absurd to be describing such a mere "duration similarity" as a memory implant.  It was not at all true that the birds sung some melody that had been artifically implanted in their heads.  The scientists in question have produced zero evidence that memories can be artificially implanted in animals.  From an example like this, we get the impression that our science journalists will uncritically parrot any claim of success in brain experiments with memory, no matter how glaring are the shortcomings of the relevant study. 

Memory Molecule Myth: PKMzeta Debunked

In a recent article at the Nautilus web site, scientist Ken Richardson suggests that his fellow scientists have been guilty of some molecular mythology. He points out that scientists have repeatedly used “action verbs” in describing DNA, telling us that inside DNA are genes that “act,” “behave,” “direct,” “control,” “design,” are “responsible for,” and so forth. But then Richardson tells us “a counter-narrative is building” to correct such erroneous ideas, and then gives us reasons for thinking that genes are merely passive chemical units that do no such things.

Another example of molecule mythology involves a protein called PKMzeta. Some neuroscientists have suggested that PKMzeta has the ability to make memories last for decades in synapses, even though the proteins that make up synapses are very short-lived (having an average lifetime of two weeks or less). Quite a few of the papers or posts spreading this idea were written or co-written by the same person, Todd C. Sacktor. It is never explained clearly how a protein molecule could perform this great feat of magic. For anyone to explain such a thing clearly, he would first need to have a clear theory of how conceptual memories and episodic memories could be stored in synapses. No neuroscientist has ever presented a clear and explicit theory of any such thing. Neuroscientists merely vaguely tell us that somehow memory storage in a brain occurs through “synapse strengthening,” without presenting any clear theory of how that could occur.

Of course, if you do not have a clear theory of how memories could be stored (for even a few minutes) in synapses, you cannot possibly have a clear theory as to how some protein molecule such as PKMzeta could possibly cause memories stored in synapses to persist for decades, even though the proteins that make up such synapses are very short-lived, lasting an average of less than two weeks. Trying to defend against the charge that synapses are totally unsuitable for storing memories for decades, because of the short lifetimes of the proteins that make up synapses, a scientific paper states, “As long as PKMZ [PKMzeta] remains active and there is an absence of forces which terminate its activity (such as LTD), it will continue to sustain the biochemical changes at the synapse which serve as the neurobiological basis of memory, allowing the memory to persist for durations far exceeding the turnover of its component molecules.” But how could such a miracle of persistence occur, which would be like a message written in wet sand at the seashore persisting for decades, even though the wet sand was being replaced and written over whenever the tide came in? The science paper does not tell us.

Again, we have the case of an “action verb” inappropriately used to describe a molecule. We are told that PKMzeta has a “sustain” super-power allowing it to preserve fantastically complicated information supposedly stored externally in synapses made up of short-lived molecules that are constantly being replaced. There is nothing in the structure of PKMzeta that should cause us to believe it can do any such thing. No theorist has presented an explicit theory as to how anything like PKMzeta could preserve a memory. Such theorists may sometimes present chemical details to impress us, but such details do not constitute a theory unless a theorist gives explicit examples of precisely how specific memories (such as someone's memory of seeing Paris or someone's memory of details learned about World War I) could be permanently stored with the aid of PKMzeta.  No theorist has done any such thing. 

I suppose that if a PKMzeta molecule were able to cause memories to persist despite rapid protein turnover,  we might imagine it as some kind of "genius" molecule that has thoughts like this:

Oh, my goodness, I see that a memory is starting to degrade because of protein turnover! The memory now states, "Ottawa is the capitol of," which isn't even a full English sentence. Why, I'd better synthesize some new proteins to fill in for those proteins that died,  so there can be a nice complete English sentence. Now, what was that country that Ottawa is the capital of?  

Of course, anything the slightest bit like this is very hard to believe in. It would seem that the most minimal requirements that a molecule would have to fulfill in order to be a "memory maintenance molecule" would be the following:

(1) The molecule would have to somehow know whenever a particular protein molecule (that was part of a memory stored in a synapse) had died or disappeared because of the short lifespans of protein molecules.
(2) The molecule would have to somehow cause a replacement protein of the same type to appear in the same place as the vanished molecule, so that the memory did not degrade. 

The problem is that no one can envision a credible scenario under which a molecule could have either of these powers. To imagine how much of a miracle it would be for memories to persist despite constant protein turnover,  you can imagine a homeowner with ten picnic tables in his backyard, each of which is filled with leaves on which a word or two is written. Imagine these leaves spell out narratives, factual information, and ideas. But the problem is that about one day in three there are winds blowing the leaves off of the tables, and scattering them far away. Also, the leaves don't last longer than a year, because they tend to crumble. Now imagine the homeowner has to keep all this information preserved in the leaves, not just for a few nights but for 50 years. That would be a mountainous job.  An equally mountainous job would have to be done if memories were to be preserved in brains despite constant protein turnover causing proteins to persist an average of less than two weeks, and no one has explained how a molecule could possibly do such a feat.  Since synapses face not only rapid protein turnover inside them but also the problem that synapses don't last for longer than a year or two,  they have the same "double degradation" problem that such a homeowner would have with his information written on leaves. 




In the article here, a PKMzeta enthusiast is asked to explain how PKMzeta could cause memories to persist. The scientist gives a lengthy answer which fails to explain how PKMzeta could do such a thing. He merely says "a cluster of PKMzeta molecules can keep themselves turned on perpetually," and then claims that this supposed ability "is a plausible mechanism for memory persistence," without justifying that claim. This fragmentary theorizing is just hand waving. It has never been demonstrated that any cluster of PKMzeta molecules is capable of storing any information (such as a list of words) for a period as long as a month.  We can imagine hypothetical lab experiments that might try to show such a thing, but they have never been done. The paper here refers to "900 synaptic proteins." PKMzeta is only one of those 900 proteins in synapses, being no more common in synapses than an average synapse protein. You don't solve the "short lifetime of proteins" problem by trying to argue that one in 900 of those proteins might somehow have some stability.  As for the scientist's use of the word "plausible," it has been noted by others that "plausible" is the most abused word in theoretical science discourse, and that scientists often carelessly use the word "plausible" without ever doing anything at all to show a likelihood. 

But the PKMzeta enthusiasts have done a few studies which they claim lends credibility to their claims. I will describe a typical such study. A small number of mice are injected with something that suppresses the PKMzeta molecules in their body (or perhaps they are genetically engineered so that they don't have any PKMzeta). Memory experiments are then done. It is sometimes found that such mice perform not as well as normal mice. Such experiments have been hailed as support for the “memory maintenance” claims about PKMzeta.

There are several reasons why such studies do not at all show the claims about PKMzeta are correct. The first is that a result such as I described could never show that PKMzeta can save memories from destruction for years. Whenever memory is tested, it's hard to figure out what the cause is for a discrepancy between two test groups. A difference in a test result might be because (1) PKMzeta is involved in perceiving whatever observation is being tested; (2) or that PKMzeta is involved in memory storage; (3) or that PKMzeta is involved in memory retrieval; (4) or that PKMzeta has something to do with attention or focus used in a memory test. A test discrepancy could never tell us which of these things was involved. And if some mice did worse in remembering things without PKMzeta, that might justify the small claim that PKMzeta has something to do with memory, but could never justify the vastly more extravagant claim that PKMzeta is capable of preserving memories for decades.

Another reason why such studies do not at all show the claims about PKMzeta are correct has to do with a general malaise in neuroscience. A general problem in modern neuroscience is the production of papers with marginal results that we cannot trust because of things such as small sample sizes and publication bias. Let us imagine that neuroscientists want to prove some idea that fits in with their ideological expectations. A great number of experiments might be done, almost all producing no support for the idea. But perhaps 1 in 20 might produce results marginally supporting the idea, probably because of chance variations in data. Now, today negative results are vastly less likely to get published than positive results. So if 19 researchers get a negative result, conflicting with what neuroscientists hope to get, it could be that 10 of them don't even bother to write up their results as a scientific paper, and that the other 9 do write up a paper but don't get it published (because of the journal bias against negative results). However the one researcher who (by chance) got a positive result will write up his result as a scientific paper. Since it will be a result neuroscientists were hoping to get, he will almost certainly get the result published.

This publication bias is a great problem affecting the reliability of scientific research. Because of it we should follow a precautionary neuroscience rule like this: don't believe something has been established unless the result turns up fairly consistently at a high level of significance, in studies with large sample sizes.

Has this happened in regard to memory experiments involving PKMzeta? Not at all. In 2011 a scientist reported three separate studies showing that inhibiting PKMzeta has no effect on memory if tested between 10 and 15 day after the memory forms.  In 2013 two groups of scientists published results conflicting with claims that PKMzeta might allow memories to persist a long time. One study by a team of scientists used genetically engineered mice that had no PKMzeta. It found that such mice “have no deficits in several hippocampal-dependent learning and memory tasks,” and concluded that PKMzeta is not required for memory or learning. Another study by a different team of scientists found that absence of PKMzeta “does not impair learning and memory in mice.” A 2015 study found that inhibiting PKMzeta has no effect on memory in tests performed 30 days after the memory forms. A 2016 paper also found that that inhibiting PKMzeta has no effect on memory in tests performed 30 days after the memory forms.

Such studies would seem to completely debunk claims that PKMzeta enables memories to persist for decades in synapses despite the short lifetimes in the proteins.

The SUNY scientists such as Sacktor who helped to spread the PKMzeta myth have tried to fight back with papers such as this 2016 paper. But in that very paper we see evidence that second-rate science is being used to try to prop up claims about PKMzeta. In Figure 7 the scientists tell us how many mice were used for their experiment involving the memory effects of PKMzeta deprivation. They used only 8 mice per study group. That's way too small a sample size to get a moderately convincing result. It is well known that at least 15 animals per study group should be used to get a moderately convincing result. If you use only 8 animals per study group, there's a very high chance you'll get a false alarm, in which the result is due merely to chance variations rather than a real effect in nature.  In fact, in her post "Why Most Published Neuroscience Studies Are False," neuroscientist Kelly Zalocusky suggests that neuroscientists really should be using 31 animals per study group to get a not-very-strong statistical power of .5, and 60 animals per study group to get a fairly strong statistical power of .8.  Compare these numbers to the 8 animals per study group mentioned in Figure 7 of the Sacktor paper. 

This is the same “too small sample size” problem (discussed here) that plagues very many or most neuroscience experiments involving animals. Neuroscientists have known about this problem for many years, but year after year they continue in their errant ways, foisting upon the public too-small-sample-size studies with low statistical power that don't prove anything because of a high chance of false alarms.

If you look up the PRKCZ gene behind the PKMZeta protein molecule, using this page and this page of the Human Protein Database, you will find no characteristics that seem unusual, and nothing suggesting any superstar status. The pages make no mention of the gene even being used in synapses, telling us that the gene is "mainly localized to the cytosol" and "in addition localized to the plasma membrane."   The very idea of some kind of "superstar protein" or "superstar gene" is contrary to the experience in recent decades of scientists, who have found in general that bodily functions almost always involve the coordinated ballet of very many different genes (typically hundreds of them to accomplish a particular task). 

The 2015 scientific paper here shows that PKMzeta rapidly degrades in synapses. The authors say that therefore a stable amount of PKMzeta "would be difficult to maintain at synapses and store memories over long time scales." The paper tells us “There is growing evidence against a role for PKMzeta in memory.” Figure 9 of the paper also shows that a kind of cousin molecule or "isoform" of PKMzeta (PKC lambda) also quickly degrades, experiencing a 50% loss or degradation every 10 hours. So it seems that there is no truth to the idea of PKMzeta (or PKC lambda) as some magic bullet that allows memories to persist for decades in synapses that are constantly having their proteins replaced.

Where does that leave neuroscientists? It leaves them without a leg to stand on in their claims that memories are stored in brains. Based on everything we know about synapses, there is no reason to believe that synapses are capable of storing a memory for even a month, let alone the 50 years that is how long older humans can remember things. As discussed here and here, equally grave problems prevent scientists from creating any credible account of how memories could be encoded into neural states or how seldom-retrieved facts learned many years ago could be instantaneously recalled from a brain that seems to lack any capability for fast look-ups from exact neural positions. We also know (as discussed here and here) that massive damage can occur to brains (such as surgical removal of half of a brain) while producing little effect on memory, which would seem to be impossible if memories are stored in brains. How long before we realize that human memory cannot be a neural thing, but must be a psychic or spiritual phenomenon?

Postscript: Some people tell tall tales about the protein CAMKII similar to the tall tales told about PKMZeta. We are sometimes told that some alleged autophosphorlyation of CAMKII can help explain stable memories. Most of the reasons I have cited against PKMZeta also apply with equal strength to CAMKII. At this link we are told an experiment debunked the idea that  autophosphorlyation of CAMKII has a role in memory storage.  The lifetime of a CAMKII molecule is only 30 hours, according to this source. The book here makes this statement:

In the mid-1980's there was much excitement about the idea that autophosphorlyated CaMKII might serve as a self-perpetuating signal that could subserve permanent memory storage. However, a variety of experimental results generated since then suggests that perpetual activation of CaMKII does not occur with LTP-inducing stimulation or memory storage.

This scientific paper says the following:

Previous models have suggested that CaMKII functions as a bistable switch that could be the molecular correlate of long-term memory, but experiments have failed to validate these predictions....The CaMKII model system is never bistable at resting calcium concentrations, which suggests that CaMKII activity does not function as the biochemical switch underlying long-term memory.

This recent scientific paper says on page 9, "Overall, the studies reviewed here argue against, but do not completely rule out, a role for persistently self-sustaining CaMKII activity in maintaining" long term memory. 

Post-Postscript:  Those who have studied the history of science are familiar with epicycles, a complicated speculation that was introduced into Ptolemy's theory of astronomy, to try to fix cases in which the theory did not match observations. We may say these CaMKII speculations and PKMZeta speculations are epicycles intended to fix the failing synaptic theory of memory storage.  But while the Ptolemaic epicycles were exact speculations, the CaMKII speculations and PKMZeta speculations are very vague, failing to specify any exact theory of memory storage.