How Many Are Put At Risk for Junk Brain Scan Experiments?

In brain-related experiments we very often see defective or questionable research practices. To give examples:

• Scientists know that the most reliable way to do an experiment is to first state a detailed hypothesis, how data will be gathered, and how data will be analyzed, using methods called "pre-registered studies" or "registered reports." But most experimental neuroscience studies do not follow such a standard, but instead follow a much less reliable "fishing expedition" technique, in which data is gathered, and then the experimenter is free to slice and dice the data in any way he wants, trying to prove any hypothesis he may dream up after collecting the data. 
• Because very many neuroscience observations are the kind of observations where subjective interpretations may be at play, a detailed and rigorous blinding protocol is an essential part of any reliable neuroscience experiment. But such a blinding protocol is rarely used, and in the minority of neuroscience experiments that claim to use blinding, the blinding will usually be only fragmentary and fractional. 
• It is well-known that neuroscience experiments trying to establish correlations will not be reliable unless they use an adequate sample size. The minimum for a moderately reliable research result is 15 subjects per study group, with each mouse or person used in the person being one such subject. But neuroscience experiments commonly use much smaller study group sizes. It is extremely common to find that a neuroscience experiment used a study group size as small as 13 subjects or 11 subjects or 9 subjects of only 6 subjects. 
• A web site describing the reproducibility crisis in science mentions a person who was told of a neuroscience lab  "where the standard operating mode was to run a permutation analysis by iteratively excluding data points to find the most significant result," and quotes that person saying that there was little difference between such an approach and just making up data out of thin air. 
• A press release says this about brain scans: "Hariri said the researchers recognized that 'the correlation between one scan and a second is not even fair, it’s poor.'...For six out of seven measures of brain function, the correlation between tests taken about four months apart with the same person was weak....Again, they found poor correlation from one test to the next in an individual. The bottom line is that task-based fMRI in its current form can’t tell you what an individual’s brain activation will look like from one test to the next, Hariri said....'We can’t continue with the same old ‘"hot spot" research,' Hariri said. “We could scan the same 1,300 undergrads again and we wouldn’t see the same patterns for each of them.” The press release is talking about a scientific study by Hariri and others that can be read here.  The study is entitled, "What is the test-retest reliability of common task-fMRI measures? New empirical evidence and a meta-analysis." The study says, "We present converging evidence demonstrating poor reliability of task-fMRI measures...A meta-analysis of 90 experiments (N=1,008) revealed poor overall reliability."

About 40%  of neuroscience experiments involve rodents. This will come as a surprise to someone who reads the science news, where the headlines rarely refer to mice when announcing experiments that involved mice. When you study the apalling research practices so common in neuroscience experiments, and the very large prevalence of junk science, you may say something like, "It's largely a sham, but at least only mice or rats are being harmed."

But very many neuroscience experiments involve humans.  The experiments that involve humans may put patients at risk, for the sake of junk science results that do nothing to provide robust evidence for anything.

Let us consider fMRI studies. Neuroscientists love to do fMRI studies that involve brain scans. Such scans are typically healthy subjects, with there being no medical reason at all for the brain scan. 

It is a dogma among neuroscientists that fMRI scans are safe. But we should remember that neuroscientists are very dogmatic creatures who often repeat claims that are dubious and unproven (as you can tell by reading the posts on this blog).  Do we really know that fMRI scans are free of any risk?

One danger of fMRI scans is well-known: the risk of the very strong magnets used by such machines causing some metal object to be hurled at a high speed, causing injury or death.  In 2001 a six-year-old boy was killed in the US during an fMRI scan, when the machine turned an oxygen canister into a flying projectile.  There is also the risk that the more powerful fMRI scans may raise the risk of cancer in the person getting the scan. 

In the wikipedia.org article for Functional Magnetic Resonance Imaging, we read the troubling passage below:

"Genotoxic (i.e., potentially carcinogenic) effects of MRI scanning have been demonstrated in vivo and in vitro, leading a recent review to recommend 'a need for further studies and prudent use in order to avoid unnecessary examinations, according to the precautionary principle'. In a comparison of genotoxic effects of MRI compared with those of CT scans, Knuuti et al. reported that even though the DNA damage detected after MRI was at a level comparable to that produced by scans using ionizing radiation (low-dose coronary CT angiography, nuclear imaging, and X-ray angiography), differences in the mechanism by which this damage takes place suggests that the cancer risk of MRI, if any, is unknown.^"

Below are some relevant research papers:

(1) Referring to cardiac magnetic resonance imaging (CMR),

the 2015 study here ("Impact of cardiac magnetic resonance imaging

 on human lymphocyte DNA integrity") states, 

"The present findings indicate that CMR should be 

used with caution and that similar restrictions may apply as 

for X-ray-based and nuclear imaging techniques in

order to avoid unnecessary damage of DNA integrity with 

potential carcinogenic effect."

(2) The 2009 study here ("Genotoxic effects of 3 T 

magnetic resonance imaging in cultured human lymphocytes")

cautions about the use of a high-intensity

("3T and above") MRI, and states that 

"potential health risks are implied in the MRI and especially

HF MRI environment due to high-static

magnetic fields, fast gradient magnetic fields, and strong 

radiofrequency electromagnetic fields," also noting that 

"these results suggest that exposure to 3 T MRI induces

 genotoxic effects in human

 lymphocytes," referring to effects that may cause cancer. 

(3) The 2015 study here ("Biological Effects of Cardiac Magnetic 

Resonance  on Human Blood Cells") found that "Unenhanced CMR

is associated with minor but significant immediate blood cell

alterations or activations figuring inflammatory response, as well as 

DNA damage in T lymphocytes observed from day 2

 until the first month but disappearing at 1-year follow-up." The 

study found such worrisome results with the less-powerful 1.5T

scanning, which is being gradually replaced with twice-as-powerful

3T scanning.

A paper tells us the following about the newer twice-as-powerful

3T MRI machines that have been replacing the older 1.5T MRI

machines, suggesting their magnetic fields are much stronger than

the strength needed to lift a car:

"The main magnetic field of a 3T system is 60,000 times

 the earth's magnet field. The strength of electromagnets

 used to pick up cars in junk yards is about the field strength 

of MRI systems with field strengths from 1.5-2.0T.

 It is strong enough to pull fork-lift tires off of machinery,

 pull heavy-duty floor buffers and mop buckets into

 the bore of the magnet, pull stretchers across the room

 and turn oxygen bottles into flying projectiles reaching

 speeds in excess of 40 miles per hour."

Let us look at an example of a morally dubious study recently in the news, a study entitled "Predicting learning and achievement using GABA and glutamate concentrations in human development."  It was a study that attempted to link levels of two brain chemicals (GABA and glutamate) to learning ability and math ability.  As so often happens, a study that failed to produce any impressive result was misleadingly represented in the press as if it had found something important. 

We can see from Figure 2 of the study that no strong correlation was found between levels of these brain chemicals and math ability. In the graphs of Figure 2 we see little circles that are all over the place on the graphs, indicating a lack of any strong correlation.   We read these results, mentioning math achievement (MA):

"In particular, the glutamate concentration in the IPS was negatively associated with MA in younger participants but positively associated with MA in mature participants (Fig 2A, β = .13, t(225) = 4.54, standard error (se) = .03, PHC0 < .0001, R2ADJ = .85, dR2ADJ = .01). In contrast, the opposite relationship was found in the same region with GABA, which was positively associated with MA in younger participants but negatively associated with MA in mature participants (Fig 2B, β = −.14, t(224) = −5.39, se = .03, PHC0 < .0001, R2ADJ = .85, dR2ADJ = .01). Concerning the MFG, glutamate concentration was negatively associated with MA in younger participants but positively associated with MA in mature participants (Fig 2C, β = .11, t(220) = 3.59, se = .03, PHC0 = .0004, R2ADJ = .85, dR2ADJ = .01)."

The funny β characters in the quote above refer to a beta coefficient, which is not much different from a correlation coefficient (a measure of how strong the correlation is between two things).  The results given for the beta coefficients are all weak. They are all less than .15, and some of the correlations are negative.  A strong beta coefficient is something higher than .5.  A press story on this paper glowingly describes an "association" between math ability and these two brain chemicals, conveniently failing to mention that the association reported was so weak and inconsistent that it was not robust evidence of any real causal relation.  Because it found associations so weak, and because of methodology problems discussed below, the GABA and glutamate study has failed to prove anything important. 

This GABA and glutamate study has used a large number of subjects, so it cannot be criticized for using too small a sample size. However, there are other problems in the study, including the following:

• The study was not a pre-registered study that announced beforehand in detail what hypothesis would be tested, and how data would be gathered and analyzed, meaning it did not follow a "best practices" approach.
• The study did not make direct measurements of GABA and glutamate levels, but made indirect estimates using a technique called magnetic resonance spectroscopy.  Estimates of trace chemicals such as GABA using spectroscopy are subjective, and may be unreliable, because of serious confounding factors such as "spectral overlap."  A study using retesting to test the reliability of such GABA estimates in two different brain regions found only "low-moderate" reliability in estimates involving one of these two regions. 
• Chemicals such as GABA and glutamate fluctuate in the body from week to week. Readings from different parts of the brain may vary. There is no reason to think that a single brain scan  gives you a reliable indication of the average yearly level of GABA and glutamate in a subject. 
• The study fails to mention any blinding protocol that was followed.  Following a carefully defined blinding protocol is an essential for a study like this to be taken seriously as robust evidence for anything.  
• Since some "math savant" humans have shown an ability to perform extremely complex math calculations at blazing near-instantaneous speed, and since neurotransmitters move around slowly in the brain, and since there is no evidence that taking GABA or glutamate supplements improve math ability, it never made any sense to suspect that estimates of GABA and glutamate would be well-correlated with math ability; and the study found no such good correlation.  A previous 2017 study on older people scanned with MRI machines had found no good correlation between cognition and GABA levels, merely finding a very weak correlation, with an R-squared of merely .12. 
• The study made an unnecessary involvement of children. The hypothesis of whether GABA and glutamate affects math ability could have been tested just as well using only adult subjects. 

The most troubling thing about the study is that it needlessly  subjected hundreds of children to high-field 3T magnetic resonance imaging. These were not children who had any medical need for such imaging, and the paper says these subjects were recruited, lured by very small monetary incentives.  The study tells us, "All MRI data were acquired using a 3T Siemens MAGNETOM Prisma MRI System equipped with a 32 channel receive-only head coil."  The study here cautions that "exposure to 3 T MRI induces genotoxic effects in human lymphocytes," referring to potentially cancer-causing effects of using 3T MRI scanners.  

How long were these children scanned using this high-powered 3T MRI scanner?  We are told "the imaging session lasted approximately 60" minutes. A typical MRI scan for medical reasons very often takes only about 15 minutes. The paper tells us in Supplemental Table 1 that the brain scans were done on 51 six-year-olds, 51 ten-year-olds,  50 14-year-olds, and 49 16-year-olds. The younger a person is, the more likely he will be to be affected by possible cancer-causing things.  The subjects were induced to participate by giving them trifling sums of money such as 25 pounds  (about 35 dollars), and given such low compensation we may assume that a large fraction of the participants were from impoverished families.  Will some of these children who participated in this poorly-designed insignificant study end up with cancer decades from now because they were subjected to 60 minutes of unneeded 3T MRI scanning which "induces genotoxic effects" according to the previously cited paper? 

We'll probably never know, because neuroscientists don't seem to keep track of the long-term health results of the people they have brain-scanned in their experiments. It's kind of a policy of "scan 'em and forget 'em." Our neuroscientists are fond of saying there is "no proof" that fMRI imaging can be harmful, but that's because they are not doing the long-term patient health followup tracking to determine whether fMRI imaging produces a greater risk of cancer over 30 years or 40 years. 

The GABA and glutamate brain-scanning study I have discussed was very unusual in having a high number of subjects (more than 200). What is very much more common in neuroscience studies is to do brain imaging experiments involving fewer than 15 subjects. Almost all such experiments are pretty worthless, because we should have no confidence in studies that used fewer than 15 subjects per study group (the chance of false alarms is too high when fewer than 15 subjects are used).  Very many people may have suffered a needless cancer health risk from participating in the usually worthless "fewer than 15 subjects" human experimental studies that are so common in modern neuroscience research. 

Don't put me down as being anti-fMRI (I've had an fMRI myself, after being advised by a doctor to do so).  In countless medical treatment cases, the benefits of an fMRI scan are greater than the small risks. But people should not be put at risk by getting unnecessary brain scans solely for the sake of poorly designed studies that fail to prove anything because they followed Questionable Research Practices. 

I am not at all suggesting anyone should avoid an fMRI scan when a doctor recommends such a thing as medically advisable. But it is rather clear that in their zeal to load up their resumes with more and more brain scanning studies, our neuroscientists are rounding up too many paid subjects for unnecessary and potentially harmful brain scans.  What is really tragic is that such a large fraction of experimental brain scan studies follow Questionable Research Practices so badly that they qualify as "junk science studies" failing to provide any robust evidence for anything important.  It seems that very often human research subjects may be needlessly put at increased risk of cancer and other health dangers by being brain-scanned in scanners such as 3T MRIs, merely so that neuroscientists can round up more subjects for badly designed studies that do nothing to advance science because they fall very short of meeting the standards of good experimental science.   

When neuroscientists say brain scans are safe, they are referring to how much health trouble is now observed in people whose brains are scanned. No one has done some 25-year longitudinal study on the topic of whether people whose brains were scanned with 3T MRIs have a higher chance of  cancer 25 or 30 years in the future.  3T MRIs were only approved by the FDA in the year 2000, and the Siemens MAGNETOM Prisma MRI System used by the GABA and glutamate brain-scanning study was only approved in 2013. 

A scientific paper states this, referring to 3T MRIs:

"An insufficient number of validated studies have been carried out to demonstrate the safety of high strength static magnetic field exposure (Shellock, 2009). While MRI has been used for many years in the clinic, at higher Tesla levels (over 3 Tesla) the technology is relatively novel. Even less information about potential negative health effects exists for specific populations such as pregnant women and children." 

If I were an ethical advisor asked to approve proposals for brain experiments, I would have the following rules:

• I would never approve the use of human brain scanning for any experimental study that used fewer than 15 subjects for any of its study groups, because such studies are way too likely to produce false alarms. 
• I would never approve the use of human brain scanning for any experimental study that had not published publicly a detailed research plan, including a precise hypothesis to be tested, along with a very exact and detailed description of how data would be gathered and analyzed. We should not be putting people at risk for studies that do not follow best practices. 
• I would never approve the use of human brain scanning for any experimental study that had not published publicly a detailed blinding protocol to be followed, discussing exactly how blinding techniques would be used to reduce the risk of experimenter bias in which the experimenter "sees what he wants to see." We should not be putting people at risk for studies that do not follow best practices. 
• I would insist that any consent form signed by a subject to be brain scanned would include a detailed discussion of the reasons why brain scanning might be potentially hazardous, with negative effects appearing far in the future, along with a fair discusssion of the scientific literature suggesting such hazards. Currently a large fraction of such consent forms fail to frankly discuss such risk. 
•  I would never approve the use of any brain scanning on children in an experiment that did not absolutely require the participation of children. 

I strongly advise all parents never to let their children participate in any brain scanning experimental study unless a doctor has told them that the brain scan is medically advisable solely for the health of the child.  I advise adults not to participate in any brain scanning experimental study unless they have read something that gives them warrant for believing that the experimenters are following best experimental practices, and that there will be not be a very high chance that the adults will be undergoing unnecesary health risks for the sake of some "bad practices" poorly designed "fishing expedition" experiment that does not advance human understanding.  If a neuroscientist looking for research subjects tells you that brain scans are perfectly safe, remember that many neuroscientists often dogmatically make claims that are unproven or doubtful, and often pretend to know things they do not actually know (see this site for very many examples). 

I also strongly advise anyone who participated in any brain scanning experiment to permanently keep very careful records of their participation, to find out and write down the name of the scientific paper corresponding to the study, to keep a copy of any forms they signed, and to keep a careful log of any health problems they have. Such information may be useful should such a person decide to file a lawsuit.

When we examine the history of MRI scans, we see a history of overconfidence, and authorities dogmatically asserting that "MRI scans are perfectly safe," when they did not actually know whether they were perfectly safe.  Not many years ago there arose the great "contrast agent" scandal.  Scientists began to learn that what are called "contrast agent" MRI scans (given to 30 million people annually) may not be so safe. In such "contrast agent" scans, a subject is given an injection that increases the visual contrast of the MRI scan.  For a long time, the main substance in such an injection was gadolinium.  A mainstream cancer web site states, "Tissue and autopsy reports have also confirmed that gadolinium can accumulate in the brain and other organs." The results can be a health disaster, as described here. A 2019 Science Daily story says, "New contrast agent could make MRIs safer," letting us know that many of them previously were not so safe. On the same Science Daily web site, we read a 2017 news story with the title "MRI contrast agents accumulate in the brain."  A 2020 paper ("Side Effect of Gadolinium MRI Contrast Agents") says this:

"Until recently, it was believed that gadolinium is effectively cleared within 24 hours after intravenous injection, and that it does not have any harmful effects on the human body. However, recent studies on animals and analyses of clinical data have indicated that gadolinium is retained in the body for many years post-administration, and may cause various diseases."

Neuroscientists extensively used such contrast agents (as described here), very often putting human subjects at risk for the sake of junk poorly designed studies falling far short of the best experimental practices. All the while,  many of our experts were making the untrue claim that "MRI scans are perfectly safe," a statement which was not clearly  true for the large fraction of MRI studies that used gadolinium contrast agents. 

A recent article gives us a clue as to why so many junk science studies are occurring. It seems the number of PhD's is quite a few times greater than the number of available tenure-track positions, creating a pressure for not-yet-tenured PhD's to compromise on research standards, so that they get more published papers and paper citations. The article says, "Fifty-eight per cent of respondents to the survey are aware of scientists feeling tempted or under pressure to compromise on research integrity and standards."