More Doubts About BICEP2: The Dubious Part of Their Main Graph

At a time when many cosmic inflation theory fans are jumping the gun and popping champagne corks over Monday's BICEP2 study results, calling it an epic breakthrough, I hate to be a killjoy. I like to join a party as much as the next man. The problem is that I keep finding reasons for doubting the claims made about the study, that it provides evidence for the theory of cosmic inflation. My main reasons were given in this blog post, and some lesser software-related reasons were given in yesterday's blog post. Now I will discuss a very big additional reason for doubting the claims being made about BICEP2, a reason I haven't previously discussed: their main graph has a very dubious feature, a curve that is quite misleading.

The BICEP2 paper has two versions of the graph, one that is logarithmic and another that is not. Below is the non-logarithmic version, which makes it easier to see how the discovered data does not match what is predicted from the theory of cosmic inflation:

In this graph the black dots represent the new BICEP2 observations of b-mode polarization. The vertical lines are error bars representing uncertainty in the data. The bottom dashed line is a prediction of b-mode polarization made by one version of the theory of cosmic inflation (a “wishful thinking” version chosen by the BICEP2 team, as I will explain in a minute). The solid line represents contributions to b-mode polarization projected to occur from gravitational lensing. The upper dashed curved line represents the b-mode polarization that could occur from a combination of gravitational lensing and the version of the inflation theory that was chosen by the BICEP2 team to make their data match inflation theory.

Now the untrained eye can spot a big problem with this graph: the observations do not match what is expected. While the first two black dots match the top dashed line (as does the last black dot), several of the other black dots are way above the top dashed line, in particular and seventh and eighth dots. On this basis, we are entitled to say: inflation theory falls way short.

But here is a very important fact about this graph: the bottom curved hill-shaped dashed line (the supposed contributions from cosmic inflation) is not “the” prediction from the theory of cosmic inflation. It is instead the prediction from a particular version of the inflation theory carefully chosen by the BICEP2 team so that their observational results can be matched to inflation theory. The version in question is one that drastically contradicts conclusions made with a 95% confidence level last year by a much larger team of scientists, using the Planck space observatory.

The “prediction from inflation” that appears as the hill-shaped red dashed line on the above graph all depends on a particular data item called the tensor-to-scalar ratio, which cosmologists represent with the letter r. In a scientific paper co-authored last year by more than 200 scientists, the Planck team concluded with a 95% confidence level that this tensor-scalar ratio is less than .11. But in the graph above the BICEP2 team chose to disregard these findings, and use on their graph an extreme version of the inflation theory in which the tensor-scalar ratio is .2 (200% higher than the maximum value set by the larger group of scientists).

Why would the BICEP2 team have done that? Because it allowed them to produce a graph showing a partial match between their observations and the predictions of a cosmic inflation theory. A triumph of wishful thinking. It's rather like a husband reassuring his wife by showing her a graph in which his projected income rises by 50% for each of the next five years.

But what would the key BICEP2 graph have looked like if they had accepted the limit set by the much larger Planck team? The graph would have looked rather like the graph below, except that the left half of the top red dashed line would have to be dropped way down, and none of the observations would be anywhere near close to matching the predictions from inflation (except for the last one, at a point in the graph where inflation is irrelevant, and all contribution is from gravitational lensing). 
 

The BICEP2 team could have produced a graph like the one above (but with the left half of the top dashed line dropped way down, to equal the green line plus the solid red line). That is exactly what they should have done. They might then have made an announcement like this:

We have some interesting new observations. But we're sorry to report that, respecting the limits set last year by a much larger team of scientists, our observations provide no evidence to back up the theory of cosmic inflation. 
 

Instead, the BICEP2 team chose to put in a bogus red dashed line in their key graph, representing a farfetched, extreme wishful-thinking version of the cosmic inflation theory, one that relies on a version of inflation with a tensor-scalar ratio (r) about twice as high as the maximum allowed value according to the larger Planck team. Rather than candidly showing such a red-dashed line as just one possible version of inflation, they put it on the graph as if it was the only version of inflation. 
 

It was a great way to grab press headlines, but not very honest or candid.

When we use the predictions of inflation using the Planck team's estimate of the upper limit of the tensor-scalar ratio (with a 95% confidence level), corresponding roughly to the green line in the graph above, we are led to think that the BICEP2 team's observations provide no support to a theory of cosmic inflation.

Postscript: This post uses the assumption that smaller values for the tensor-to-scalar ratio (r) cause the "hill" of the inflation prediction to drop much smaller, a point that is clear from looking at this site. 

Best Practices Software and Cowboy Coding

Note: I have revised this post to remove its original references to the programming sins of one particular programmer. I have decided to take mercy on this person, and remove all references to his coding sins. 

Let's look at the difference between two very different types of programming: best practices software and cowboy coding.

Best-practices Software

Best-practices software is software developed according to software industry guidelines for quality. Examples of these best-practices include the items below. There is not always time to follow all of these practices, but the overall quality and maintainability of the code depends on how many of these standards are followed:

1. Each source file contains a comment specifying the type of code in that file.
2. Each method, subroutine or function contains a comment explaining what is done by that method or subroutine. The only exception to this rule is when the name of the method, subroutine, or function leaves no doubt as to exactly what is being done.
3. There is a short description of each argument to any method that takes arguments, except in the case when the name of an argument leaves no doubt as to what that argument is.
4. There are comments explaining the logic in any particularly complicated or hard-to-understand parts of the code.
5. Variables are given names that help to document what they stand for.
6. Good coding practices are followed by each developer.
7. Once the code is finished, it is placed in a version control system. Whenever a source code file is changed, the new version is checked into the version control system, with a comment discussing what changes were made.
8. The code is developed by a team of developers, who can cross-check each others' work.
9. Once the code is written, documentation is written explaining how the code works and how it can be modified.
10. A team of quality assurance experts (known as the QA staff) are finally brought in to rigorously test the code to find any bugs in it.
11. Once the code has been released, a meticulous record is kept of all changes in the code and all reported bugs, along with which of the bugs were fixed.
12. Any known defects or limitations of the code are clearly documented.
13. Each subsequent release of the code is given a new version number, with a description of exactly how the code changed during the latest release.

Practices such as these are followed by mission-critical software, or software on which great amounts of money are riding, or software on which lives depend. For example, if a company were writing software for a nuclear reactor, or software for an expensive space mission, or software for guiding a jetliner, it would tend to follow most or all of these best practices.

However, there is a totally different way of programming that is often used, a quick-and-dirty way of programming. This way of programming is sometimes called cowboy coding.

Cowboy Coding

Cowboy coding is what happens when a single developer produces some code, typically in a quick-and-dirty method. The cowboy coder isn't interested in any quality guidelines that will slow him down. He typically grinds out some software without doing much to document it. He may make no use of version control. He may then release his work without having had anyone check it other than himself. Typically the cowboy coder just kind of says, “It seems to work well when I try it – let me know if you find anything wrong with it.” A typical cowboy coder makes little or no attempt to produce written documentation for his software, and may take no care to document different versions or to document exactly which bugs were fixed. 

 

Now cowboy coding certainly has its place. Lots of programs are not mission critical, and need not be developed using best practices. It would be overkill to follow the best practices listed above when creating some little graphic utility for doing something like allowing a user to add text to an image.

However, it must be noted that cowboy coding is a severe danger if it is used for some critical part of a hugely important scientific study. This is because cowboy coding isn't very reliable. Maybe it does the right thing, and maybe it doesn't. It can be hard for anyone to tell except the original cowboy coder, and probably he doesn't even know. This is no exaggeration. Poorly documented software code is very hard to read, even if you are the original developer. Countless cowboy coders simply don't know whether their cowboy-coded projects work correctly. I've cowboy-coded quite a few little projects, and then when I went back to them much later, I could often hardly figure out any more what exactly they were doing.

The Huge Problem of Cowboy Coding in Scientific Studies

There is a very big problem that modern scientific studies often rely on dubious software solutions that have been cowboy-coded. Modern science involves incredibly high amounts of specialized data processing. Scientists cannot buy off-the-shelf software to handle these specialized needs. This is because each scientific specialty requires its own specific type of software, and the market for such software is so small that few software publishers will cater to it. 

What very often happens is that scientists will often write their own software programs to handle their own specialized needs. Such efforts are often one-man cowboy-coded efforts that do not come anywhere close to meeting the best practices of modern software development.  We have many scientists writing amateurish code that any full-time software developer would be ashamed to put his name on.  But such code might become a critical linchpin in some scientific study that uses up millions of federal dollars. 

We need new standards to minimize this problem. One possibility is to include software professionals as part of the peer review process for scientific studies. There are major scientific studies that are 30% science and 70% data processing. But in the peer review process, only scientists review the study. This makes no sense. Software professionals should be included in the process, to a degree that depends on how much data processing was done by the study.    

BICEP2 Study Has Not Confirmed Cosmic Inflation

The BICEP2 study was released yesterday, and found some evidence of something called b-mode polarization in the early universe. Advocates of the theory of cosmic inflation were quick to trumpet these results, with many of them claiming that the study had finally confirmed the theory of cosmic inflation. This theory maintains that the universe underwent a period of exponential inflation during a fraction of its first second.

But there are several reasons why the BICEP2 study does not confirm cosmic inflation or even provide substantial evidence for it.

The first reason is that a single scientific study rarely proves anything. Having followed scientific developments closely for more than four decades, I have lived through many a case of scientific announcements that did not stand the test of time. I remember back around 1980 an announcement in which scientists announced a fate for the universe (collapse) that is the exact opposite of the fate they now predict for it (unending expansion). I also remember the famous “life on Mars” announcement in the 1990's which did not pan out. At this web site a scientist says that there is only a 50% chance that the results from this BICEP2 study will hold.

Another reason for doubting this BICEP2 study is that it makes an estimate for an important cosmological ratio called the tensor-to-scalar ratio, and that estimate is about twice the maximum possible value, according to the estimate from a very definitive source on the topic, the Planck team of scientists (larger than the BICEP group). Apparently some group of scientists are in serious error in regard to this matter, and there is a 50% chance that it is the team that made yesterday's BICEP2 study. If they are the ones who are wrong, it throws much of their study into doubt.

A third reason why the BICEP2 study does not confirm the theory of cosmic inflation is that BICEP2's results do not match well with the predictions of that theory.

The supporters of the inflation theory are citing the graph below from the BICEP2 study. The black dots are the BICEP2 observations, with the vertical lines being error bars (representing uncertainty in the data). The bottom red dashed line is what we expect from the cosmic inflation theory.

Considering just what is predicted from the theory of cosmic inflation, the results do not match well at all. The little black dots show a rise in the line exactly where the cosmic inflation theory predicts a fall in the line.

To patch up this embarrassing discrepancy, the authors add a “gravitational lensing” factor, which seems like quite the little fudge factor. Gravitational lensing is a very exotic effect that is not an easy thing to predict or nail down with any certainty. Estimating the amount of gravitational lensing that occurred long ago is like estimating the total tonnage of asteroids that have struck in the past billion years – very much a type of estimate that involves a huge amount of uncertainty. The BICEP2 authors seem to have got an estimate for gravitational lensing by making inputs to a 3-year old computer program called LensPix. There are lots of ways to go wrong there, either in the inputs or in the software (and the site for the software says “there are almost certainly bugs” in this software).

What is interesting, however, is that even if you accept as gospel truth this estimate of the amount of gravitational lensing, the data from BICEP2 ends up strongly diverging from the expected results produced from the estimated amount of gravitational lensing and cosmic inflation.

It is very hard to tell how big this discrepancy is from the graph shown above, because it uses two sneaky data presentation techniques to make the discrepancy look much smaller than it is. The techniques are: (1) the graph unnecessarily includes a whole load of irrelevant data in the top half of the graph, causing the scale of the graph to be unnecessarily large; (2) the graph uses a logarithmic scale (a type of scale that often tends to make two data items look closer than they are).

I can use exactly the same techniques to make a graph that makes it look like a dishwasher makes almost the same amount of money as a Wall Street bond trader.

But, thankfully, buried within the BICEP2 scientific paper is a nice simple non-logarithmic graph that shows just how great the difference is between the BICEP2 results and the results predicted from the cosmic inflation theory. The graph is below.

In the graph above the black dots are the new BICEP2 observations. The vertical lines are uncertainties in the data. The bottom red dotted line is the prediction from the theory of cosmic inflation. The solid red line is the estimated gravitational lensing factor. The upper red dashed line is the result predicted given a combination of the gravitational lensing factor and the theory of cosmic inflation.

Notice the big difference between the observed results and the expected result. Even if we include this highly uncertain gravitational lensing fudge factor, the predicted results from the cosmic inflation theory do not closely match the observed results. Note that the sixth and seventh black dots are way above the top dashed red line.

Therefore these results are far from being a confirmation of the theory of cosmic inflation. They can't even be called good evidence for cosmic inflation.

I may also note that there are numerous non-inflationary cosmological models that might produce the type of polarization observations that BICEP2 has produced. If there is currently a shortage of such models, it is largely because cosmic inflation speculations have almost monopolized the activities of theoretical cosmologists during recent decades.

The BICEP2 observations can be explained by the decay of exotic particles, or by some noninflationary exotic phase transition. Or it could be that all of the observed effect is produced by gravitational lensing and none of it produced by cosmic inflation. Scientists are already assuming that most of the observed effect is being produced by gravitational lensing; it's a short jump from “most” to “all” (particularly given the many uncertainties involved in estimating the amount of gravitational lensing).

If cosmologists spend as much time producing non-exponential non-inflationary models of the early universe as they do producing models that involve inflationary exponential expansion, they will probably find that non-exponential non-inflationary models are able to explain the observed BICEP2 results just as well, and perhaps even better.

Because the effect observed by the BICEP2 study can be produced by gravitational lensing, and because we will for many decades be highly uncertain about how much gravitational lensing has occurred in the past, it is very doubtful that any study like the BICEP2 will ever be able to provide real evidence for a theory of cosmic inflation. Just as UFO photographs rarely prove anything (because there are so many ways in which lights in the sky can be produced), a study like BICEP2 doesn't prove cosmic inflation (because there are other ways, such as gravitational lensing, that the observed polarization effect can be produced).

The case for the theory of cosmic inflation theory is much weaker than many think. In a nutshell the standard sales pitch for the theory is that it solves two cosmological problems: one called the flatness problem and the other called the horizon problem. The flatness problem is an apparent case of cosmic fine-tuning, and the horizon problem is an example of cosmic uniformity. The weakness in trying to solve these problems with a theory of cosmic inflation is that we have many other apparent cases of cosmic fine-tuning and many other cases of astonishing cosmic uniformity (including laws of nature and constants that are uniform throughout the universe). Inflation theory claims to solve only one of these many cases of apparent cosmic fine-tuning, and only one of the many cases of cosmic uniformity. That puts it in not a very good position, rather like a theory of the origin of species that only explains the origin of lions and tigers without explaining the origin of any other animals. I will explain this point more fully in a later blog post.

What is particularly ironic is that the theory of cosmic inflation claims to help in getting rid of some cosmic fine-tuning, but the theory itself requires abundant fine-tuning of its own to work, as many parameters in the theory have to be adjusted in just the right way to get a universe that starts exponentially inflating and stops inflating in a way that matches observations. 

Postscript: The chart below (in which I have added a green line) shows one way we can explain the BICEP2 observations without requiring any cosmic inflation.  We simply imagine a slightly higher amount of gravitational lensing (shown in the green line). The shape of this line matches the shape of the gravitational lensing estimated by the BICEP2 study (solid red line). Because the green line passes through all of the vertical error bars, it is consistent with the BICEP2 observations. 

BICEP2 graph with an added trend line (green)

Post-postscript: at this link cosmologist Neil Turok says, "I believe that if both Planck and the new results agree, then together they would give substantial evidence against inflation!"

Post-post-postscript: See the post here for a discussion of wishful thinking and cherry picking involved in the main graph shown above.   

Post-post-post-postscript: See this link for a National Geographic story on how the BICEP2 results may be caused by dust, not cosmic inflation. 

Yet another postscript: see this post for a discussion of a talk at Princeton University in which a scientist gives a presentation that gives a devastating blow to the inflated claims of the BICEP2 study. The scientist gives projections of dust and gravitational lensing which show how such common phenomena (not from the Big Bang or cosmic inflation) can explain the BICEP2 observations.  

Yet another postscript: In this article in the scientific journal Nature, it is explained that two recent scientific papers have concluded that there is no significant evidence the BICEP2 signals are from cosmic inflation or gravitational waves, with dust and cosmological lensing being an equally plausible explanation.

Teenage World Savior: A Science Fiction Story

All attempts to defeat the hostile extraterrestrial invasion had failed utterly. A meeting of military officers was convened at the house of Jonas MacDonald, a physicist who specialized in high-energy physics. The officers were there to ask the physicist if he knew of any high-tech way that the invading extraterrestrials could be attacked, perhaps with something such as lasers or electromagnetic pulse weapons.

“So far our military efforts have been a complete disaster,” said General Curtis. “After the aliens landed in New Jersey, and wiped out many people, we've hit them with every conventional weapon we had. We've dropped countless bombs. We've strafed them with our jets again and again. We've shelled the hell out of them with our best artillery. But we're getting nowhere. The alien stronghold keeps growing larger and larger.”

“Why aren't such attacks working?” asked MacDonald.

“They seem to have some kind of strange energy bubble around their landing area,” explained Curtis. “It's some kind of super-strong energy field that is able to vaporize incoming bombs and bullets. Whenever we shoot something at the alien stronghold, our bombs and bullets just kind of melt as soon as they touch the protective energy bubble.”

“Have you thought about using nuclear weapons?” asked MacDonald.

“No, that's out of the question,” explained General Curtis. “The prevailing winds would cause radioactive fallout to drift on to New York City.”

“Do you have a picture of what these extraterrestrials look like?” asked MacDonald.

General Wheeler produced a photograph, and put a picture on the table.

“Let me think,” said MacDonald. “There might be some kind of high-energy proton beam we could use to attack these things.”

MacDonald's 13-year-old son Artie walked into the room. Artie should have been at school, but he had got suspended for starting a big food fight in his high school cafeteria.

“Is that what the aliens look like?” asked Artie. “Cool.”

“This meeting is classified,” said MacDonald. “Artie, clear out of here.”

The men continued to discuss MacDonald's ideas for a high-energy proton beam. Twenty minutes later Artie came back into the room.

“Dad, I know I'm not supposed to be here,” said Artie. “But I've got an idea. I've got an idea about how you might defeat the aliens.”

“Artie, have you lost your senses?” asked MacDonald. “Nobody wants to hear a teenager's ideas on saving the world from an alien invasion.”

“But, Dad, it's a really good idea,” said Artie.

“Let the boy speak,” said General Curtis. “Right now, we're desperate for new ideas.”

“I got the idea from the cafeteria food fight I got suspended for,” said Artie. “We can fight the aliens with food.”

“Very funny,” said MacDonald. “Now go to your room, and don't bother us again.”

“I'm not kidding, Dad,” said Artie. “There's a way to do it. Look at that picture of the alien. He has no real nose. Just a kind of a slit for a nose. So my guess is these aliens are probably sensitive to particles in the air. If we bombard them with fine particles, it may kill them. The easiest way to bombard them with fine particles is by using spices.”

“Spices of what type?” asked General Wheeler.

“Any type of spice that is a very fine powder,” explained Artie. “Cinnamon or curry powder would probably do the job.”

“That's the craziest idea I've ever heard,” said MacDonald. “The aliens are protected by an energy bubble that would make it so that the powder couldn't even fall into the alien stronghold.”

“But it just might work,” said General Wheeler. “Who knows – maybe their protective energy bubble was only designed for things like bombs and bullets. Maybe a fine powder could get through that thing. Let's give it a try.”

So the conventional high-explosive bombs were taken out of a military jet. Two giant vats of cinnamon and curry powder were loaded into the jet. The jet made a bombing run of the alien stronghold, dumping the curry powder and cinnamon on to the strange alien structures.

The protective energy bubble of the aliens had been designed to destroy only incoming objects larger than about a millimeter. The curry powder and cinnamon fell right through the protective bubble.

The aliens breathed in the curry powder and cinnamon, and all died instantly. They came from a dustless planet, and had never evolved any apparatus for protecting their lungs from fine particles.

And so the teenage boy who had started a food fight at his high school cafeteria became known as the unlikely world savior who started a food fight that saved planet Earth.