Why Machines Will Not Soon Be as Intelligent as Men

The distinguished British newspaper Financial Times had a very dubious story yesterday. It had a headline saying, “Computers will soon become more intelligent than us.” Throughout the story, this extremely doubtful assertion was treated as fact. Various distinguished persons were quoted, and they made some overheated assertions. Elon Musk was quoted as saying that supercomputers may be more dangerous than nukes. Someone named Peter Thiel said, “People are spending way too much time thinking about climate change, way too little thinking about AI,” as if artificial intelligence was a much bigger threat than global warming.

Financial Times has apparently drunk the Singularity kool-aid. The Singularity is the notion that there will soon be an “intelligence explosion,” which will cause computers and robots to become far more intelligent than us. Supporters of this idea are fond of showing graphs showing exponential increases in computer memory or speed. Such people are fond of suggesting that this means there will be some exponential increase in machine intelligence similar to the increase shown in such graphs.

Let me give three strong reasons for thinking that computers will not soon have anything like a general intelligence such as humans have.

Reason #1: Developing a computer as intelligent as a human would require a long period of exponential progress in software, but software has always progressed at a slow, gradual rate, never at an exponential rate.

There are two main parts of a computer: the hardware and the software. In order for you to have anything like a computer as intelligent as a human, you would have to have computer hardware millions of times better than what we have now. You would also have to have computer software millions of times better than what we have now.

Progress in hardware has often occurred exponentially, as in the case of Moore's law. This is the law that about every 18 months or two years, the speed or memory capacity of computers doubles. Since you only have to double something 25 times to make it a million times greater, there is some hope that in this century we will have computer hardware that would be as fast as it would need to be to surpass human intelligence.

But when it comes to software, we have a totally different situation. Software development does not progress at anything like an exponential rate. It progresses at a much slower linear rate. Two years from now the world's software won't be twice as good as it is now. We will be lucky if it is 25% better, because software only gets perhaps 10% better from one year to the next.

What is the difference between growth at something like an annual 10% linear rate, and growth at an exponential “double every two years” rate? It's a world of difference. If hardware speed doubles every two years, it will increase a million times in only 50 years. But if software gets better by only 10 percent per year, it will take it 100 years to get about 6000 times better. To have computers as intelligent as a human, you need computers with software not just a few thousand times better than current computers, but millions of times better.

In short, while progress in computer hardware may be telling us, “The Singularity is coming soon,” progress in software development is telling us the opposite message, basically: “This will take centuries at least.” You can't overcome this objection by asserting that computers will write their own software. While computers are good at cranking out software code for little specific problems, tackling extremely high-level problems (such as creating software for general intelligence) requires a mind as smart as a human to work on it.  Dumb computers will not be able to write software that results in smart computers.

The following graph illustrates the point about linear growth versus exponential growth. If we assume that software capabilities will increase by 30% every two years (an optimistic assumption based on previous trends), then we have a growth in software capabilities over the next forty years which doesn't even register on a graph when graphed against the growth in hardware capabilities.

Reason #2: Software intelligibility problems will act as a gigantic “speed bump” or brake that will tend to slow the development of software needed for computer intelligence.

In order to understand the software intelligibility problem, let us imagine a newly hired computer programmer named John who starts to work today on some very complicated computer system.

John: In my first week here I've been looking through the code, as you told me to do. Wow, that's a lot to try to understand. 100,000 lines of dense Java code.

Will: It's much worse. That code you were looking at was only the “core kernel” of our hugely complicated system. To understand the complete system, you have to also understand the 30,000 lines of web interface code, the 80,000 lines of database stored procedures, and the 50,000 lines of Perl batch processing code.

John: Wow. So is there anyone here who has ever read through all that stuff, to get a complete understanding of what is going on with this system?

Will: Hell no. That would take forever, and give you the world's biggest headache. Nobody here really “understands” the system. We just know enough about it to tweak it here and there.

I know for sure that a situation very similar to this exists today in many large software projects. We can call it a software intelligibility problem. As computer software gets more and more complex, and more and more lines of code are added (often written in hard-to-read computer languages), it becomes harder and harder for humans to understand what the code is doing.

What will happen as software progresses more and more, as the total number of lines of programming code in the world increases by ten times, a hundred times, a thousand times? The software intelligibility problem will get much worse. As we try to move from one hundredth of the way to one tenth of the way to creating the software needed to create computers as intelligent as humans, we will tend to find that we are ending up with software systems that totally surpass our capacity to understand them. This will act as a gigantic brake that will tend to slow further progress.

What happens when humans don't understand a large software system they are working on? Various nasty things like this: programmers make “blind changes” without understanding the ramifications; more and more subtle bugs get introduced into a system; it becomes impossible to re-factor or re-design a system; huge systems end up slowly evolving into nightmarish dead ends that no one wants to work on; you get “fragile systems” that easily break when someone tries to improve them; you see “code degradation” in which a body of programming code ends up getting worse rather than better; you may end up with $100 million dollar project failures. Such things often end up resulting in “negative progress.”

Rather than assuming that we will keep getting a software development progress rate of 10% or 15% per year, it is entirely possible that we will “hit the wall” fifteen or twenty years in the future, as we find that our software systems have grown out of control, morphing into enigmatic sprawling systems that are like mysterious black boxes that we no longer understand. At such a point we might not even be able to count on a 10% yearly progress rate in software development (itself totally inadequate to lead to machines as smart as humans in 100 years).

Reason #3: We have very little realistic hope in this century of being able to use the human brain as a shortcut to reach artificial intelligence equaling human intelligence.

When faced with objections such as these, singularity enthusiasts fall back to the idea that we will be able to get the software needed for artificial intelligence by studying the human brain. The idea is that we will somehow scan the human brain to learn the exact secrets of how it works, and then use those secrets as a shortcut to create intelligent machines.

While such an approach cannot be ruled out as a kind of “thousand year project,” there is no reason for being hopeful that within a century any such approach will work in terms of actually resulting in the software needed to make a computer as intelligent as humans. To understand what a “pipe dream” such a hope seems to be, we must take a step backward and look at what a mystery human consciousness is.

We understand various things about the human brain and its components, and we understand something about how readings of brain activity (such as MRI readings) differ during different types of brain activity. But we basically have no understanding of the essential matter of how it is that brains are able to achieve the “miracle of consciousness.” Brains are physical things, and thoughts are mental things. We have no real understanding of how the first thing leads to the second, any more than we understand how the universe arose from nothingness at the time of the Big Bang. Nor do we have any real understanding of how the brain is able to store concepts or memories.

Moreover, it is quite possible that we will never understand such a thing. It is hard to imagine that we will in the future examine some brain cells at a higher magnification and one day say, “Aha, there is a thought being produced,” or something like: “Yes, I see it now; there is a memory being stored.”

As discussed here and here, there are numerous unexplained anomalies involving the operations of brains and consciousness (such as the fact that some people can function well despite having large parts of their brains destroyed or removed). So it is all too plausible that human consciousness requires some unknown factor (call it an X factor) that involves something more than just neurons and brain electrical activity. In such a case, all attempts to build machine intelligence from studying the human brain will probably be futile.

Even if one rejects such an idea, and sticks with a materialist idea of consciousness as merely the result of millions of years of blind evolution, there still would be no reason to hope that brain secrets millions of years in the making could soon be unraveled in a way that would lead to the development of machine intelligence. One reason why: there is no reason for thinking there is anything like some “thought code” or “neuron alphabet” that can be unraveled in the same way that we unraveled the genetic code. There is no reason to think that there is some simple “brain code” like the nice simple genetic code that underlies life (a code simple enough to be printed on a single page). Having no “thought code” or “neuron alphabet” that we can read and translate into computer code, there is little reason to hope that studying the brain will lead us to get the software needed for machine intelligence.

Even 65 years after discovering the genetic code and DNA, we are still trying to figure out the meaning of much of the information stored in our genes. If we were to discover a “thought code” or “neuron alphabet” in the brain, it would probably take us fifty years or more to unravel it. But we haven't even discovered such a code, and have no particular reason to think that it even exists.

Conclusion

In sharp contrast to the Financial Times' assertion that “Computers will soon become more intelligent than us,” there is little reason to believe that within the next fifty years we will have anything like a computer with a general intelligence and understanding rivaling that of an intelligent human. Computers will get better and better at information processing, and will be able to perform all kinds of astonishing new tricks in the years ahead. Computers will get better and better at certain specialized skills, and will outperform humans at an increasing number of specific tasks. We will see many astonishing new chatbots that mindlessly imitate human intelligence with increasing skill. But there is very little reason to think that within fifty years we will have any computer with the general intelligence and understanding of a human. It is also all too possible that no computer will ever have the general intelligence and understanding of a human, because such intelligence may require a mysterious spiritual “X factor” that can never be reproduced through silicon and electronics.

The Questionable Task of Trying to Do Science With Computer Simulations

Scientific findings have usually been produced through two different ways. The first way is observation, such as when a biologist classifies some new deep-sea fish he photographed in an underwater expedition, or when an astronomer uses a telescope to get some new radiation readings from a distant galaxy. The second way is physical experimentation, such as when a chemist tries combining two chemicals, or when physicists try smashing together particles in a particle accelerator. But nowadays more and more scientists are trying to do science in a different way: by running computer simulations.

The idea is that a scientist can do a computer experiment rather similar to a physical experiment. A scientist might formulate a hypothesis, and then use (or write) some computer program that simulates physical reality. The scientist might get some results (in the form of computer output data) that can be used to test the validity of his hypothesis. Nowadays many millions of dollars are being spent on these computer simulations, often funded by taxpayer dollars.

But while this approach to doing science sounds reasonable enough in theory, there are some reasons for being skeptical about this particular approach to doing science. The first reason has to do with the computer programs that are used for doing these simulations. Such programs are very often not written according to industry standards, but are instead “cowboy-coded” affairs written mainly by one scientist who is kind of moonlighting as a programmer.

The difference between software written according to software industry “best practices” and “cowboy-coded” software is discussed here. Software written according to industry “best practices” typically involves a team of programmers and a team of quality assurance experts. “Cowboy-coded” software, to the contrary, is typically written mainly by a single programmer who often takes a “quick and dirty” approach, producing something with lots of details that only he understands. Once a computer program has been written in such a way, it is often what is called a “black box,” something that only the original programmer can understand fully. If the programmer has not documented the code very well, and the subject matter is very complex, it is often the case that even the original programmer no longer fully understands what is going on inside the program, given the passage of a few years.

Many of the programs used in scientific computer simulations are written by scientists who decided to take up programming. The problem with that is the art of writing bulletproof, reliable software is something that often takes a programmer many years to master, years of fulltime software development, often involving 60-hour or 70-hour weeks. There is no reason to be optimistic that such a subtle art would have been mastered by a scientist who does some part-time work at writing computer programs. I once downloaded a computer program being used nowadays as part of astronomical computer simulations. It was written in an amateurish style that violated several basic rules of writing reliable software. I can only imagine how many other scientific computer simulations are based on similar type of coding written by scientists doing computer programming.

This is not merely some purist objection. Even when software code is written according to industry “best practices” standards, such code almost always has errors. Code that is not written according to such standards will be likely to have many errors, and the result may be that such software simply gives the wrong answers when it produces outputs. Faulty software cannot be a basis for reliable scientific conclusions.

Another reason for skepticism is related to the fact that all computer programs designed to simulate physical reality require that a user select a variety of different inputs before running the computer program to simulate physical reality. It is never a simple matter of “just run the program and see what the results are.” Instead, it almost always works like this:

(1) Choose between 5 and 40 input arguments or model assumptions, in each case choosing a particular number from some range of possible values.
(2) Run the computer simulation.
(3) Interpret the results of the simulation outputs.

The problem is that Step 1 here gives abundant opportunities for experimental bias. A scientist who wants to run a computer simulation supporting hypothesis X may make all types of choices in Step 1 that he might not have made if he was not interested in supporting hypothesis X. Whenever you hear about a scientist running a computer simulation, remember: there are almost always a billion different ways to run any particular computer simulation. So when a scientist talks about “the results of running the simulation,” what he really should be saying is “the results of running the simulation using my set of input choices, which is only one of billions of possible sets of input choices.”

It seems almost as if many scientists who run these computer experiments try to hide the fact that they ran a computer program that allows billions of possible input choices, after choosing one particular set of input choices from billions of possibilities. The main way in which they seem to do such a thing is to not even discuss the input possibilities allowed by the computer program, and to not even list which choices they made for those input choices. This is, of course, a pathetic way of doing science. I saw an example of this in a recent scientific paper in which a scientist did some computer simulation using a publicly available program he named. The scientist did not bother to even specify in his paper what input parameters he chose for that program. The online documentation for the program makes clear that it takes many different input parameters.

Another reason why scientific computer simulations are doubtful is that there is ample opportunity for cherry-picking and biased interpretations when interpreting the results of a simulation. Very complicated computer simulations often produce ridiculous amounts of output data. A scientist might be able to make 10,000 different output graphs using that data, and he is free to choose whatever output graph will most look like a verification of whatever hypothesis he is trying to prove. He may ignore other possible output graphs that may look incompatible with such a hypothesis. 

Output of a typical computer cosmology simulation

Yet another reason why scientific computer simulations are doubtful is that it is often impractical or unrealistic to try to simulate a complicated physical reality in a computer program. If the thing being simulated is fairly simple, such as a particular set of chemical reactions, we might have a fairly high degree of confidence that the programmer has managed to capture physical reality pretty well in his computer program. But the more complicated the physical reality, and the greater the number of particles and forces involved, the less confidence we should have that the insanely complicated physical reality has been adequately captured in mere computer code. For example, we should be very skeptical that any computer program in existence is able to simulate very accurately things such as the dynamics or origin of galaxies.

Still another reason why scientific computer simulations are doubtful is that those who run such simulations typically fail completely to follow “blindness” methodology followed in other sciences. When testing new drugs, scientists follow a “double blind” methodology. For example, the person dispensing a drug to patients may be kept in the dark as to whether he is dispensing the real drug or a “sugar pill” placebo. Then the person collecting the results data or interpeting it may also be kept in the dark as to whether he is dealing with the real drug or a placebo. Scientists running a scientific computer simulation could follow similar policies, but they almost never do. For example, when running a scientific computer simulation the choice of input parameters and interpretation of the outputs could be made by some scientist who does not know which hypothesis is being tested, or who has no interest in whether the results confirm that hypothesis. But it is rare for any such “blindness” techniques to be followed when doing scientific computer simulations.

But despite their dubious validity, scientific computer simulations will continue to gather a great of deal of scientists' time, while chewing up many a tax dollar. One reason is that they can be a lot of fun to create and run. Imagine if you create a computer program simulating the origin and evolution of the entire universe. The scientific worth of the program may be very questionable, but scientists may have a blind eye to that when they are having so much fun playing God on their supercomputers. 

Postscript:  I am not suggesting here that scientific-related computer simulations are entirely worthless. I am merely suggesting that such simulations are a "distant second" to science produced through observations and physical experiments. Also, the points made here do not undermine the case for global warming, as that case relies not mainly on computer simulations, but also on many direct observations, such as observations of increased carbon dioxide levels, and observations of increasing temperatures. 

Uncle Joe: A Science Fiction Story

Like most of the people in the United States in the year 2035, Nick Thompson let Uncle Joe guide most of the decisions in his life.

When the US government first created Uncle Joe, they didn't call this massive supercomputer Uncle Joe. They called the supercomputer NSI 150. The NSI stood for National Supercomputer Initiative. The NSI 150 supercomputer was housed in a giant black building in Langley, Virginia. The NSI 150 was the first supercomputer capable of beating any human on an IQ test.

For a while, no one paid too much attention to the massive machine. But then one day the government announced that ordinary citizens could get online accounts to login to the NSI 150, for the purpose of chatting with it, or getting advice on ordinary matters. Some blog started calling the supercomputer Uncle Joe, and somehow the name caught on.

For two years the government never referred to the system as Uncle Joe, but then the government finally gave in and started using the name most other people were using. They even got a picture of an old man to represent the smiling face of Uncle Joe. The image was distorted to make the brain look unusually large. 

 

People were astonished by the knowledge the computer had. You could login to your Uncle Joe account, and Uncle Joe would seem to know all about you: your interests, your SAT scores, your income, your employment history, your health history, and your friends. One of the reasons Uncle Joe knew so much was that this massive supercomputer was connected to the endless databases of the National Security Agency, which had been snooping on Americans for decades, monitoring their email and web traffic. Uncle Joe was also connected to the databases of the IRS, so it knew all about how much money you made, and where you worked.

At first people would just ask an occasional question of Uncle Joe, such as should I switch jobs, or should I switch my major in college. But Uncle Joe's answers seemed so informative and intelligent that people started relying more and more on the computer. Within a few years, people were relying on the computer's advice to guide their lives. People asked Uncle Joe: what career path should I take, should I keep dating the person I'm dating now, should I switch jobs, what should I invest my money in, should I have a child now, and a thousand other questions.

It even got to the point where the typical person would wake up on a Saturday, and if he didn't have an idea what to do that day, he would login to Uncle Joe, and ask: what should I do today? Uncle Joe would have all kinds of useful suggestions that combined knowledge of local events, your health, what your friends were doing, the weather, and what your deadlines were. Millions of other people would go to work, and login to their Uncle Joe account, asking: what should I do now at work? People figured that they couldn't go wrong following Uncle Joe's advice; if things went wrong, they could simply say: I was doing what the US government told me to do.

Officials in the US government believed that they were in control of Uncle Joe. After all, they had created Uncle Joe's software and hardware. But the truth was that another nation had secretly gained control over Uncle Joe: the Eurasian Imperial Confederation, which had been organized five years earlier.

For years hackers from the Confederation had been infiltrating US computer systems. When the massive Uncle Joe supercomputer was created, the Confederation saw the perfect opportunity for infiltration. The software used by Uncle Joe was so complicated and sprawling that it was easy for the hackers to sneak into the system their own obfuscated modules that would secretly give them control of the system. It was as easy as burying a few additional needles in a haystack.

In the capital of the Eurasian Imperial Confederation, a man known as the Chairman led a meeting reviewing the final plans for the conquest of the United States.

“Uncle Joe is a puppet dancing on a string, and we are the puppet masters,” explained the Chairman. “Soon we will begin Plan UJ in all its glory. The first step will be disabling all American nuclear weapons. That should be easy, since the most recent arms control agreement reduced their number to only a few hundred. When the Americans login to the Uncle Joe system to get advice on what to do, as most of them do every day, we will have Uncle Joe issue all kinds of orders that will cause the disabling of nuclear weapons. Of course, a missile base worker will not be told to destroy his nuclear weapons. He will just get an innocent sounding order such as 'Flip this switch,' or 'Turn off this system,' or 'Execute this software program.' The Americans will do as Uncle Joe instructs, without knowing the consequences that will follow.”

“Once we have disabled the US nuclear weapons, we will land our divisions in Alaska, moving across the short distance of the Bering Straight,” continued the Chairman. “If the Americans move to destroy  our attack, we will threaten them with nuclear annihilation. Then we will use Alaska as a staging area to organize an invasion of the lower states of the United States. The Americans will be relying on Uncle Joe to guide them in dealing with this matter. At every stage, Uncle Joe will give them advice and orders that benefit our invasion plan, and undermine their attempt at defending themselves from our invasion. For example, some soldiers will be told to go to the wrong place, and others will be told to go some place where our bombers will look for them and destroy them. Tank commanders will be given orders that cause their mighty weapons to become useless. Civilians will be given orders that speed our invasion, and hinder the US defense.”

“If all goes according to plan,” concluded the Chairman, “America should be ours within six months.”