Does the the undecidability of the Halting Problem eliminate the possibility of 'Hard AI'?

Question

I'm defining 'Hard AI' as a human-equivalent intelligent machine, or beyond that. Contrast with 'Soft AI' the type of software that runs on your email filter for example.

I've been chewing on this problem for a little while now, and would like some feedback on my logic. This is the only place I could think to put it, so please feel free to move/remove if this is not appropriate.

My argument hinges on the definition of 'human-equivalent intelligence' as 'capable of solving any given instance of the Halting Problem'. This is something no machine can ever do - while they may be able to solve for sub-sets of the Halting Problem, they can never accept an arbitrary instance of it (ie, specific inputs I for a specific program P) and produce an answer.

Humans, however, do this all the time! The entire profession of programming revolves around fixing incarnations of the Halting Problem that pop up in one's own codebase. There are problem-tracking instances full of real-world examples of the Halting Problem - "Subsystem Z stops responding after receiving A with B options", etc. These examples are fixed by humans, and the problem goes away. Sometimes instructions are written for the machine to verify that the problem hasn't returned, taking advantage of the recognizability of solutions to the Halting Problem.

Since any 'Hard AI' would need to be just as capable as humans in any intellectual pursuit, and we KNOW that a machine that solves the Halting Problem cannot exist, we must conclude that there can never be human-intelligent machines.

Answer 1

You're understanding of "solve all instances of the Halting Problem" is flawed. All instances means ALL instances. Every single one, ever.

For example, no human knows if a Turing Machine verifying the Collatz Conjecture will halt. Bam, right there, your premise that humans can solve "all" incarnations of the Halting problem is flawed.

There is no reason to think that a sufficiently good AI wouldn't be able to solve the same, or more, instances of the Halting Problem that humans can.

The undecidability of the Halting problem means that there is no algorithm which solves the halting problem. That means there is no algorithm Humans can use, and no algorithm machines can use. The proof doesn't rely on what the algorithm runs on.

Another way to look at it is that the halting problem, and other undecidable problems, essentially boil down to searching an infinite space. The inability of humans to time travel and examine an infinite number of instances doesn't negate our intelligence, so there's no reason it would for machines either.

Answer 2

Here's a thought experiment: the mathematical framework with which we usually work is called ZFC, and it has 9 axioms. Given a program $M$ and input $x$, a Turing Machine could iterate over all proofs of two steps, then all proofs of three steps, and so forth, until it finds a proof of whether $M$ halts on $x$. If there exists a proof, then this Turing Machine will find it. As you point out, this is different from the statement, 'if $M$ halts, then this machine will prove it,' because the halting problem is undecidable.

For your AI premise, let's suppose that for some $M,x$, there is no proof, but there is a human who says that $M$ never halts on $x$. Good for him, but now what? If he is right, nobody can verify that he is right by running the machine until it halts, and we cannot find a proof, either. This person could not satisfactorily explain his reasoning to others because to do so would be to prove it formally, which we assumed was not possible (in ZFC). What if two humans conjecture differently? Nobody could distinguish between a human who was always right about such machines, and one who was always wrong. This is important, because humans do not have this skill innately, it must be learned. But how do we learn if there are no examples? And if there are examples, why can't a computer be programmed to learn in the same way?

In summary, there cannot be a human who both (a) gets the halting problem right on undecidable instances and (b) can give a satisfactorily detailed explanation of why he is right.

This is a contrived attempt at a proof, but there is a better way: if humans can answer the halting problem, then (by simple reduction) they can also tell in how many steps they will halt. For a simple introduction to just how mind-numbingly small our knowledge to this extend is, see the wikipedia page for Busy Beaver. Currently, we do not even know how for long the longest-running 3-state 3-symbol Turing Machine runs.

The instance you mention of 'subsystem Z stops responding with input X' isn't always a clever solution by humans. Usually, programmers empirically see how long a program should take before it resumes responding and set a simple timer. So not only is it not a proof, it may produce false positives! My computer sometimes does this when starting Kerbal Space Program, for example, because that program loads a gigabyte of data from disk before it starts. Some programs can be proved to halt, and most programs written in a sufficiently high-level programming language have very simple proofs because they mostly just iterate over finite data structures, do an operation a given number of times or do other things that obviously halt, because those are the kinds of programs that programmers usually write.