Undecidable problems

Consider the following sets:

• $ \mathcal{M}$ - the set of all Turing Machines
• $ \mathbb{H}\text{om}(\mathbb{N},\{0,1\})$ - the set of all decision problems.

Note: in this lecture, we will interchangeably refer to problems as:

• functions over numbers: $ f:\mathbb{N}\rightarrow\{0,1\}$
• functions over words: $ f:\mathbb{\Sigma^*}\rightarrow\{0,1\}$

Proposition (Cardinal of $ \mathcal{M}$ ):

The set $ \mathcal{M}$ of Turing Machines is countably infinite.

Proof:

Let $ M\in\mathcal{M}$ be a Turing Machine. The Proposition which states the existence of the Universal Turing Machine provides a unique identifier $ enc(M)\in\Sigma^*$ for $ \mathcal{M}$ . However, each Turing Machine may have its own alphabet $ \Sigma$ . We will see later that we can transform any Turing Machine running on a dedicated alphabet $ \Sigma_M$ into a Turing Machine running on a binary alphabet (with few extra symbols), which we henceforth simply call $ \Sigma$ . We consider this fact to be known in the proof.

Since $ enc:\mathcal{M} \rightarrow \Sigma^*$ is injective, we can view $ \mathcal{M}$ as a subset of $ \Sigma^*$ . Since $ \Sigma^* \simeq \mathbb{N}$ (and every infinite subset of a countable set is countable), it follows that $ \mathcal{M}$ is countable.

Proposition (Cardinal of $ \mathbb{H}\text{om}(\mathbb{N},\{0,1\})$ ):

The set $ \mathbb{H}\text{om}(\mathbb{N},\mathbb{N})$ of functions $ f : \mathbb{N} \rightarrow \mathbb{N}$ is uncountably infinite.

Proof:

It is sufficient to show that $ \mathbb{H}\text{om}(\mathbb{N}, \{0, 1\})$ is uncountably infinite. We build a proof by contraposition. We assume $ \mathbb{H}\text{om}(\mathbb{N}, \{0, 1\})$ is countably infinite. Hence, each natural number $ n \in \mathbb{N}$ corresponds to a function $ f_n \in \mathbb{H}\text{om}(\mathbb{N}, \{0, 1\})$ . We build a matrix as follows: Columns describe functions $ f_n : n \in \mathbb{N}$ . Rows describe inputs $ k \in \mathbb{N}$ . Each matrix content $ m_{i,j}$ is the value of $ f_j(i)$ (hence, the expected output for input $ i$ , from function $ f_j$ ).

We illustrate the matrix building process below:

$ \begin{array}{ll} \mbox{ } & f_0 & f_1 & f_2 & \ldots & f_n & \ldots \\ 0 & 1 & 1 & 0 & \ldots & 0 & \ldots \\ 1 & 0 & 1 & 1 & \ldots & 0 & \ldots \\ 2 & 1 & 0 & 1 & \ldots & 1 & \ldots \\ \ldots & \ldots & \ldots & \ldots & \ldots & \ldots & \ldots \\ n & 1 & 1 & 0 & \ldots & 1 & \ldots \\ \ldots & \ldots & \ldots & \ldots & \ldots & \ldots & \ldots \end{array} \\$

The value $ m_{i,j}$ have been filled out purely for the illustration. We now devise a problem $ f^*$ as follows:

$ f^*(x)=\left\{\begin{array}{ll}1 & \mbox{iff } f_x(x)=0 \\ 0 & \mbox{iff } f_x(x)=1 \end{array} \right.$

Since $ f^* \in Hom(\mathbb{N}, \{0, 1\})$ it must also have a number assigned to it: $ f^* = f_\alpha$ for some $ \alpha \in \mathbb{N}$ . Then $ f^*(\alpha) = 1$ if $ f_\alpha(\alpha) = 0$ . But $ f_\alpha(\alpha) = f^*(\alpha)$ . Contradiction. On the other hand $ f^*(\alpha) = 0$ if $ f_\alpha(\alpha) = 1$ . As before we obtain a contradiction.

Our two propositions certify that:

• there are infinitely many problems than Turing Machines, in other words:
• there are infinitely more problems than ways of solving them.

Hence:

• there exist problems which cannot be solved by the Turing Machine

In what follows, we search for problems which cannot be solved by Turing Machines. To this end, we refine and more rigorously state the concept of solvability.

We start by observing that Turing Machines (like conventional programs) may never halt. We write $ M(w) = \perp$ to designate that $ M$ loops on input $ w$ .

Definition (Decision, acceptance):

Let $ M$ be a Turing Machine and $ f \in Hom(\mathbb{N}, \{0, 1\})$ .

• $ M$ decides $ f$ iff for all $ n \in \mathbb{N}$ : $ M(w) = 1$ whenever $ f(n^w) = 1$ and $ M(w) = 0$ whenever $ f(n^w) = 0$ .
• $ M$ accepts $ f$ iff for all $ n \in \mathbb{N}$ : $ M(w) = 1$ iff $ f(n^w) = 1$ , and $ M(w) = \perp$ iff $ f(n)=0$ .

Note that, in contrast with acceptance, decision is, intuitively, a stronger means of computing a function (i.e. solving a problem). In the latter case, the TM at hand can provide both a yes and a no answer to any problem instance, while in the former, the TM can only provide an answer of yes. If the answer to the problem instance at hand is no, the TM will not halt.

Based on the two types of problem solving, we can classify problems (functions) as follows:

Definition (R and RE):

Let $ f \in Hom(\mathbb{N}, \{0, 1\})$ be a decision problem.

• $ f$ is recursive (decidable) iff there exists a TM $ M$ which decides $ f$ . The set of recursive functions is $ R = \{f \in Hom(\mathbb{N}, \{0,1\}) \mid \mbox{f is recursive} \}$
• $ f$ is recursively enumerable (semi-decidable) iff there exists a TM $ M$ which accepts $ f$ . The set of recursive-enumerable function is $ RE = \{f \in Hom(\mathbb{N},\{0,1\}) \mid \mbox{f is recursively-enumerable} \}$

For a moment, let us turn our attention to the Universal Turing Machine. Its existence stimulates the following question:

• is it possible to construct a Halting Turing Machine, which, just like the UTM, receives a TM and word as input, and establishes if the TM terminates on input M?

In other words, we enquire if it is possible to solve the following problem:

$ f_h(enc(M)\_w)= \left\{ \begin{array}{ll} 1 & \mbox{iff } M(w) \mbox{halts} \\ 0 & \mbox{iff } M(w) = \perp \end{array} \right.$

The first, rather straightforward observation is that $ f_h \in RE$ .

Proposition (Halting is semi-decidable):

$ f_h \in RE$

Proof:

We build a Turing Machine $ M_h$ which accepts $ f_h$ . Essentially, $ M_h$ is the Universal Turing Machine. $ M_h(enc(M)\_w)$ simulates $ M$ , and if $ M(w)$ halts, then it outputs 1. If $ M(w)$ does not halt, $ M_h(enc(M)\_w) = \perp$ .

Second, we investigate the membership $ f_h\in R$ :

Proposition (Halting is not decidable):

$ f_h \notin R$ .

Proof:

Assume $ f_h \in R$ and denote by $ M_h$ the Turing Machine which decides $ f_h$ . We build the Turing Machine $ D$ , as follows:

$ D(enc(M)) = \left\{ \begin{array}{ll} \perp & \mbox{iff } M_h(enc(M)\_enc(M)) = 1 \\ 1 & \mbox{iff } M_h(enc(M)\_enc(M)) = 0 \end{array} \right.$

The existence of the Universal Turing Machine guarantees that $ D$ can indeed be built, since $ D$ simulates $ M_h$ . We note that $ M_h(enc(M)\_enc(M))$ decides if the $ TM$ $ M$ halts with “itself” as input (namely $ enc(M)$ ).

Assume $ D(enc(D)) = 1$ . Hence $ M_h(enc(D),enc(D)) = 0$ , that is, machine $ D$ does not halt for input $ enc(D)$ . Hence $ D(enc(D)) = \perp$ . Contradiction.

Assume $ D(enc(D)) = \perp$ . Hence $ M_h(enc(D),enc(D)) = 1$ , and thus $ D(enc(D))$ halts. Contradiction.

We note that the construction of $ D$ mimics the technique which we applied for the proof that the set of decision problems is not countable, which is called diagonalization.

We can use our results on the halting problem, to make a more general statement:

Proposition:

$ R \subsetneq RE$

Proof:

$ R \subseteq RE$ is straightforward from the definitions of $ R$ and $ RE$ . Let $ f \in R$ , and $ M_f$ be the $ TM$ which decides $ M_f$ . We build the $ TM$ $ M^\prime$ such that $ M^\prime(w)=1$ iff $ M_f(w)=1$ and $ M^\prime(w)=\perp$ iff $ M_f(w)=0$ . $ M^\prime$ simulates $ M$ but enters into an infinite loop whenever $ M_f(w)=0$ . $ M^\prime$ accepts $ f$ hence $ f \in RE$ .

$ R \neq RE$ has already been shown using the halting problem as (counter)example. $ f_h \in RE$ but $ f_h \notin R$ .

How can we prove that other problems such as PCP are undecidable? The argument used to show halting undecidability seems difficult to generalise for other problems (such as PCP).

Fortunately, general schemes for proving that problems are semi-decidable or undecidable are known. These schemes rely on:

• the concept of problem transformation: i.e. deciding a problem $ f_1$ by adapting an algorithm for a problem $ f_2$ , which is assumed to exist;
• a contraposition argument: if problem $ f_1$ is known to be undecidable, then our above assumption leads to impossibility, which shows that our assumption (the existence of an algorithm which decides $ f_2$ ) is false.

We illustrate this scheme via an example.

Consider the following problem:

$ f_{all}(enc(M)) = \left\{ \begin{array}{ll} 1 & \mbox{iff } M \mbox{ halts for all inputs} \\ 0 & \mbox{otherwise} \end{array} \right.$

We will show that $ f_{all}$ is not decidable.

For now, we imagine that there exists a Turing Machine which decides $ f_{all}$ . We shall call it $ M_{all}$ . We use $ M_{all}$ to solve the halting problem.

Let $ M$ be a Turing Machine and $ w \in \Sigma^*$ be a word.

• We build the Turing Machine $ \Pi_{M,w}(\omega) = $ “Replace $ \omega$ by $ w$ and then simulate $ M(w)$ ”.
• The construction of $ \Pi_{M,w}$ depends on $ M$ and $ w$ , as suggested by the indices.
• Note that building $ \Pi_{M,w}$ can be easily derived from the Universal Turing Machine.
• Now, the halting problem can be decided as follows:
  • let $ M$ and $ w$ be the input of the halting problem
  • build $ \Pi_{M,w}$ (which is a straightforward programming task if $ M$ and $ w$ are known)
  • run $ M_{all}$ with input $ enc(\Pi_{M,w})$ , and return the output.
  • it is easy to see that:
    • $ (\forall \omega \in \Sigma^* : \Pi_{M,w}(\omega) \mbox{ halts})$ iff $ M(w)$ halts.

The first step has constructed an algorithm for solving the halting problem, by relying on the assumption that $ f_{all}$ is decidable hence on the existence of $ M_{all}$ which terminates on all inputs.

The relationship between $ f_{all}$ and $ f_h$ is called Turing-reducibility, and is written: $ f_h \leq_T f_{all}$

This relation indicates a strong bond between the two problems which can be interpreted as $ f_{all}$ is at least as hard as $ f_{h}$ , in the sense that if we can decide $ f_{all}$ using a Turing Machine, then it is trivial to decide $ f_h$ .

More formally:

Definition (Turing-reducibility):

Let $ f_A, f_B \in Hom(\mathbb{N}, \{0, 1\})$ . We say $ f_A$ is Turing-reducible to $ f_B$ , and write $ f_A \leq_T f_B$ iff there exists a decidable transformation $ T \in Hom(\mathbb{N},\mathbb{N})$ such that $ f_A(n) = 1$ iff $ f_B(T(n)) = 1$ .

Having shown Turing-Reducibility between $ f_{all}$ and the halting problem, the contraposition argument follows naturally:

• we have assumed $ f_{all}$ is decidable
• we have arrived at the conclusion that $ f_{h}$ is decidable, which is false
• hence our assumption was false.

Proposition:

$ f_{all}$ is not decidable.

To prove a problem $ f_1$ is not decidable:

• select a problem $ f_2$ which is known to be not decidable
• show that $ f_2 \leq_T f_1$ , hence that $ f_1$ is at least as hard as $ f_2$
• apply the contraposition argument

When building a transformation $ T$ , keep in mind that:

• the transformation procedure should be itself decidable, i.e. implementable as a Turing Machine which halts on all inputs. Consider a transformation:
  • whose inputs are two Turing Machines $ enc(M_1)$ and $ enc(M_2)$ , as well as a word.
  • whose outputs is $ enc(M_1)$ if $ M_1(w)$ halts, and $ M_2$ - otherwise.
  • this transformation is not decidable! In contrast, building $ \Pi_{M,w}$ is decidable.
• the transformation procedure should be correct: our constructed Turing Machine should indeed solve the problem at hand.

Show that $ f_{all}$ is recursively-enumerable. </blockquote>

$ f_{111}enc(M)) = \left\{ \begin{array}{ll} 1 & \mbox{iff } M(111) \mbox{ halts} \\ 0 & \mbox{otherwise} \end{array} \right.$

We define:

$ f_{any}(enc(M)) = \left\{ \begin{array}{ll} 1 & \mbox{iff } M(w) \mbox{ halts for some } w \in \Sigma^* \\ 0 & \mbox{otherwise} \end{array} \right.$

We define:

$ f_{eq}(enc(M_1)\_enc(M_2)) = \left\{ \begin{array}{ll} 1 & \mbox{for all } w \in \Sigma^* \mbox{ : } M_1(w) \mbox{ halts iff } M_2(w) \mbox{ halts} \\ 0 & \mbox{otherwise} \end{array} \right.$