====== Undecidable problems ======

===== How many problems and how many ways of solving them? =====

Consider the following sets:
  * $math[\mathcal{M}] - the set of all Turing Machines
  * $math[\mathbb{H}\text{om}(\mathbb{N},\{0,1\})] - the set of all decision problems.

**Note**: in this lecture, we will interchangeably refer to problems as:
  * functions over **numbers**: $math[f:\mathbb{N}\rightarrow\{0,1\}]
  * functions over **words**: $math[f:\mathbb{\Sigma^*}\rightarrow\{0,1\}]

$prop[Cardinal of $math[\mathcal{M}]]
The set $math[\mathcal{M}] of Turing Machines is countably infinite.
$end

$proof 
Let $math[M\in\mathcal{M}] be a Turing Machine. The Proposition which states the existence of the Universal Turing Machine provides a unique identifier $math[enc(M)\in\Sigma^*] for $math[\mathcal{M}]. However, each Turing Machine may have its own alphabet $math[\Sigma]. We will see later that we can transform any Turing Machine running on a dedicated alphabet $math[\Sigma_M] into a Turing Machine running on a **binary alphabet** (with few extra symbols), which we henceforth simply call $math[\Sigma]. We consider this fact to be known in the proof.

Since $math[enc:\mathcal{M} \rightarrow \Sigma^*] is injective, we can view $math[\mathcal{M}] as a subset of $math[\Sigma^*]. Since $math[\Sigma^* \simeq \mathbb{N}] (and every infinite subset of a countable set is countable), it follows that $math[\mathcal{M}] is countable.
$end


$prop[Cardinal of $math[\mathbb{H}\text{om}(\mathbb{N},\{0,1\})]]
The set $math[\mathbb{H}\text{om}(\mathbb{N},\mathbb{N})] of functions $math[f : \mathbb{N} \rightarrow \mathbb{N}] is uncountably infinite.
$end

$proof
It is sufficient to show that $math[\mathbb{H}\text{om}(\mathbb{N}, \{0, 1\})] is uncountably infinite. We build a proof by contraposition. We assume $math[\mathbb{H}\text{om}(\mathbb{N}, \{0, 1\})] is countably infinite. Hence, each natural number $math[n \in \mathbb{N}] corresponds to a function $math[f_n \in \mathbb{H}\text{om}(\mathbb{N}, \{0, 1\})]. We build a matrix as follows: Columns describe functions $math[f_n : n \in \mathbb{N}]. Rows describe inputs $math[k \in \mathbb{N}]. Each matrix content $math[m_{i,j}] is the value of $math[f_j(i)] (hence, the expected output for input $math[i], from function $math[f_j]).

We illustrate the matrix building process below:

$math[\begin{array}{ll} \mbox{ } & f_0 & f_1 & f_2 & \ldots & f_n & \ldots \\ 0 & 1 & 1 & 0 & \ldots & 0 & \ldots \\ 1 & 0 & 1 & 1 & \ldots & 0 & \ldots \\ 2 & 1 & 0 & 1 & \ldots & 1 & \ldots \\ \ldots & \ldots & \ldots & \ldots & \ldots & \ldots & \ldots \\ n & 1 & 1 & 0 & \ldots & 1 & \ldots \\ \ldots & \ldots & \ldots & \ldots & \ldots & \ldots & \ldots \end{array} \\]

The value $math[m_{i,j}] have been filled out purely for the illustration. We now devise a problem $math[f^*] as follows:

$math[f^*(x)=\left\{\begin{array}{ll}1 & \mbox{iff } f_x(x)=0 \\ 0 & \mbox{iff } f_x(x)=1 \end{array} \right.]

Since $math[f^* \in Hom(\mathbb{N}, \{0, 1\})] it must also have a number assigned to it: $math[f^* = f_\alpha] for some $math[\alpha \in \mathbb{N}]. Then $math[f^*(\alpha) = 1] if $math[f_\alpha(\alpha) = 0]. But $math[f_\alpha(\alpha) = f^*(\alpha)]. Contradiction. On the other hand $math[f^*(\alpha) = 0] if $math[f_\alpha(\alpha) = 1]. As before we obtain a contradiction.
$end

Our two propositions certify that:
  * **there are infinitely many problems** than **Turing Machines**, in other words:
  * **there are infinitely more problems** than ways of solving them.

Hence:
  * **there exist problems which cannot be solved by the Turing Machine**

===== Termination and unsolvability =====

In what follows, we search for problems which cannot be solved by Turing Machines. To this end, we refine and more rigorously state the concept of **solvability**.

We start by observing that Turing Machines (like conventional programs) may never halt. We write $math[M(w) = \perp] to designate that $math[M] loops on input $math[w]. 

$def[Decision, acceptance]
Let $math[M] be a Turing Machine and $math[f \in Hom(\mathbb{N}, \{0, 1\})].
  * $math[M] **decides** $math[f] iff for all $math[n \in \mathbb{N}]: $math[M(w) = 1] whenever $math[f(n^w) = 1] and $math[M(w) = 0] whenever $math[f(n^w) = 0].
  * $math[M] **accepts** $math[f] iff for all $math[n \in \mathbb{N}]: $math[M(w) = 1] iff $math[f(n^w) = 1], and $math[M(w) = \perp] iff $math[f(n)=0].
$end

Note that, in contrast with acceptance, decision is, intuitively, a stronger means of computing a function (i.e. solving a problem). In the latter case, the ''TM'' at hand can provide both a //yes// and a //no// answer to any problem instance, while in the former, the ''TM'' can only provide an answer of //yes//. If the answer to the problem instance at hand is //no//, the ''TM'' will not halt.

Based on the two types of problem solving, we can classify problems (functions) as follows:

$def[R and RE]
Let $math[f \in Hom(\mathbb{N}, \{0, 1\})] be a decision problem.

  * $math[f] is **recursive** (decidable) iff there exists a ''TM'' $math[M] which decides $math[f]. The set of recursive functions is $math[R = \{f \in Hom(\mathbb{N}, \{0,1\}) \mid \mbox{f is recursive} \}]
  * $math[f] is **recursively enumerable** (//semi-decidable//) iff there exists a ''TM'' $math[M] which accepts $math[f]. The set of recursive-enumerable function is $math[RE = \{f \in Hom(\mathbb{N},\{0,1\}) \mid \mbox{f is recursively-enumerable} \}]
$end

==== The halting problem ====

For a moment, let us turn our attention to the Universal Turing Machine. Its existence stimulates the following question:

  * is it possible to construct a //Halting Turing Machine//, which, just like the UTM, receives a TM and word as input, and establishes if **the TM terminates on input M?**

In other words, we enquire if it is possible to solve the following problem:

$math[f_h(enc(M)\_w)= \left\{ \begin{array}{ll} 1 & \mbox{iff } M(w) \mbox{halts} \\ 0 & \mbox{iff } M(w) = \perp \end{array} \right.]

The first, rather straightforward observation is that $math[f_h \in RE].

$prop[Halting is semi-decidable]
$math[f_h \in RE]
$end

$proof
We build a Turing Machine $math[M_h] which accepts $math[f_h]. Essentially, $math[M_h] is the Universal Turing Machine. $math[M_h(enc(M)\_w)] simulates $math[M], and if $math[M(w)] halts, then it outputs 1. If $math[M(w)] does not halt, $math[M_h(enc(M)\_w) = \perp].
$end
 
Second, we investigate the membership $math[f_h\in R]:

$prop[Halting is not decidable]
$math[f_h \notin R].
$end

$proof
Assume $math[f_h \in R] and denote by $math[M_h] the Turing Machine which decides $math[f_h]. We build the Turing Machine $math[D], as follows:

$math[D(enc(M)) = \left\{ \begin{array}{ll} \perp & \mbox{iff } M_h(enc(M)\_enc(M)) = 1 \\ 1 & \mbox{iff } M_h(enc(M)\_enc(M)) = 0 \end{array} \right.]

The existence of the Universal Turing Machine guarantees that $math[D] can indeed be built, since $math[D] simulates $math[M_h]. We note that $math[M_h(enc(M)\_enc(M))] decides if the $math[TM] $math[M] halts with "//itself//" as input (namely $math[enc(M)]).

Assume $math[D(enc(D)) = 1]. Hence $math[M_h(enc(D),enc(D)) = 0], that is, machine $math[D] does not halt for input $math[enc(D)]. Hence $math[D(enc(D)) = \perp]. Contradiction.

Assume $math[D(enc(D)) = \perp]. Hence $math[M_h(enc(D),enc(D)) = 1], and thus $math[D(enc(D))] halts. Contradiction.
$end

We note that the construction of $math[D] mimics the technique which we applied for the proof that the set of decision problems is not countable, which is called //diagonalization//.

We can use our results on the halting problem, to make a more general statement:

$justprop
$math[R \subsetneq RE]
$end

$proof
$math[R \subseteq RE] is straightforward from the definitions of $math[R] and $math[RE]. Let $math[f \in R], and $math[M_f] be the $math[TM] which decides $math[M_f]. We build the $math[TM] $math[M^\prime] such that $math[M^\prime(w)=1] iff $math[M_f(w)=1] and $math[M^\prime(w)=\perp] iff $math[M_f(w)=0]. $math[M^\prime] simulates $math[M] but enters into an infinite loop whenever $math[M_f(w)=0]. $math[M^\prime] accepts $math[f] hence $math[f \in RE].

$math[R \neq RE] has already been shown using the halting problem as (counter)example. $math[f_h \in RE] but $math[f_h \notin R].

$end


===== A general technique for investigating decidability =====

How can we prove that other problems such as PCP are undecidable? The argument used to show halting undecidability seems difficult to generalise for other problems (such as PCP).

Fortunately, general schemes for proving that problems are semi-decidable or undecidable are known. These schemes rely on:
  * the concept of **problem transformation**: i.e. deciding a problem $math[f_1] by adapting an algorithm for a problem $math[f_2], which is assumed to exist;
  * a **contraposition argument**: if problem $math[f_1] is **known** to be undecidable, then our above assumption leads to impossibility, which shows that our assumption (the existence of an algorithm which decides $math[f_2]) is false.

We illustrate this scheme via an example.

==== Halting on all inputs ====

Consider the following problem:

$math[f_{all}(enc(M)) = \left\{ \begin{array}{ll} 1 & \mbox{iff } M \mbox{ halts for all inputs} \\ 0 & \mbox{otherwise} \end{array} \right.]

We will show that $math[f_{all}] is not decidable.

=== Step 1 (transformation) ===

For now, we imagine that there exists a Turing Machine which decides $math[f_{all}]. We shall call it $math[M_{all}]. We use $math[M_{all}] to solve the **halting problem**.

Let $math[M] be a Turing Machine and $math[w \in \Sigma^*] be a word. 
  * We build the Turing Machine $math[\Pi_{M,w}(\omega) = ]//"Replace $math[\omega] by $math[w] and then simulate $math[M(w)]"//. 
  * The construction of $math[\Pi_{M,w}] depends on $math[M] and $math[w], as suggested by the indices. 
  * Note that building $math[\Pi_{M,w}] can be easily derived from the Universal Turing Machine.
  * Now, the halting problem can be decided as follows:
    * let $math[M] and $math[w] be the **input** of the halting problem
    * build $math[\Pi_{M,w}] (//which is a straightforward programming task if $math[M] and $math[w] are known//)
    * run $math[M_{all}] with input $math[enc(\Pi_{M,w})], and return the output.
    * it is easy to see that:
      * $math[(\forall \omega \in \Sigma^* : \Pi_{M,w}(\omega) \mbox{ halts})] iff $math[M(w)] halts.
 
=== Step 2 (contraposition) ===

The first step has constructed an algorithm for solving the halting problem, by relying on the assumption that $math[f_{all}] is **decidable** hence on the existence of $math[M_{all}] which terminates on all inputs.

The relationship between $math[f_{all}] and $math[f_h] is called **Turing-reducibility**, and is written:
$math[f_h \leq_T f_{all}]

This relation indicates a strong bond between the two problems which can be interpreted as //$math[f_{all}] is at least as hard as $math[f_{h}]//, in the sense that if we can decide $math[f_{all}] using a Turing Machine, then it is trivial to decide $math[f_h].

More formally:

$def[Turing-reducibility]
//Let $math[f_A, f_B \in Hom(\mathbb{N}, \{0, 1\})]. We say $math[f_A] is// Turing-reducible //to $math[f_B], and write $math[f_A \leq_T f_B] iff there exists a// decidable transformation// $math[T \in Hom(\mathbb{N},\mathbb{N})] such that $math[f_A(n) = 1] iff $math[f_B(T(n)) = 1].//
$end

Having shown **Turing-Reducibility** between $math[f_{all}] and the halting problem, the contraposition argument follows naturally:
  * we have **assumed** $math[f_{all}] is decidable
  * we have arrived at the conclusion that $math[f_{h}] is decidable, which is **false**
  * hence our **assumption** was false.

$justprop
$math[f_{all}] is not decidable.
$end

=== Recap ===

To prove a problem $math[f_1] is not decidable:
  * select a problem $math[f_2] which is **known to be not decidable**
  * show that $math[f_2 \leq_T f_1], hence that $math[f_1] is //at least as hard as// $math[f_2]
  * apply the contraposition argument

=== About transformations ===

When building a transformation $math[T], keep in mind that:
  * the transformation procedure should be itself **decidable**, i.e. implementable as a Turing Machine **which halts on all inputs**. Consider a transformation:
      * whose inputs are two Turing Machines $math[enc(M_1)] and $math[enc(M_2)], as well as a word.
      * whose outputs is $math[enc(M_1)] if $math[M_1(w)] halts, and $math[M_2] - otherwise.
      * this transformation is **not decidable**! In contrast, building $math[\Pi_{M,w}] is decidable.
  * the transformation procedure should be **correct**: our constructed Turing Machine should indeed solve the problem at hand. 


$justexercise
Show that $math[f_{all}] is recursively-enumerable.
$end



===== More undecidable problems =====

==== Halting on 111 ====

$math[f_{111}enc(M)) = \left\{ \begin{array}{ll} 1 & \mbox{iff } M(111) \mbox{ halts} \\ 0 & \mbox{otherwise} \end{array} \right.]

{{##
We show $math[f_h \leq_T f_{111}]. Assume $math[M_{111}] decides $math[f_{111}]. Given a Turing Machine $math[M] and word $math[w], we construct the machine:

$math[\Pi_{M,w}(\omega) = \mbox{ if } \omega = 111 \mbox{ then } M(w) \mbox{ else loop }]

We observe that:
  * the transformation from $math[enc(M)\_w] to $math[enc(\Pi_{M,w})] is decidable since it involves adding precisely three states to $math[M]: these states check the input $math[\omega], and if it is $math[111] - replace it with $math[w] and run $math[M]; 
  * $math[M_{111}(\Pi_{M,w}) = 1 \mbox{ iff } M(w) \mbox{halts}]. The reduction is complete. $math[f_{111} \notin R].

##}}

==== Halting on some input ====

We define:

$math[f_{any}(enc(M)) = \left\{ \begin{array}{ll} 1 & \mbox{iff } M(w) \mbox{ halts for some } w \in \Sigma^* \\ 0 & \mbox{otherwise} \end{array} \right.]

{{## 
We show $math[f_{111} \leq_T f_{any}]. We assume $math[f_{any}] is decided by $math[M_{any}]. We construct:

$math[\Pi_M(\omega) = \mbox{ Replace } \omega \mbox{ by } 111 \mbox{ and } M(111)]

Now, $math[M_{any}(enc(\Pi_M)) = 1 \mbox{ iff } M(111) = 1], hence we can use $math[M_{any}] to build a machine which decides $math[f_{111}]. Contradiction $math[f_{any} \notin R].

##}}

==== Machine halt equivalence ====

We define:

$math[f_{eq}(enc(M_1)\_enc(M_2)) = \left\{ \begin{array}{ll} 1 & \mbox{for all } w \in \Sigma^* \mbox{ : } M_1(w) \mbox{ halts iff } M_2(w) \mbox{ halts} \\ 0 & \mbox{otherwise} \end{array} \right.]

{{##

We show $math[f_{all}\leq_T f_{eq}]. Let $math[M_{triv}] be a one-state Turing Machine which halts on every input, and $math[M_{eq}] be the Turing Machine which decides $math[f_{eq}]. Then $math[M_{eq}(enc(M)\_enc(M_{triv})) = 1 \mbox{ iff } M] halts on all inputs. We have shown we can use $math[M_{eq}] in order to build a machine which decides $math[f_{all}]. Contradiction. $math[f_{eq} \notin R].

##}}