====== Beyond RE ======

===== Recap =====

In the last lecture:
  * we have introduced the classes $math[R] and $math[RE], which classify **decision problems** into:
    * //decidable// (which can be solved by algorithms which always terminate).
    * //semi-decidable// (which can be solved by algorithms which terminate only if the answer is //yes//).


  * we have introduced a technique for proving that a problem $math[f] **is not decidable**:
    * find another problem $math[f^*] which **is known NOT to be decidable**;
    * prove that $math[f^* \leq_T f] (**Turing-reducibility** - i.e. $math[f] is **at least as hard as** $math[f^*]) 

In this lecture:
  * we will examine some problems which are **outside the class** $math[RE]
  * we will use the concept of **Turing-reducibility** to prove that some problems are not $math[RE].

===== The complement of a problem =====

$def[Complement of a problem]
//Let $math[f \in Hom(\mathbb{N},\{0,1\})]. We denote by $math[\overline{f}] the problem://

$math[\overline{f}(n) = \left\{ \begin{array}{ll} 1 & \mbox{iff } f(n)=0\\0 & \mbox{iff } f(n)=1 \end{array} \right.]

//We call $math[\overline{f}] the// complement //of $math[f].//
$end

For instance, the complement of $math[f_h] is the problem which asks if a Turing Machine $math[M] does not halt for input $math[w]. We also note that $math[\overline{\overline{f}} = f] - i.e. the **double complement** of a problem is the problem itself.

We define the class:

$math[coRE = \{f \in Hom(\mathbb{N}, \{0; 1\}) \mid \overline{f} \in RE\}]

$math[coRE] contains the set of all problems whose complement is in $math[RE].

We study the relationship between $math[coRE] and the classes $math[R] and $math[RE].

$justprop
$math[RE \cap coRE = R].
$end

$proof
We first show $math[RE \cap coRE \subseteq R]. Assume $math[f \in RE \cap coRE]. Hence, there exists a Turing Machine $math[M] which accepts $math[f] and a Turing Machine $math[\overline{M}] which accepts $math[\overline{f}]. We build the Turing Machine:

$math[M^*(w) = \mbox{ for } i \in \mathbb{N} \left\{ \begin{array}{ll} \mbox{ run } M(w) \mbox{ for } i \mbox{ steps.}\\ \mbox{ If } M(w) = 1 \mbox{, return } 1 \mbox{. Otherwise:}\\ \mbox{ run } \overline{M}(w) \mbox{ for } i \mbox{ steps.}\\ \mbox{ If } \overline{M}(w) = 1 \mbox{, return } 0 \mbox{.} \end{array} \right.]

Since $math[M] and $math[\overline{M}] will always halt when the expected result is 1, they can be used together to decide $math[f]. Hence $math[f \in R].

Show $math[R \subseteq RE \cap coRE] as an exercise.
$end

$justprop
$math[f \in R \mbox{ iff } \overline{f} \in R \mbox{.}]
$end

$proof
The proposition follows immediately since the Turing Machine which decides $math[f] can be used to decide $math[\overline{f}], by simply switching its output from $math[0] to $math[1] and $math[1] to $math[0]. The same holds for both directions.
$end

===== Some problems which are (strictly) in coRE =====

First of all, the **complement of the halting problem** is in $math[coRE]: this follows immediately from the complement definition and the above propositions.

All problems for which we can prove $math[RE] membership are problems in $math[coRE].

===== Outside coRE ====

==== Halting on all inputs ====

$math[f_{all}(w) = \left\{ \begin{array}{ll} 1 & \mbox{iff } M(w) \mbox{ halts for all } w \in \Sigma^* \\ 0 & \mbox{otherwise} \end{array} \right.]

We have already seen that $math[f_{all} \not\in R], and we suspect that $math[f_{all}\not\in RE]. The intuitive reason is that:
  * $math[f_{all}] implies a doubly-infinite exploration: 
    * for an **infinite number** of inputs $math[w]
    * run $math[M(w)] for $math[k\in\mathbb{N}] of steps, and see if $math[M] reaches a final state. There are **infinitely many steps**. 

We validate our intuition by proving $math[f_{all}\not\in RE]. We apply **precisely the same technique** as in the $math[f_{all}\not\in R] case. We choose $math[\overline{f_h}\not\in RE], and we prove:
  * $math[\overline{f_h}\leq_T f_{all}]

We must find a transformation from the input $math[M,w] of the halting complement to the input $math[M'] of $math[f_{all}]. We build $math[M'] as follows:
  * $math[M'(\omega) = ]
    * run the simulation $math[M(w)] for a number $math[|\omega|] of steps, then stop the simulation.
    * if $math[M] has reached a final state, enter a looping state;
    * otherwise, halt.

We prove that the transformation is correct. Direction $math[\implies]:
  * suppose $math[M(w)] loops. Then, for any number $math[k] of steps, the $math[k]-step simulation $math[M(w)] will not bring $math[M] in a final state. Hence, for all $math[\omega], $math[M'] halts.
Direction $math[\impliedby]:
  * suppose $math[M'] halts for all inputs. Therefore, for all $math[k\in\mathbb{N}], the $math[k]-step simulation of $math[M(w)] does not bring $math[M] to a final state. Hence $math[M(w)] loops.

We now ask a final question: 
  * **is it the case that** $math[f_{all}\not\in coRE] ? 

To study this statement, we need to construct the complement of $math[f_{all}]:

$math[\overline{f_{all}}(w) = \left\{ \begin{array}{ll} 1 & \mbox{iff there exists } w \in \Sigma^* \mbox{ such that } M(w)=\bot \\ 0 & \mbox{otherwise} \end{array} \right.]

and then examine the question:
  * **is it the case that** $math[\overline{f_{all}} \in RE] ?

The intuitive answer seems to be **//no//**, because there is no known way to provide a //yes// answer, since here //yes// involves **non-termination**.  We validate our intuition by selecting a problem which is not in $math[RE] and proving Turing-reducibility from it:
  * $math[\overline{f_h}\leq_T\overline{f_{all}}]

We must find a transformation from the input $math[M,w] of the halting complement to the input $math[M'] of $math[\overline{f_{all}}]. We build $math[M'] as follows:
  * $math[M'(\omega) = ]
    * if $math[\omega = \epsilon] run $math[M(w)]
    * otherwise halt.

We prove the transformation is correct. Direction $math[\implies]:
  * suppose $math[M(w)] does not halt. Then, $math[M'(\epsilon)] does not halt, hence there exists a string for which $math[M'] does not halt.

Direction $math[\impliedby]:
   * suppose there exists a string for which $math[M'] does not halt. By construction of $math[M'], that string must be $math[\epsilon]. Hence $math[M(w)] does not halt.

==== Studying the machine equivalence problem ====

Study the decidability of the problem $math[f_{eq}] from the previous lecture.

==== Where does $math[f_{all}] belong? ====

So far we have seen that $math[f_{all} \not\in R, \not\in RE, \not\in coRE]. So, where does $math[f_{all}] belong?

It is possible to construct a **hierarchy** of classes which capture the notion of **degree of undecidability**. We will give a brief and informal view of this hierarchy.

Suppose for a moment that we live in a Universe where the halting problem is **decidable**. Then, all problems which we reduced to it are also decidable, via the transformations that we find. More formally, this means that, in our hypothetical Universe, all problems in $math[RE] are decidable, and then also those from $math[coRE]. Are there any undecidable problems in this Universe? We have already shown that $math[f_{all}] is such a problem.

More formally, $math[f_{all}] remains undecidable, even if we assume we have an **oracle** which can decide the halting problem. The problem $math[\overline{f_{all}}] can be shown to be **recursively-enumerable** in this Universe. Construct a Turing Machine which:
  * for each word $math[w\in\Sigma^*], it runs the halting problem oracle $math[M(w)].
    * If the answer of the oracle is 0, then output 1
    * otherwise, continue (looking)

Note that our machine will always terminate if there exists an input for which $math[M] does not halt since the oracle **always terminate**.

To conclude, in o
ur Universe, $math[f_{all}\in coRE]. 

There is an infinity of classes $math[(co)RE^Y], which contain (co)-recursively enumerable problems, given an oracle from class $math[Y]. Hence, formally, $math[f_{all}\in coRE^{RE}].

===== A general undecidability result (Rice's Theorem) ====

$prop[Rice]
Let $math[\mathcal{C} \subseteq RE]. Given a Turing Machine $math[M], we ask:// "The problem accepted by $math[M] is in $math[\mathcal{C}]?". //Answering this question is not in $math[R] (not decidable).
$end

Before looking at the proof, let us examine the statement made by the theorem. Suppose $math[A\subsetneq B]. The elements of $math[A] can be interpreted as members of $math[B] with a **given property**. Hence, $math[A] can be identified with **the property itself**. 

The same is the case with $math[\mathcal{C}]. It represents a **//property//** of recursively-enumerable problems. The theorem states undecidability of a very general problem.

The **input** of the problem is //another (recursively-enumerable) problem itself//, given in the form of the Turing Machine which accepts it. Note that we have no means of encoding problems as words. But a recursively-enumerable problem can be completely specified by any of its accepting Turing Machines.

The **output** of the problem is to decide if the given problem has property $math[\mathcal{C}], where $math[\mathcal{C}] is arbitrary but fixed, and it is not a trivial set (the emptyset).

This decision problem is **undecidable**. To better understand its meaning, imagine that $math[\mathcal{C}] may stand for:
  * //the set of problems which return 1 for odd inputs//
  * //the set of problems whose accepting Turing Machines implement a 'virus-like' behaviour during their computation//

In general, verifying non-trivial (and non-structural) properties of programs (such as 'virus-like' behaviour) is undecidable.

$proof
We consider that the trivial problem $math[f(n) = 0] whose answer is $math[0] for any instance, is not in $math[\mathcal{C}]. Since $math[\mathcal{C}] is non-empty, suppose $math[f^* \in \mathcal{C}], and since $math[f^*] is recursively-enumerable, let $math[M^*] be the Turing Machine which accepts $math[f^*].

We apply a reduction from a variant of $math[f_{111}], namely $math[f_x]. $math[f_x] asks if a Turing Machine halts for input $math[x]. Suppose we can decide the membership $math[f \in \mathcal{C}] by some Turing Machine. Based on the latter, we construct a Turing Machine which decides $math[f_x] (i.e. solves the halting problem for a particular input). Let $math[M_x] be the Turing Machine which //accepts// $math[f_x].

Let:

$math[\Pi_M(\omega) = \mbox{ if } M_x(M) \mbox{halts, then run} M^*(\omega) \mbox{.}]

If $math[f_{\Pi_M}] is the problem accepted by $math[\Pi_M], we show that:

$math[f_{\Pi_M} \in \mathcal{C} \mbox{ iff } M_x(M) \mbox{ halts}]

$math[(\Rightarrow)]. Suppose $math[f_{\Pi_M} \in \mathcal{C}]. Then $math[\Pi_M(\omega)] cannot loop for every input $math[\omega \in \Sigma^*]. If there were so, then $math[f_{\Pi_M}] would be the trivial function always returning $math[0] for any input, which we have assumed is not in $math[\mathcal{C}]. Thus, $math[M_x(M)] halts.

$math[(\Leftarrow)]. Suppose $math[M_x(M)] halts. Then the
 behaviour of $math[\Pi_M(\omega)] is precisely that of
$math[M^*(\omega)]. $math[\Pi_M(\omega)] will return $math[1] whenever $math[M^*(\omega)] will return $math[1] and $math[\Pi_M(\omega) = \perp] whenever $math[M^*(\omega) = \perp]. Since $math[f \in \mathcal{C}], then also $math[f_{\Pi_M} \in \mathcal{C}].
$end

===== Generation =====

In what follows, we look at an interesting property of recursively-enumerable problems - property which gives the name of this class of problems.

$prop
A problem $math[f \in Hom(\mathbb{N}, \{0, 1\})] is recursively enumerable iff there exists a Turing Machine which can **enumerate/generate** all elements in $math[A_f = \{w \in \mathbb{N} \mid f(n^w) = 1\}]. Intuitively, $math[A_f] is the set of inputs of $math[f] for which the answer at hand is yes.
$end

==== Proof ====
$math[\Longrightarrow] Suppose $math[f] is recursively-enumerable and $math[M] accepts $math[f]. We write $math[w_i] to refer to the //i//th word from $math[\Sigma^*]. We specify the $math[TM] generating $math[A_f] by the following pseudocode:

$algorithm[$math[GEN()]]
$math[\mbox{static } A_f = \emptyset \mbox{;}]

$math[k=0 \mbox{;}]

$math[\mathbf{while} \mbox{ } \mathit{True} \mbox{ } \mathbf{do}]

$math[\quad \mathbf{for} \mbox{ } 0 \leq i \leq k \mbox{ } \mathbf{do}]

$math[\quad \quad \mbox{run } M(w_i) \mbox{;}]

$math[\quad \quad \mathbf{if} \mbox{ } M(w_i) \mbox{ } halts \mbox{ } before \mbox{ } k \mbox{ } steps \mbox{ } and \mbox{ } i \notin A_f \mbox{ } \mathbf{then}]

$math[\quad \quad \quad A_f = A_f \cup \{ w_i \};]

$math[\quad \quad \quad \mathbf{return} \mbox{ } w_i;]  

$math[\quad \quad \mathbf{end}]

$math[\quad \mathbf{end}]

$math[\quad k=k+1;]

$math[\mathbf{end}]
$end

The value of $math[k] from the **for** has a two-fold usage. First, it is used to explore all inputs $math[w_i : 0 \leq i \leq k]. Second, it is used as a time-limit for $math[M]. For each $math[w_i] we run $math[M(w_i)], for precisely $math[k] steps. If $math[M(w_i) = 1] in at most $math[k] steps, then $math[w_i] is added to $math[A_f], and then returned (written on the tape). Also, $math[w_i] is stored for a future execution of $math[GEN]. If $math[M(w_i) = 1] for some $math[w_i], then there must exist a $math[k : k \geq i] such that $math[M(w_i)] halts after $math[k] steps. Thus, such a $math[k] will eventually be reached.

$math[\Longleftarrow] Assume we have the Turing Machine $math[GEN] which generates $math[A_f]. We construct a Turing Machine $math[M] which accepts $math[f]. $math[M] works as follows:

$algorithm[$math[M(w)]]
$math[A_f=\emptyset, \mbox{ } n=0;]

$math[\mathbf{while} \mbox{ } w \notin A_f \mbox{ } \mathbf{do}]

$math[\quad v=GEN();]

$math[\quad A_f = A_f \cup \{ v \};]

$math[\mathbf{end}]

$math[\mathbf{return} \mbox{ } 1;]
$end

$math[M] simply uses $math[GEN] to generate elements in $math[A_f] . If $math[w \in A_f] , if will eventually be generated, and $math[M] will output $math[1]. Otherwise $math[M] will loop. Thus $math[M] accepts $math[f].


The Proposition is useful since in some cases, it may be easier to find a generator for $math[f], instead of an accepting Turing Machine.