Beyond RE

In the last lecture:

• we have introduced the classes $ R$ and $ RE$ , which classify decision problems into:
  • decidable (which can be solved by algorithms which always terminate).
  • semi-decidable (which can be solved by algorithms which terminate only if the answer is yes).

• we have introduced a technique for proving that a problem $ f$ is not decidable:
  • find another problem $ f^*$ which is known NOT to be decidable;
  • prove that $ f^* \leq_T f$ (Turing-reducibility - i.e. $ f$ is at least as hard as $ f^*$ )

In this lecture:

• we will examine some problems which are outside the class $ RE$
• we will use the concept of Turing-reducibility to prove that some problems are not $ RE$ .

Definition (Complement of a problem):

Let $ f \in Hom(\mathbb{N},\{0,1\})$ . We denote by $ \overline{f}$ the problem:

$ \overline{f}(n) = \left\{ \begin{array}{ll} 1 & \mbox{iff } f(n)=0\\0 & \mbox{iff } f(n)=1 \end{array} \right.$

We call $ \overline{f}$ the complement of $ f$ .

For instance, the complement of $ f_h$ is the problem which asks if a Turing Machine $ M$ does not halt for input $ w$ . We also note that $ \overline{\overline{f}} = f$ - i.e. the double complement of a problem is the problem itself.

We define the class:

$ coRE = \{f \in Hom(\mathbb{N}, \{0; 1\}) \mid \overline{f} \in RE\}$

$ coRE$ contains the set of all problems whose complement is in $ RE$ .

We study the relationship between $ coRE$ and the classes $ R$ and $ RE$ .

Proposition:

$ RE \cap coRE = R$ .

Proof:

We first show $ RE \cap coRE \subseteq R$ . Assume $ f \in RE \cap coRE$ . Hence, there exists a Turing Machine $ M$ which accepts $ f$ and a Turing Machine $ \overline{M}$ which accepts $ \overline{f}$ . We build the Turing Machine:

$ M^*(w) = \mbox{ for } i \in \mathbb{N} \left\{ \begin{array}{ll} \mbox{ run } M(w) \mbox{ for } i \mbox{ steps.}\\ \mbox{ If } M(w) = 1 \mbox{, return } 1 \mbox{. Otherwise:}\\ \mbox{ run } \overline{M}(w) \mbox{ for } i \mbox{ steps.}\\ \mbox{ If } \overline{M}(w) = 1 \mbox{, return } 0 \mbox{.} \end{array} \right.$

Since $ M$ and $ \overline{M}$ will always halt when the expected result is 1, they can be used together to decide $ f$ . Hence $ f \in R$ .

Show $ R \subseteq RE \cap coRE$ as an exercise.

Proposition:

$ f \in R \mbox{ iff } \overline{f} \in R \mbox{.}$

Proof:

The proposition follows immediately since the Turing Machine which decides $ f$ can be used to decide $ \overline{f}$ , by simply switching its output from $ 0$ to $ 1$ and $ 1$ to $ 0$ . The same holds for both directions.

First of all, the complement of the halting problem is in $ coRE$ : this follows immediately from the complement definition and the above propositions.

All problems for which we can prove $ RE$ membership are problems in $ coRE$ .

$ f_{all}(w) = \left\{ \begin{array}{ll} 1 & \mbox{iff } M(w) \mbox{ halts for all } w \in \Sigma^* \\ 0 & \mbox{otherwise} \end{array} \right.$

We have already seen that $ f_{all} \not\in R$ , and we suspect that $ f_{all}\not\in RE$ . The intuitive reason is that:

• $ f_{all}$ implies a doubly-infinite exploration:
  • for an infinite number of inputs $ w$
  • run $ M(w)$ for $ k\in\mathbb{N}$ of steps, and see if $ M$ reaches a final state. There are infinitely many steps.

We validate our intuition by proving $ f_{all}\not\in RE$ . We apply precisely the same technique as in the $ f_{all}\not\in R$ case. We choose $ \overline{f_h}\not\in RE$ , and we prove:

• $ \overline{f_h}\leq_T f_{all}$

We must find a transformation from the input $ M,w$ of the halting complement to the input $ M'$ of $ f_{all}$ . We build $ M'$ as follows:

• $ M'(\omega) = $
  • run the simulation $ M(w)$ for a number $ $ of steps, then stop the simulation.
  • if $ M$ has reached a final state, enter a looping state;
  • otherwise, halt.

We prove that the transformation is correct. Direction $ \implies$ :

• suppose $ M(w)$ loops. Then, for any number $ k$ of steps, the $ k$ -step simulation $ M(w)$ will not bring $ M$ in a final state. Hence, for all $ \omega$ , $ M'$ halts.

Direction $ \impliedby$ :

• suppose $ M'$ halts for all inputs. Therefore, for all $ k\in\mathbb{N}$ , the $ k$ -step simulation of $ M(w)$ does not bring $ M$ to a final state. Hence $ M(w)$ loops.

We now ask a final question:

• is it the case that $ f_{all}\not\in coRE$ ?

To study this statement, we need to construct the complement of $ f_{all}$ :

$ \overline{f_{all}}(w) = \left\{ \begin{array}{ll} 1 & \mbox{iff there exists } w \in \Sigma^* \mbox{ such that } M(w)=\bot \\ 0 & \mbox{otherwise} \end{array} \right.$

and then examine the question:

• is it the case that $ \overline{f_{all}} \in RE$ ?

The intuitive answer seems to be no, because there is no known way to provide a yes answer, since here yes involves non-termination. We validate our intuition by selecting a problem which is not in $ RE$ and proving Turing-reducibility from it:

• $ \overline{f_h}\leq_T\overline{f_{all}}$

We must find a transformation from the input $ M,w$ of the halting complement to the input $ M'$ of $ \overline{f_{all}}$ . We build $ M'$ as follows:

• $ M'(\omega) = $
  • if $ \omega = \epsilon$ run $ M(w)$
  • otherwise halt.

We prove the transformation is correct. Direction $ \implies$ :

• suppose $ M(w)$ does not halt. Then, $ M'(\epsilon)$ does not halt, hence there exists a string for which $ M'$ does not halt.

Direction $ \impliedby$ :

• suppose there exists a string for which $ M'$ does not halt. By construction of $ M'$ , that string must be $ \epsilon$ . Hence $ M(w)$ does not halt.

Study the decidability of the problem $ f_{eq}$ from the previous lecture.

So far we have seen that $ f_{all} \not\in R, \not\in RE, \not\in coRE$ . So, where does $ f_{all}$ belong?

It is possible to construct a hierarchy of classes which capture the notion of degree of undecidability. We will give a brief and informal view of this hierarchy.

Suppose for a moment that we live in a Universe where the halting problem is decidable. Then, all problems which we reduced to it are also decidable, via the transformations that we find. More formally, this means that, in our hypothetical Universe, all problems in $ RE$ are decidable, and then also those from $ coRE$ . Are there any undecidable problems in this Universe? We have already shown that $ f_{all}$ is such a problem.

More formally, $ f_{all}$ remains undecidable, even if we assume we have an oracle which can decide the halting problem. The problem $ \overline{f_{all}}$ can be shown to be recursively-enumerable in this Universe. Construct a Turing Machine which:

• for each word $ w\in\Sigma^*$ , it runs the halting problem oracle $ M(w)$ .
  • If the answer of the oracle is 0, then output 1
  • otherwise, continue (looking)

Note that our machine will always terminate if there exists an input for which $ M$ does not halt since the oracle always terminate.

To conclude, in o ur Universe, $ f_{all}\in coRE$ .

There is an infinity of classes $ (co)RE^Y$ , which contain (co)-recursively enumerable problems, given an oracle from class $ Y$ . Hence, formally, $ f_{all}\in coRE^{RE}$ .

Proposition (Rice):

Let $ \mathcal{C} \subseteq RE$ . Given a Turing Machine $ M$ , we ask: “The problem accepted by $ M$ is in $ \mathcal{C}$ ?”. Answering this question is not in $ R$ (not decidable).

Before looking at the proof, let us examine the statement made by the theorem. Suppose $ A\subsetneq B$ . The elements of $ A$ can be interpreted as members of $ B$ with a given property. Hence, $ A$ can be identified with the property itself.

The same is the case with $ \mathcal{C}$ . It represents a property of recursively-enumerable problems. The theorem states undecidability of a very general problem.

The input of the problem is another (recursively-enumerable) problem itself, given in the form of the Turing Machine which accepts it. Note that we have no means of encoding problems as words. But a recursively-enumerable problem can be completely specified by any of its accepting Turing Machines.

The output of the problem is to decide if the given problem has property $ \mathcal{C}$ , where $ \mathcal{C}$ is arbitrary but fixed, and it is not a trivial set (the emptyset).

This decision problem is undecidable. To better understand its meaning, imagine that $ \mathcal{C}$ may stand for:

• the set of problems which return 1 for odd inputs
• the set of problems whose accepting Turing Machines implement a 'virus-like' behaviour during their computation

In general, verifying non-trivial (and non-structural) properties of programs (such as 'virus-like' behaviour) is undecidable.

Proof:

We consider that the trivial problem $ f(n) = 0$ whose answer is $ 0$ for any instance, is not in $ \mathcal{C}$ . Since $ \mathcal{C}$ is non-empty, suppose $ f^* \in \mathcal{C}$ , and since $ f^*$  is recursively-enumerable, let $ M^*$ be the Turing Machine which accepts $ f^*$ .

We apply a reduction from a variant of $ f_{111}$ , namely $ f_x$ . $ f_x$ asks if a Turing Machine halts for input $ x$ . Suppose we can decide the membership $ f \in \mathcal{C}$ by some Turing Machine. Based on the latter, we construct a Turing Machine which decides $ f_x$ (i.e. solves the halting problem for a particular input). Let $ M_x$ be the Turing Machine which accepts $ f_x$ .

Let:

$ \Pi_M(\omega) = \mbox{ if } M_x(M) \mbox{halts, then run} M^*(\omega) \mbox{.}$

If $ f_{\Pi_M}$ is the problem accepted by $ \Pi_M$ , we show that:

$ f_{\Pi_M} \in \mathcal{C} \mbox{ iff } M_x(M) \mbox{ halts}$

$ (\Rightarrow)$ . Suppose $ f_{\Pi_M} \in \mathcal{C}$ . Then $ \Pi_M(\omega)$ cannot loop for every input $ \omega \in \Sigma^*$ . If there were so, then $ f_{\Pi_M}$ would be the trivial function always returning $ 0$ for any input, which we have assumed is not in $ \mathcal{C}$ . Thus, $ M_x(M)$ halts.

$ (\Leftarrow)$ . Suppose $ M_x(M)$ halts. Then the behaviour of $ \Pi_M(\omega)$ is precisely that of $ M^*(\omega)$ . $ \Pi_M(\omega)$ will return $ 1$ whenever $ M^*(\omega)$ will return $ 1$ and $ \Pi_M(\omega) = \perp$ whenever $ M^*(\omega) = \perp$ . Since $ f \in \mathcal{C}$ , then also $ f_{\Pi_M} \in \mathcal{C}$ .

In what follows, we look at an interesting property of recursively-enumerable problems - property which gives the name of this class of problems.

Proposition ():

A problem $ f \in Hom(\mathbb{N}, \{0, 1\})$ is recursively enumerable iff there exists a Turing Machine which can enumerate/generate all elements in $ A_f = \{w \in \mathbb{N} \mid f(n^w) = 1\}$ . Intuitively, $ A_f$ is the set of inputs of $ f$ for which the answer at hand is yes.

$ \Longrightarrow$ Suppose $ f$ is recursively-enumerable and $ M$ accepts $ f$ . We write $ w_i$ to refer to the ith word from $ \Sigma^*$ . We specify the $ TM$ generating $ A_f$ by the following pseudocode:

Algorithm ($ GEN()$ ):

$ \mbox{static } A_f = \emptyset \mbox{;}$

$ k=0 \mbox{;}$

$ \mathbf{while} \mbox{ } \mathit{True} \mbox{ } \mathbf{do}$

$ \quad \mathbf{for} \mbox{ } 0 \leq i \leq k \mbox{ } \mathbf{do}$

$ \quad \quad \mbox{run } M(w_i) \mbox{;}$

$ \quad \quad \mathbf{if} \mbox{ } M(w_i) \mbox{ } halts \mbox{ } before \mbox{ } k \mbox{ } steps \mbox{ } and \mbox{ } i \notin A_f \mbox{ } \mathbf{then}$

$ \quad \quad \quad A_f = A_f \cup \{ w_i \};$

$ \quad \quad \quad \mathbf{return} \mbox{ } w_i;$

$ \quad \quad \mathbf{end}$

$ \quad \mathbf{end}$

$ \quad k=k+1;$

$ \mathbf{end}$

The value of $ k$ from the for has a two-fold usage. First, it is used to explore all inputs $ w_i : 0 \leq i \leq k$ . Second, it is used as a time-limit for $ M$ . For each $ w_i$ we run $ M(w_i)$ , for precisely $ k$ steps. If $ M(w_i) = 1$ in at most $ k$ steps, then $ w_i$ is added to $ A_f$ , and then returned (written on the tape). Also, $ w_i$ is stored for a future execution of $ GEN$ . If $ M(w_i) = 1$ for some $ w_i$ , then there must exist a $ k : k \geq i$ such that $ M(w_i)$ halts after $ k$ steps. Thus, such a $ k$ will eventually be reached.

$ \Longleftarrow$ Assume we have the Turing Machine $ GEN$ which generates $ A_f$ . We construct a Turing Machine $ M$ which accepts $ f$ . $ M$ works as follows:

Algorithm ($ M(w)$ ):

$ A_f=\emptyset, \mbox{ } n=0;$

$ \mathbf{while} \mbox{ } w \notin A_f \mbox{ } \mathbf{do}$

$ \quad v=GEN();$

$ \quad A_f = A_f \cup \{ v \};$

$ \mathbf{end}$

$ \mathbf{return} \mbox{ } 1;$

$ M$ simply uses $ GEN$ to generate elements in $ A_f$ . If $ w \in A_f$ , if will eventually be generated, and $ M$ will output $ 1$ . Otherwise $ M$ will loop. Thus $ M$ accepts $ f$ .

The Proposition is useful since in some cases, it may be easier to find a generator for $ f$ , instead of an accepting Turing Machine.