Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
aa:intro:complexity_theory [2016/08/02 15:58] malex |
aa:intro:complexity_theory [2016/08/03 11:34] (current) malex |
||
|---|---|---|---|
| Line 312: | Line 312: | ||
| The fundamental property of problems from $math[NP] is that "//solutin candidates can be// verified //in polynomial time//". An analogy with solving crosswords is possible. Generating all possible solutions (by "lexicographically" generating all posible words) is obviously exponential. But since //verifying// whether a solution is correct can be done in polynomial time w.r.t. the size of the crossword, the problem in $math[NP]. We shall soon see that all algorithms which solve hard problems from $math[NP] can be split up into an exponential "generating" procedure, and a polynomial "verification procedure". | The fundamental property of problems from $math[NP] is that "//solutin candidates can be// verified //in polynomial time//". An analogy with solving crosswords is possible. Generating all possible solutions (by "lexicographically" generating all posible words) is obviously exponential. But since //verifying// whether a solution is correct can be done in polynomial time w.r.t. the size of the crossword, the problem in $math[NP]. We shall soon see that all algorithms which solve hard problems from $math[NP] can be split up into an exponential "generating" procedure, and a polynomial "verification procedure". | ||
| + | |||
| + | ===== - Hard and complete problems for a class ===== | ||
| + | Recall that proving $math[f \notin R] for a given problem $math[f] was done using contraposition. Essentially, the proof relied on finding a Turing reduction $math[f^* \leq_T f] to a problem $math[f^*] for which $math[f^* \notin R] is known. | ||
| + | |||
| + | First, we note that $math[R] could be easily replaced with any complexity class $math[X]. Thus, a more general proof scheme could be described as: | ||
| + | |||
| + | $math[f^* \notin X, f \leq_X f^* \Longrightarrow f \notin X] | ||
| + | |||
| + | where $math[f^*] and $math[X] are given in advance. We observe that we have replaced $math[\leq_T] by $math[\leq_X], in order to highlight that the reduction $math[\leq_X] depends on the choice of $math[X]. We shall later see that we cannot allways use Turing Reductions, and there are $math[X]'s for which more restrictions on the reduction must be in place. | ||
| + | |||
| + | Returning to the proof scheme, we make a second note: to prove $math[f \notin R], we must first find some $math[f^* \notin R]. But if this very fact is shown by the same technique, then we need another problem $math[f^{**} \notin R] and so on. This situation is similar to: "//Which came first, hens or eggs?//". | ||
| + | |||
| + | For the case of $math[R], this issue was settled by the proof $math[f_h \notin R] using diagonalization. This introduced an initial undecidable problem, and the aforementioned technique could be employed. | ||
| + | |||
| + | Now, let us turn our attention to $math[X=NP]. First, let us examine the properties which the reduction should have, in order to make our technique work in this particular case. Let $math[f^* \notin NP]. We need to show that $math[f \notin NP]. | ||
| + | |||
| + | The first part consists of assuming $math[f \in NP]. Next, we ??CUVANT LIPSA?? a reduction $math[T:I_{f^*} \rightarrow I_f] where $math[I_{f^*}] and $math[I_f] are the inputs of the problem from the subscript (In order to make explicit the direction of the transformation, we ignore the fact that both $math[I_{f^*}] and $math[I_f] are (subsets of) naturals.). We recall that the reduction needs to satisfy: | ||
| + | |||
| + | $math[\forall i \in I_{f^*} : f^*(i)=1 \Longleftrightarrow f(T(i))=1] | ||
| + | |||
| + | In words: "//we can solve all instances $math[i] of $math[I_{f^*}] by solving instances $math[T(i)] of $math[I_f]//". This is done as follows: | ||
| + | |||
| + | - receive $math[i \in I_{f^*}] | ||
| + | - compute $math[T(i) \in I_f] | ||
| + | - run the $math[NTM] $math[M_f(T(i))] ($math[M_f] must exist since $math[f \in NP]) and return it's answer. | ||
| + | |||
| + | After a careful look, we observe that the conclusion $math[f^* \in NP] (which is our objective, in order to complete the contrapositive argument) is not immediate. If, for instance, the computation of $math[T(i)] takes exponential time, the the proof does not work: the $math[NTM] which performs 1. 2. 3. runs in (non-deterministic) exponential time. | ||
| + | |||
| + | Thus, the restriction that $math[T] is decidable is insufficient. We further need that $math[T] is computable in **polynomial time**. We write $math[f^* \leq_p f] iff there exists a polynomial-time transformation which allows solving $math[f^*] via $math[f], as illustrated by the scheme above. | ||
| + | |||
| + | We now turn our attention to the "//hen and eggs//" issue. We need an initial problem, which is known **not** to be in $math[NP]. Unfortunately, such a problem is not known, although there have been major (unsuccessful) efforts in trying to find one. The same holds for the class $math[P]. Hence, the issue: | ||
| + | |||
| + | $math[P \subsetneq NP] | ||
| + | |||
| + | is currently open, and it is generlly believed that it is true, but all proof attempts have failed. The good news is that our effort in classifying problems is not fruitless. Transformations: | ||
| + | |||
| + | $math[f^* \leq_p f \quad f^* \leq_T f] | ||
| + | |||
| + | establish a relation between problems, which is denoted as //hardness//. No matter the reduction type (polynomial or Turing), $math[f] is **at least as hard** as $math[f^*]. With the machine $math[M_f] (together with computing a transformation) we can solve all instances of $math[f^*]. It may be possible that $math[T] is bijective: hence each input of $math[f^*] is uniquely mapped to an input of $math[f] and vice-versa. Then, $math[f] and $math[f^*] are //equally hard//. However, this is not generally the case, hence the term "//at least//"as hard. | ||
| + | |||
| + | We can naturally extend hardness to complexity classes: | ||
| + | |||
| + | $def[1.6.1] | ||
| + | //A problem $math[f] is called $math[NP]-hard iff for all $math[f^\prime \in NP], $math[f^\prime \leq_p f].// | ||
| + | $end | ||
| + | |||
| + | Thus, a problem is hard w.r.t. a class, iff it is at least as hard as any problem in the class at hand. Note that hardness can be defined w.r.t. any complexity class and not just $math[NP], provided that the appropriate type of transformation is employed. | ||
| + | |||
| + | $def[1.6.2] | ||
| + | //A problem $math[f] is called $math[NP]-complete iff it is $math[NP]-hard and $math[f \in NP].// | ||
| + | $end | ||
| + | |||
| + | Informally, $math[NP]-complete problems are the hardest problems in $math[NP]. In the more general case, complete problems w.r.t. a class are the hardest of that class. | ||
| + | |||
| + | It is likely that if $math[f] is $math[NP]-complete, then $math[f \notin P], however, this is not a proven fact. | ||
| + | |||
| + | Thus, instead of trying to disprove membership of a class ($math[f \notin P]), in complexity theory, we prove completeness for immediate upper (or greater) class ($math[f] is $math[NP]-complete). | ||
| + | |||
| + | The intuition is that class membership provides an upper bound, while hardness - a lower bound for the difficulty of a problem. We illustrate this by an abstract example. Recall: | ||
| + | |||
| + | $math[P \subseteq NP \subseteq EXPTIME] | ||
| + | |||
| + | Let $math[f] be a problem. Suppose we find an algorithm for $math[f] which runs in exponential time, however, we cannot find one which runs in polynomial time on a $math[NTM]. At this point, we have $math[f \in EXPTIME]. Suppose we know $math[f] is $math[P]-hard, thus, $math[f] can be used to solve any problem in $math[P]. We now know that $math[f] can be solved exponentially and it is unlikely that $math[f] can be solved in sub-polynomial (e.g. logarithmic) time. Thus, the likely variants are: $math[f] may be solved in polynomial time (i) by a convetional $math[TM \mbox{ } f \in P] or (ii) by a $math[NTM \mbox{ } f \in NP], and (iii) in exponential time, again by a conventional $math[TM \mbox{ } f \in EXPTIME]. In the best case $math[f] is polynomially solvable. In the worst case - it is exponentially solvable. | ||
| + | |||
| + | Suppose now we also find that $math[f] is $math[NP]-hard. We cannot rule out $math[f \in P] by a proof, but Complexity Theory predicts that such a membership is not likely. Hence, the feasible variants remain (ii) and (iii). | ||
| + | |||
| + | Finally, if we manage to improve our exponential algorithm for $math[f] and turn it into a non-deterministic polynomial algorithm, then $math[f \in NP] and, hence, it is $math[NP]-complete. Case (iii) remains of course true, but it does not carry useful information. At this point, we have an exact characterisation of the difficulty of $math[f]. | ||
| + | |||
| + | ==== Proving $math[NP]-completeness ==== | ||
| + | For a problem $math[f] to be $math[NP]-complete, it must satisfy two conditions. The first: $math[f \in NP] is shown by finding a $math[NTM] which decides $math[f] in polynomial time. For the second part ($math[f] is $math[NP]-hard), we can employ precisely the "reduction-finding" technique illustrated at the beginning of this section. | ||
| + | |||
| + | ==== - Proposition ==== | ||
| + | //A problem $math[f] is $math[NP]-hard iff there exists a problem $math[g] which is $math[NP]-hard, such that $math[g \leq_p f].// | ||
| + | |||
| + | //Proof:// Suppose $math[g \leq_p f] and $math[g] is $math[NP]-hard. Hence, for all $math[h \in NP], $math[h \leq_p g]. By transitivity, we also have that $math[h \leq_p f]. It follows that $math[f] is $math[NP]-hard. | ||
| + | |||
| + | In the former proof we have made use of the transitivity of $math[\leq_p], without showing it. We now state several properties of $math[\leq_p] including transitivity, and leave the proofs as exercises. | ||
| + | |||
| + | ==== - Proposition ==== | ||
| + | //$math[\leq_p] is reflexive and transitive.// | ||
| + | |||
| + | ==== - Proposition ==== | ||
| + | //The set of $math[NP]-hard problems is closed under $math[\leq_p].// | ||
| + | |||
| + | ==== - Proposition ==== | ||
| + | //The set of $math[NP]-complete problems together with $math[\leq_p] is an equivalence class.// | ||
| + | |||
| + | ==== - Proposition ==== | ||
| + | //Assume $math[f] is $math[NP]-complete. If $math[f \in P], then $math[P=NP].// | ||
| + | |||
| + | The former proposition, whose proof follows immediately from the underlying definitions, makes the case for the common belief that $math[P \neq NP]. If some efficient algorithm can be found for some $math[NP]-complete problem, then **all** problems in $math[NP] can be solved in polynomial time. | ||
| + | |||
| + | The $math[P=NP] issue can also be given another intuitive interpretation: "//The verification of a solution candidate is as difficult as generating it//" or, alternatively: "//Verifying a given proof $math[P] for $math[A], is as difficult as finding a proof for $math[P]//". | ||
| + | |||
| + | Finally, to better understand the implications of $math[P=NP], consider several facts which would be arguably true, in the case the former equality holds: | ||
| + | * We can provide a solution to the astronaut's problem (see the first chapter). | ||
| + | * Partial program correctness can be solved effciently. Technique such as model checking can be applied to a wide range of applications (including operating system kernels). Bugs are almost removed. Windows bluescreens are no longer happening. | ||
| + | * Generation of exponentially many training sets would make tasks such as voice recognition, computer vision, natural language processing - computationlly easy. | ||
| + | * Mathematical proofs (of, say 100 pages) can be generated efficiently. Computers can be used to find proofs for some open problems. | ||
| + | * We can exponential search to find passwords, or to break encryption keys in polynomial time. Internet privacy is no longer possible using encryption (e.g. using SSH). Internet commerce and banking is no longer possible. Safe communication is no longer possible (at all levels). Any computer-controlled facility (public, militry, etc.), which is connected to the Internet has considerable potential of being compromised. | ||
| + | |||
| + | ==== 1.6.1 Remark (Practical applications of reductions) ==== | ||
| + | //As illustrated before, reductions of the type $math[\leq_p] are a theoretical tool which is useful for providing $math[NP]-hardness. Reductions also have practical applications. For instance, most $math[NP]-complete problems are solved by employing $math[SAT] solvers, which, as discussed in the former chapters, may be quite fast in general case. Thus, a specific problem instance is cast (via an appropriate transformation) into a formula $math[\varphi], such that $math[\varphi] is satisfiable iff the answer to the instance is// yes. | ||
| + | |||
| + | ==== $math[SAT]. The first $math[NP]-complete problem. ==== | ||
| + | We observe that the "//hen and eggs//" issue still holds in our scenario. To apply our technique, we need an initial $math[NP]-hard problem in the first place. This problem is provided by Cook's Theorem, which proves that $math[SAT] is $math[NP]-complete. The technique for the proof relies on building, for each $math[NTM \mbox{ } M] (and hence, for each problem in $math[NP]), a formula $math[\varphi_M] such that it is satisfiable iff there exists a sequence in the computation tree leading to **success**. | ||