====== Time and space ======

In Computability Theory, we have used classes such as $math[R] and $math[RE] in order to establish a hierarchy of hardness for problem solvability (in terms of the Turing Machine).

We now take into account the **resources** spent by the Turing Machine, during the computation process.

Let us consider a few statements regarding **resources**. Which do you think is a resource that must be taken into account?
   * the number of **states** of the Turing Machine;
   * the **size of the alphabet**;
   * the **encoding chosen for the input**;
   * the **amount of tape**;
   * the **direction in which the head is moving**;
   * the **number of transitions per word**;


===== Measuring resources =====

The amount of spent resources (time/space) by a Turing Machine $math[M] may be expressed as functions: $math[\mathcal{T}_M, \mathcal{S}_M : \Sigma^* \rightarrow \mathbb{N}] where $math[\mathcal{T}_M(w)] (resp. $math[\mathcal{S}_M(w)]) is the number of steps performed (resp. tape cells used) by $math[M], when running on input $math[w].

This definition suffers from un-necessary overhead, which makes time and space analysis difficult. We formulate some examples to illustrate why this is the case:

$algorithm[$math[Alg(n)]]
$math[\mathbf{while} \mbox{ } n \lt 100]

$math[\quad n=n+1]

$math[\mathbf{return} \mbox{ } 1]
$end

We note that $math[Alg] runs 100 steps for $math[n=0] while only one step, for $math[n \geq 100]. However, in practice, it is often considered that each input is as likely to occur as any other.

In general, it is inconvenient to account for the number of transitions of a Turing Machine w.r.t. to the **value** of the input. However, the following is a straightforward observation:

  * **The number of steps performed by a Turing Machine //is expected to increase// as the size of the input word increases** 
  * Can you find a (programming) situation where **this is NOT the case** ? {{## In Insertion Sort: Take a sorted vector of size 5 (takes approx. 3 comparisons) , vs a reversely-sorted vector of size 4:  ##}}

$def[Running time of a TM]
//The// running time //of a Turing Machine $math[M] is given by $math[\mathcal{T}_M : \mathbb{N} \rightarrow \mathbb{N}] iff:// $math[\forall \omega \in \Sigma^*] : the nb. of transitions performed by $math[M] is at most $math[\mathcal{T}_M(\mid 
\omega \mid)].
$end

==== Consumed space of a TM ====

  * Consider the Turing Machine which checks if a number encoded in binary is a power of $math[2]. If we are looking at the amount of space consumed on the tape, we find that it is **exactly** the size of the input. However, the computation process does not use additional space, and **does not write** on the tape.
  * Consider the Turing Machine which increments a number. As before, the consumed space coincides with the size of the input. The Machine writes several cells, but does not use additional space.

The above examples motivate a more careful accounting for the **consumed space**. Informally:
  * we must treat the Turing Machine as having a **input tape**, and **computation/output tape**;
  * we shall only count what the Turing Machine writes on the **computation/output tape**;

==== Common mistakes when accounting for time and space ====

Consider the following algorithm:

$algorithm[$math[P(n)]]

$math[i=0, s = 0]

$math[\mathbf{while} \mbox{ } i \lt n]

$math[\quad s=s+i]

$math[\quad i=i+1]

$math[\mathbf{return} \mbox{ } 1]
$end

  * What is **a running time** function for $math[P(n)] ? {{## T(k) = 2^k (1 + 1) + 1  ##}}
  * What is a **consumed space** function for $math[P(n)] {{##the value computed is n(n+1)/2, which takes 2k+1 bits, to which we add k bits for storing i, thus S(k)=3k+1 ##}}

===== The encoding does not matter =====

$justprop
Let $math[f] be a problem which is decidable by a Turing Machine with alphabet $math[Sigma] and with running time $math[T]. Then $math[f] is decidable in time $math[4 \cdot log (K) T(n)] by a Turing Machine with alphabet $math[\{ 0,1,\#\}], where $math[K = \mid\Sigma\mid].
$end

$proof
We build a Turing Machine $math[M^*] with the desired property, by relying on $math[M] - the TM which decides $math[f], as follows:
  * we encode each symbol of $math[\Sigma] using $math[log(K)] bits;
  * we simulate $math[M] as follows:
    * we read the symbol from the current cell in $math[log(K)] steps; In order to do so, we need to create (at most) $math[2^0 + 2^1 + ... + 2^{K} = 2^{K+1}-1] states, **for each existing state in M**. These states are connected as a complete binary tree. Each level $math[i] consisting of $math[2^i] states are responsible for reading the first $math[i] bits from the tape.
    * for writing each symbol, we require $math[log(K)] states, and the process takes exactly $math[log(K)] steps (moving backwards over the read symbol).
    * for moving the head, we require $math[0], $math[log(K)]  states (and **steps**) depending on the direction (//hold// and //left/right//, respectively.
    * we also require //do-nothing// transitions, between the **reading**, **writing** and **head moving** phases. We finally require a transition to take us to the next-state.

Hence, at most $math[4log(K)] steps are performed by $math[M*] per each transition of $math[M].
$end

===== Towards resources consumed by programs =====

Most observations which we have done for the Turing Machine, extend naturally to programming languages:
  * in general, each instruction is considered to spend 1 execution step (we'll see later why this assumption does not affect analysis)
  * it is generally considered that algorithms take more time on larger input - hence execution times are **monotonically increasing functions**;
  * temporal complexity of a program/algorithm is always computed by taking **the worst-case** number of execution steps, and **is expressed w.r.t. //size// of the input**
  * when evaluating the consumed space by an algorithm/procedure, we never account for the **space consumed by the input**. We only count the registers / memory employed **additionally**.

==== Discussion ====
Consider the following program:
<code>
read(i);
while (i>0){
   i++;
}
</code>

What is the consumed time ?

{{## The total number of steps of this program compiled in C in the worst-case, is sizeof(int)/2+1. Worst-case is when i=0. We have overflow. Formally, we can claim that this program takes constant time. We can also claim it takes exponential time w.r.t. the input, if we do not make any assumptions on the maximum store size for i. ##}}

==== Case study ====

The complexity of Merge-sort:

<code>
int* mergesort (int* v, int n) {
   int* v1 = mergesort (v, n/2);
   int* v2 = mergesort (v+n/2, n);
   return merge(v1, n/2, v2, n - n/2);
}
</code>

where: 

<code>
int* merge(int* v1, int n1, int* v2, int n2){                     
 	int* r = malloc((n1+n2)*sizeof(int));          
 	int i=0,j=0,k=0;                               
 	while (i<n1 && j<n2)                           
 		if (v1[i]>v2[j])                                        
 			r[k++] = v2[j++];                      
 		else
 			r[k++] = v1[i++];                       
 	
 	while (i<n1)                                   
 		r[k++] = v1[i++];

 	while (j<n2)                                   
 		r[k++] = v2[j++]
 	
 	return r;

 }
</code>

  * what is an execution time for **merge** ?
  * what is an execution time for **mergesort** ?