Asymptotic Notations

• Contacts: Dumitru Mihai-Valentin, Matei Popovici
• Published: 9.11.2015

An execution time is a function :\mathbb{N}\rightarrow\mathbb{N}$, which maps a value $ representing the size of the input of an algorithm to the number of instructions (/execution steps) performed by the algorithm.

Suppose : \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}$ is a non-necessarily monotone function over positive reals. We define the sets (f(n)) \subseteq \mathbb{H}om(\mathbb{R}^{+},\mathbb{R}^{+})$, with \in\{O,o,\Theta,\omega,\Omega\}$, which we call asymptotic notations (or simply notations).

Question: Why are asymptotic notations defined for positive real functions, instead of positive integers? (as in execution times)

$$ \Theta(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \exists c_1, c_2 \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ c_1f(n) \le g(n) \le c_2f(n) \}$$

We say that $\Theta(f(n))$ is the class (set) of functions with the same asymptotic growth as f(n).

Let (n) = n^2 + 8$ and (n) = 3n^2 + 4n + 2$
We see that, for = 1, c_1 = 1, c_2 = 4$:

thus (n) \in \Theta(f(n))$

$$ O(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \exists c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le g(n) \le cf(n) \}$$

We say that (f(n))$ is the class (set) of functions that grow asymptotically //at most as much as// (n)$.

Let (n) = nlog(n) + n$ and (n) = nlog(n) + n + 10000$
We see that, for = 10000, c = 2$:
$0 \le nlog(n) + n + 10000 \le 2nlog(n) + 2n$
thus (n) \in O(f(n))$

$$ \Omega(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \exists c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le cf(n) \le g(n) \}$$

We say that $\Omega(f(n))$ is the class (set) of functions that grow asymptotically at least as much as (n)$.

Let (n) = n^4$ and (n) = 2^n$
We see that, for = 16, c = 1$:
$0 \le n^4 \le 2^n$
thus (n) \in \Omega(f(n))$

$$ o(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \forall c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le g(n) \le cf(n) \}$$

We say that (f(n))$ is the class (set) of functions that grows asymptotically //strictly less than// (n)$.

Let (n) = n^2 + 3n + 6$ and (n) = 8n + 3$
We see that, for = 1, n_0 = 5$:
$0 \le 8n + 3 \le n^2 + 3n + 6$

for = 0.5, n_0 = 7$:
$0 \le 8n + 3 \le 0.5n^2 + 1.5n + 3$

for = 0.1, n_0 = 77$:
$0 \le 8n + 3 \le 0.1n^2 + 0.3n + 0.6$
etc. thus (n) \in o(f(n))$

$$ \omega(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \forall c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le cf(n) \le g(n) \}$$

We say that $\omega(f(n))$ is the class (set) of functions that grows asymptotically strictly more than (n)$.

Let (n) = log(n)$ and (n) = n$
We see that, for = 1, n_0 = 1$:
$0 \le log(n) \le n$

for = 10, n_0 = 64$:
$0 \le 10log(n) \le n$

for = 100, n_0 = 1024$:
$0 \le 100log(n) \le n$
etc.
thus (n) \in \omega(f(n))$

Asymptotic notations are often used to refer to arbitrary functions that have a certain growth. To simplify things, we can write arithmetic expressions as follows: $(n) = \Theta(n) + O(log(n))$$
which should be read as: $ \exists g \in \Theta(n)$ and $ \exists h \in O(log(n))$ such that (n) = g(n) + h(n),\ \forall n \in \mathbb{R}^{+}$.

We can also write equations like the following: $$\Theta(n^2) = O(n^2) + o(n)$$
which should be read as:
$ \forall f \in \Theta(n^2),\ \exists g \in O(n^2)$ and \in o(n)$ such that (n) = g(n) + h(n),\ \forall n \in \mathbb{R}^{+}$.

Note that equations are not symmetric and should only be read from left to right. Consider: $$\Theta(n) = O(n)$$ While it is true that, for any function in $\Theta(n)$ there is a function equal to it in (n)$, we can clearly see that there are functions in (n)$ for which there is no correspondent in $\Theta(n)$ (e.g. (n) = 1$, (n) = log(n)$ etc.)

As a rule, each asymptotic notation on the left side of the equal sign should be read as an universally quantified function ($ \forall f$) from that class and each asymptotic notation on the right should be read as an existentially quantified function ($ \exists g$) from that class.

$$\left(\frac{\omega(n^2)}{\Theta(n)}\right) = \Omega(n) + o(n)$$

$ \forall f \in \omega(n^2)\ and\ \forall g \in \Theta(n),\ \exists h \in \Omega(n)\ and\ \exists j \in o(n)\ such\ that$ $\left(\frac{f(n)}{g(n)}\right) = h(n) + j(n),\ \forall n \in \mathbb{R}^{+}$

Which of the following is true:

• (f(n)) \cap \omega(f(n)) = \emptyset$ * (f(n)) \subsetneq O(f(n))$
• (f(n)) \setminus o(f(n)) = \Theta(f(n))$

Check the truth of the following:

• $\sqrt{n} \in O(\log(n))$
• (n) \in O(\log(\log(n)))$ * \in O(\log(n)\cdot\sqrt{n})$
• + \log(n) \in \Theta(n)$

Prove that:

• $\log(n\cdot \log(n))\in\Theta(\log(n))$
• $\sqrt{n}\in\omega(\log(n))$
• (n) + g(n) \in O(n\cdot\log(n))$ for (n)\in\Theta(n)$ and (n)\in O(n\cdot\log n)$

• $\frac{O(n\sqrt{n})}{\Theta(n)} = \ldots$
• $\frac{\Theta(n)}{O(\log(n))} = \ldots$
• $|\Theta(f(n)) - \Theta(f(n))| = \ldots$

Prove/disprove the following:

• (n) = \Omega(\log(n))$ and (n)=O(n)$ $\implies$ (n)=\Omega(\log(g(n))$ * (n) = \Omega(\log(n))$ and (n)=O(n)$ $\implies$ (n)=\Theta(\log(g(n))$
• (n) = \Omega(g(n))$ and (n)=O(n^2)$ $\implies$ $\frac{g(n)}{f(n)}=O(n)$

Find two functions $ and $ such that:

• (n) = O(g^2(n))$ * (n) = \omega(\log(g(n)))$
• (n) = \Omega(f(n)\cdot g(n))$ * (n) = \Theta(g(n)) + \Omega(g^2(n))$