Asymptotic Notations

• Contacts: Dumitru Mihai-Valentin, Matei Popovici
• Published: 9.11.2015

An execution time is a function $ T:\mathbb{N}\rightarrow\mathbb{N}$ , which maps a value $ n$ representing the size of the input of an algorithm to the number of instructions (execution steps) performed by the algorithm.

Suppose $ f: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}$ is a non-necessarily monotone function over positive reals. We define the sets $ X(f(n)) \subseteq \mathbb{H}om(\mathbb{R}^{+},\mathbb{R}^{+})$ , with $ X\in\{O,o,\Theta,\omega,\Omega\}$ , which we call asymptotic notations (or simply notations).

Question: Why are asymptotic notations defined for positive real functions, instead of positive integers? (as in execution times)

$$ \Theta(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \exists c_1, c_2 \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ c_1f(n) \le g(n) \le c_2f(n) \}$$

We say that $ \Theta(f(n))$ is the class (set) of functions with the same asymptotic growth as $ f(n)$ .

Let $ f(n) = n^2 + 8$ and $ g(n) = 3n^2 + 4n + 2$
We see that, for $ n_0 = 1, c_1 = 1, c_2 = 4 \Rightarrow \ n^2 + 8 \le 3n^2 + 4n + 2 \le 4n^2 + 32$
Thus, $ g(n) \in \Theta(f(n))$

$$ O(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \exists c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le g(n) \le cf(n) \}$$

We say that $ O(f(n))$ is the class (set) of functions that grow asymptotically at most as much as $ f(n)$ .

Let $ f(n) = n\log(n) + n$ and $ g(n) = n\log(n) + n + 10000$
We see that, for $ n_0 = 10000, c = 2 \Rightarrow 0 \le n\log(n) + n + 10000 \le 2n\log(n) + 2n$
Thus, $ g(n) \in O(f(n))$

$$ \Omega(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \exists c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le cf(n) \le g(n) \}$$

We say that $ \Omega(f(n))$ is the class (set) of functions that grow asymptotically at least as much as $ f(n)$ .

Let $ f(n) = n^4$ and $ g(n) = 2^n$
We see that, for $ n_0 = 16, c = 1 \Rightarrow 0 \le n^4 \le 2^n$
Thus, $ g(n) \in \Omega(f(n))$

$$ o(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \forall c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le g(n) \le cf(n) \}$$

We say that $ o(f(n))$ is the class (set) of functions that grows asymptotically strictly less than $ f(n)$ .

Let $ f(n) = n^2 + 3n + 6$ and $ g(n) = 8n + 3$
We see that, for $ c = 1, n_0 = 5 \Rightarrow 0 \le 8n + 3 \le n^2 + 3n + 6$
for $ c = 0.5, n_0 = 7 \Rightarrow 0 \le 8n + 3 \le 0.5n^2 + 1.5n + 3$
for $ c = 0.1, n_0 = 77 \Rightarrow 0 \le 8n + 3 \le 0.1n^2 + 0.3n + 0.6$
etc.
Thus, $ g(n) \in o(f(n))$

$$ \omega(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \forall c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le cf(n) \le g(n) \}$$

We say that $ \omega(f(n))$ is the class (set) of functions that grows asymptotically strictly more than $ f(n)$ .

Let $ f(n) = \log(n)$ and $ g(n) = n$
We see that, for $ c = 1, n_0 = 1 \Rightarrow 0 \le \log(n) \le n$
for $ c = 10, n_0 = 64 \Rightarrow 0 \le 10 \log(n) \le n$
for $ c = 100, n_0 = 1024 \Rightarrow 0 \le 100 \log(n) \le n$
etc.
Thus, $ g(n) \in \omega(f(n))$

Asymptotic notations are often used to refer to arbitrary functions that have a certain growth. To simplify things, we can write arithmetic expressions as follows: $$ f(n) = \Theta(n) + O(\log(n)) $$
which should be read as: $ \exists g \in \Theta(n)$ and $ \exists h \in O(\log(n))$ such that $ f(n) = g(n) + h(n),\ \forall n \in \mathbb{R}^{+}$ .

We can also write equations like the following: $$\Theta(n^2) = O(n^2) + o(n)$$
which should be read as: $ \forall f \in \Theta(n^2),\ \exists g \in O(n^2)$ and $ h \in o(n)$ such that $ f(n) = g(n) + h(n),\ \forall n \in \mathbb{R}^{+}$ .

Note that equations are not symmetric and should only be read from left to right. Consider: $$\Theta(n) = O(n)$$ While it is true that, for any function in $ \Theta(n)$ there is a function equal to it in $ O(n)$ , we can clearly see that there are functions in $ O(n)$ for which there is no correspondent in $ \Theta(n)$ (e.g. $ f(n) = 1, f(n) = \log(n)$ etc.)

As a rule, each asymptotic notation on the left side of the equal sign should be read as an universally quantified function ($ \forall f$ ) from that class and each asymptotic notation on the right should be read as an existentially quantified function ($ \exists g$ ) from that class.

$$\left(\frac{\omega(n^2)}{\Theta(n)}\right) = \Omega(n) + o(n)$$

$ \forall f \in \omega(n^2)\ and\ \forall g \in \Theta(n),\ \exists h \in \Omega(n)$ and $ \exists j \in o(n)$ such that $ \left(\frac{f(n)}{g(n)}\right) = h(n) + j(n),\ \forall n \in \mathbb{R}^{+}$

Which of the following is true:

• $ o(f(n)) \cap \omega(f(n)) = \emptyset$
• $ o(f(n)) \subsetneq O(f(n))$
• $ O(f(n)) \setminus o(f(n)) = \Theta(f(n))$

Check the truth of the following:

• $ \sqrt{n} \in O(\log(n))$
• $ \log(n) \in O(\log(\log(n)))$
• $ n \in O(\log(n)\cdot\sqrt{n})$
• $ n + \log(n) \in \Theta(n)$

Prove that:

• $ \log(n\cdot \log(n))\in\Theta(\log(n))$
• $ \sqrt{n}\in\omega(\log(n))$
• $ f(n) + g(n) \in O(n\cdot\log(n))$ for $ f(n)\in\Theta(n)$ and $ g(n)\in O(n\cdot\log n)$

• $ \frac{O(n\sqrt{n})}{\Theta(n)} = \ldots$
• $ \frac{\Theta(n)}{O(\log(n))} = \ldots$
• $ \lvert{\Theta(f(n)) - \Theta(f(n))\rvert} = \ldots$

Prove/disprove the following:

• $ f(n) = \Omega(\log(n))$ and $ g(n)=O(n) \implies f(n)=\Omega(\log(g(n))$
• $ f(n) = \Omega(\log(n))$ and $ g(n)=O(n) \implies f(n)=\Theta(\log(g(n))$
• $ f(n) = \Omega(g(n))$ and $ g(n)=O(n^2) \implies \frac{g(n)}{f(n)}=O(n)$

Find two functions $ f$ and $ g$ such that:

• $ f(n) = O(g^2(n))$
• $ f(n) = \omega(\log(g(n)))$
• $ f(n) = \Omega(f(n)\cdot g(n))$
• $ f(n) = \Theta(g(n)) + \Omega(g^2(n))$