Edit this page Backlinks This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong. ====== Asymptotic Notations ====== * Contacts: [[mihai.dumitru2201@stud.acs.upb.ro|Dumitru Mihai-Valentin]], [[matei.popovici@cs.pub.ro|Matei Popovici]] * Published: 9.11.2015 ===== Outline ===== An **execution time** is a function $T:\mathbb{N}\rightarrow\mathbb{N}$, which maps a value $n$ representing **the size of the input** of an algorithm to the **number of instructions** (/**execution steps**) performed by the algorithm. Suppose $f: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}$ is a non-necessarily monotone function over positive reals. We define the sets $X(f(n)) \subseteq \mathbb{H}om(\mathbb{R}^{+},\mathbb{R}^{+})$, with $X\in\{O,o,\Theta,\omega,\Omega\}$, which we call **asymptotic notations** (or simply notations). **Question**: Why are asymptotic notations defined for positive real functions, instead of positive integers? (as in execution times) ===== Notations ===== ==== $\Theta$ (Theta) Notation ==== $$ \Theta(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \exists c_1, c_2 \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ c_1f(n) \le g(n) \le c_2f(n) \}$$ {{ :labs:theta_fn.png |}} We say that $\Theta(f(n))$ is the class (set) of functions with //the same asymptotic growth as// f(n). === Example === Let $f(n) = n^2 + 8$ and $g(n) = 3n^2 + 4n + 2$\\ We see that, for $n_0 = 1, c_1 = 1, c_2 = 4$:\\ $n^2 + 8 \le 3n^2 + 4n + 2 \le 4n^2 + 32$\\ thus $g(n) \in \Theta(f(n))$ ==== $O$ (Big O) Notation ==== $$ O(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \exists c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le g(n) \le cf(n) \}$$ {{ :labs:o_fn.png |}} We say that $O(f(n))$ is the class (set) of functions that grow asymptotically //at most as much as// $f(n)$. === Example === Let $f(n) = nlog(n) + n$ and $g(n) = nlog(n) + n + 10000$\\ We see that, for $n_0 = 10000, c = 2$:\\ $0 \le nlog(n) + n + 10000 \le 2nlog(n) + 2n$\\ thus $g(n) \in O(f(n))$ ==== $\Omega$ (Big Omega) Notation ==== $$ \Omega(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \exists c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le cf(n) \le g(n) \}$$ {{ :labs:omega_fn.png |}} We say that $\Omega(f(n))$ is the class (set) of functions that grow asymptotically //at least as much as// $f(n)$. === Example === Let $f(n) = n^4$ and $g(n) = 2^n$\\ We see that, for $n_0 = 16, c = 1$:\\ $0 \le n^4 \le 2^n$\\ thus $g(n) \in \Omega(f(n))$ ==== $o$ (Little O) Notation ==== $$ o(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \forall c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le g(n) \le cf(n) \}$$ We say that $o(f(n))$ is the class (set) of functions that grows asymptotically //strictly less than// $f(n)$. === Example === Let $f(n) = n^2 + 3n + 6$ and $g(n) = 8n + 3$\\ We see that, for $c = 1, n_0 = 5$:\\ $0 \le 8n + 3 \le n^2 + 3n + 6$\\ \\ for $c = 0.5, n_0 = 7$:\\ $0 \le 8n + 3 \le 0.5n^2 + 1.5n + 3$\\ \\ for $c = 0.1, n_0 = 77$:\\ $0 \le 8n + 3 \le 0.1n^2 + 0.3n + 0.6$\\ etc. thus $g(n) \in o(f(n))$ ==== $\omega$ (Little Omega) Notation ==== $$ \omega(f(n)) = \{ g: \mathbb{R}^{+} \rightarrow \mathbb{R}^{+}\ |\ \begin{array}{l} \forall c \in \mathbb{R}^+\cr \exists n_0 \in \mathbb{N} \end{array}\ such\ that\ \forall\ n \ge n_0,\ \ 0 \le cf(n) \le g(n) \}$$ We say that $\omega(f(n))$ is the class (set) of functions that grows asymptotically //strictly more than// $f(n)$. === Example === Let $f(n) = log(n)$ and $g(n) = n$\\ We see that, for $c = 1, n_0 = 1$:\\ $0 \le log(n) \le n$\\ \\ for $c = 10, n_0 = 64$:\\ $0 \le 10log(n) \le n$\\ \\ for $c = 100, n_0 = 1024$:\\ $0 \le 100log(n) \le n$\\ etc.\\ thus $g(n) \in \omega(f(n))$ ===== Syntactic sugars ====== Asymptotic notations are often used to refer to arbitrary functions that have a certain growth. To simplify things, we can write //arithmetic expressions// as follows: $$f(n) = \Theta(n) + O(log(n))$$\\ which should be read as: $ \exists g \in \Theta(n)$ and $ \exists h \in O(log(n))$ such that $f(n) = g(n) + h(n),\ \forall n \in \mathbb{R}^{+}$. We can also write //equations// like the following: $$\Theta(n^2) = O(n^2) + o(n)$$\\ which should be read as:\\ $ \forall f \in \Theta(n^2),\ \exists g \in O(n^2)$ and $h \in o(n)$ such that $f(n) = g(n) + h(n),\ \forall n \in \mathbb{R}^{+}$. Note that equations are not symmetric and should only be read from left to right. Consider: $$\Theta(n) = O(n)$$ While it is true that, for any function in $\Theta(n)$ there is a function equal to it in $O(n)$, we can clearly see that there are functions in $O(n)$ for which there is no correspondent in $\Theta(n)$ (e.g. $f(n) = 1$, $f(n) = log(n)$ etc.) As a rule, each asymptotic notation on the left side of the equal sign should be read as an **universally quantified** function ($ \forall f$) from that class and each asymptotic notation on the right should be read as an **existentially quantified** function ($ \exists g$) from that class. $$\left(\frac{\omega(n^2)}{\Theta(n)}\right) = \Omega(n) + o(n)$$ $ \forall f \in \omega(n^2)\ and\ \forall g \in \Theta(n),\ \exists h \in \Omega(n)\ and\ \exists j \in o(n)\ such\ that$ $\left(\frac{f(n)}{g(n)}\right) = h(n) + j(n),\ \forall n \in \mathbb{R}^{+}$ ===== (Counter)intuitive examples ===== Which of the following is true: * $o(f(n)) \cap \omega(f(n)) = \emptyset$ * $o(f(n)) \subsetneq O(f(n))$ * $O(f(n)) \setminus o(f(n)) = \Theta(f(n))$ ===== Exercises (asymptotic notations) ===== Check the truth of the following: * $\sqrt{n} \in O(\log(n))$ * $log(n) \in O(\log(\log(n)))$ * $n \in O(\log(n)\cdot\sqrt{n})$ * $n + \log(n) \in \Theta(n)$ Prove that: * $\log(n\cdot \log(n))\in\Theta(\log(n))$ * $\sqrt{n}\in\omega(\log(n))$ * $f(n) + g(n) \in O(n\cdot\log(n))$ for $f(n)\in\Theta(n)$ and $g(n)\in O(n\cdot\log n)$ ===== Exercises (syntactic sugars) ===== * $\frac{O(n\sqrt{n})}{\Theta(n)} = \ldots$ * $\frac{\Theta(n)}{O(\log(n))} = \ldots$ * $|\Theta(f(n)) - \Theta(f(n))| = \ldots$ ===== Practice ===== Prove/disprove the following: * $f(n) = \Omega(\log(n))$ and $g(n)=O(n)$ $\implies$ $f(n)=\Omega(\log(g(n))$ * $f(n) = \Omega(\log(n))$ and $g(n)=O(n)$ $\implies$ $f(n)=\Theta(\log(g(n))$ * $f(n) = \Omega(g(n))$ and $g(n)=O(n^2)$ $\implies$ $\frac{g(n)}{f(n)}=O(n)$ Find two functions $f$ and $g$ such that: * $f(n) = O(g^2(n))$ * $f(n) = \omega(\log(g(n)))$ * $f(n) = \Omega(f(n)\cdot g(n))$ * $f(n) = \Theta(g(n)) + \Omega(g^2(n))$