Differences
This shows you the differences between two versions of the page.
|
ep:labs:05:contents:tasks:ex1 [2020/11/11 17:20] ioan_adrian.cosma [Google Colab Notebook] |
ep:labs:05:contents:tasks:ex1 [2026/03/30 22:15] (current) radu.mantu |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ==== Google Colab Notebook ==== | + | ==== 01. [20p] iostat & iotop ==== |
| + | === [10p] Task A - Monitoring the behaviour with Iostat === | ||
| + | <note tip> | ||
| + | Parameteres for iostat: | ||
| + | * -x for extended statistics | ||
| + | * -d to display device stastistics only | ||
| + | * -m for displaying r/w in MB/s | ||
| + | <code bash> | ||
| + | $ iostat -xdm | ||
| + | </code> | ||
| + | Use iostat with -p for specific device statistics: | ||
| + | <code bash> | ||
| + | $ iostat -xdm -p sda | ||
| + | </code> | ||
| + | </note> | ||
| - | For this lab, we will use Google Colab for exploring numpy and matplotlib. Please solve your tasks [[https://github.com/cosmaadrian/ml-environment/blob/master/EP_Plotting_I.ipynb|here]] by clicking "**Open in Colaboratory**". | + | * Run //iostat -x 1 5//. |
| + | * Considering the last two outputs provided by the previous command, calculate **the efficiency of IOPS** for each of them. Does the amount of data written per I/O **increase** or **decrease**? | ||
| - | You can then export this python notebook as a PDF (**File -> Print**) and upload it to **Moodle**. | + | Add in your archive screenshot or pictures of the operations and the result you obtained, also showing the output of iostat from which you took the values. |
| + | |||
| + | <note> | ||
| + | How to do: | ||
| + | |||
| + | * Divide the kilobytes read (//rkB/s//) and written (//wkB/s//) per second by the reads per second (//r/s//) and the writes per second (//w/s//). | ||
| + | * If you happen to have quite a few [[https://en.wikipedia.org/wiki/Loop_device|loop devices]] in your **iostat** output, find out what they are exactly: | ||
| + | |||
| + | <code bash> | ||
| + | $ df -kh /dev/loop* | ||
| + | </code> | ||
| + | </note> | ||
| + | |||
| + | |||
| + | <solution -hidden> | ||
| + | The way to calculate the efficiency of IOPS is to divide the reads per second //(r/s)// and writes per second //(w/s)// by the kilobytes read //(rkB/s)// and written //(wkB/s)// per second. | ||
| + | |||
| + | Example: the amount of data written per I/O for ///dev/sda// increases during each iteration: | ||
| + | |||
| + | {{ :ep:labs:ep2017_l3_ex01.png?700 |}} | ||
| + | |||
| + | <code> | ||
| + | 53040/105 = 505KB per I/O | ||
| + | 71152/102 = 697KB per I/O | ||
| + | </code> | ||
| + | |||
| + | If everything is zero in iostat - perform some I/O operations... | ||
| + | </solution> | ||
| + | |||
| + | |||
| + | === [10p] Task B - Monitoring the behaviour with Iotop === | ||
| + | <note tip> | ||
| + | **Iotop** is an utility similar to top command, that interfaces with the kernel to provide per-thread/process I/O usage statistics. | ||
| + | |||
| + | <code> | ||
| + | Debian/Ubuntu Linux install iotop | ||
| + | $ sudo apt-get install iotop | ||
| + | |||
| + | How to use iotop command | ||
| + | $ sudo iotop OR $ iotop | ||
| + | </code> | ||
| + | |||
| + | Supported options by iotop command: | ||
| + | |||
| + | | **Options** | **Description** ^^ | ||
| + | | --version | show program’s version number and exit || | ||
| + | | -h, --help | show this help message and exit || | ||
| + | | -o, --only | only show processes or threads actually doing I/O || | ||
| + | | -b, --batch | non-interactive mode || | ||
| + | | -n NUM, --iter=NUM | number of iterations before ending [infinite] || | ||
| + | | -d SEC, --delay=SEC | delay between iterations [1 second] || | ||
| + | | -p PID, --pid=PID | processes/threads to monitor [all] || | ||
| + | | -u USER, --user=USER | users to monitor [all] || | ||
| + | | -P, --processes | only show processes, not all threads || | ||
| + | | -a, --accumulated | show accumulated I/O instead of bandwidth || | ||
| + | | -k, --kilobytes | use kilobytes instead of a human friendly unit || | ||
| + | | -t, --time | add a timestamp on each line (implies –batch) || | ||
| + | | -q, --quiet | suppress some lines of header (implies –batch) || | ||
| + | </note> | ||
| + | |||
| + | |||
| + | * Run iotop (install it if you do not already have it) in a separate shell showing only processes or threads actually doing I/O. | ||
| + | * Inspect the script code ({{:ep:laboratoare:dummy.sh|dummy.sh}}) to see what it does. | ||
| + | * Monitor the behaviour of the system with iotop while running the script. | ||
| + | * Identify the PID and PPID of the process running the dummy script and kill the process using command line from another shell (sending SIGINT signal to both parent & child processes). | ||
| + | * Hint - [[https://superuser.com/questions/150117/how-to-get-parent-pid-of-a-given-process-in-gnu-linux-from-command-line|How to get parent PID of a given process in GNU/Linux from command line?]] | ||
| + | |||
| + | <solution -hidden> | ||
| + | {{:ep:laboratoare:lab3-ex4.png?600}} | ||
| + | |||
| + | **dd** performs disk writes... wow! Reason why there are no disk reads is because it uses pseudo-files from **devtmpfs** that don't record these statistics. Also, **tmpfs** are not disks :p | ||
| + | |||
| + | Find PPID from PID: ps -o ppid= -p PID | ||
| + | Send SIGINT signal: kill -SIGINT PID,PPID | ||
| + | </solution> | ||
