Differences
This shows you the differences between two versions of the page.
|
ep:labs:03:contents:tasks:ex1 [2020/08/03 16:18] cristian.marin0805 [03. [5p] Iotop] |
ep:labs:03:contents:tasks:ex1 [2021/10/26 10:46] (current) radu.mantu [01. [10p] Rotational delay - IOPS calculations] |
||
|---|---|---|---|
| Line 10: | Line 10: | ||
| </note> | </note> | ||
| To calculate the **IOPS range** divide 1 by the sum of the average latency in ms and the average seek time in ms. The formula is: | To calculate the **IOPS range** divide 1 by the sum of the average latency in ms and the average seek time in ms. The formula is: | ||
| - | <code>average IOPS = (1 / (average latency in ms + average seek time in ms). | + | <code>average IOPS = 1 / (average latency in ms + average seek time in ms). |
| </code> | </code> | ||
| Line 25: | Line 25: | ||
| === [10p] Task A - Calculate rotational delay === | === [10p] Task A - Calculate rotational delay === | ||
| - | Calculate the rotational delay (RD) for a 5400 RPM drive. | + | Add in your archive the operations and the result you obtained. (Screenshot, picture of calculations made by hand on paper) |
| + | |||
| + | Calculate the Rotational Delay, and then the IOPS for a __5400 RPM drive__. | ||
| <solution -hidden> | <solution -hidden> | ||
| Line 39: | Line 41: | ||
| </code> | </code> | ||
| </solution> | </solution> | ||
| - | ==== 02. [10p] Iostat ==== | ||
| - | |||
| - | <note tip> | ||
| - | Parameteres for iostat: | ||
| - | * -x for extended statistics | ||
| - | * -d to display device stastistics only | ||
| - | * -m for displaying r/w in MB/s | ||
| - | <code bash> | ||
| - | $ iostat -xdm | ||
| - | </code> | ||
| - | Use iostat with -p for specific device statistics: | ||
| - | <code bash> | ||
| - | $ iostat -xdm -p sda | ||
| - | </code> | ||
| - | </note> | ||
| - | |||
| - | === [10p] Task A - Monitoring the behaviour === | ||
| - | * Run //iostat -x 1 5//. | ||
| - | * Considering the last two outputs provided by the previous command, calculate **the efficiency of IOPS** for each of them. Does the amount of data written per I/O **increase** or **decrease**? | ||
| - | |||
| - | <note> | ||
| - | How to do: | ||
| - | |||
| - | * Divide the kilobytes read (//rkB/s//) and written (//wkB/s//) per second by the reads per second (//r/s//) and the writes per second (//w/s//). | ||
| - | * If you happen to have quite a few [[https://en.wikipedia.org/wiki/Loop_device|loop devices]] in your **iostat** output, find out what they are exactly: | ||
| - | |||
| - | <code bash> | ||
| - | $ df -kh /dev/loop* | ||
| - | </code> | ||
| - | </note> | ||
| - | |||
| - | |||
| - | <solution -hidden> | ||
| - | The way to calculate the efficiency of IOPS is to divide the reads per second //(r/s)// and writes per second //(w/s)// by the kilobytes read //(rkB/s)// and written //(wkB/s)// per second. | ||
| - | |||
| - | Example: the amount of data written per I/O for ///dev/sda// increases during each iteration: | ||
| - | |||
| - | {{ :ep:labs:ep2017_l3_ex01.png?700 |}} | ||
| - | |||
| - | <code> | ||
| - | 53040/105 = 505KB per I/O | ||
| - | 71152/102 = 697KB per I/O | ||
| - | </code> | ||
| - | |||
| - | If everything is zero in iostat - perform some I/O operations... | ||
| - | </solution> | ||
| - | ==== 03. [10p] Iotop ==== | ||
| - | |||
| - | === Good to know === | ||
| - | |||
| - | **Iotop** is an utility similar to top command, that interfaces with the kernel to provide per-thread/process I/O usage statistics. | ||
| - | |||
| - | <code> | ||
| - | Debian/Ubuntu Linux install iotop | ||
| - | $ sudo apt-get install iotop | ||
| - | |||
| - | How to use iotop command | ||
| - | $ sudo iotop OR $ iotop | ||
| - | </code> | ||
| - | |||
| - | Supported options by iotop command: | ||
| - | |||
| - | | **Options** | **Description** ^^ | ||
| - | | --version | show program’s version number and exit || | ||
| - | | -h, --help | show this help message and exit || | ||
| - | | -o, --only | only show processes or threads actually doing I/O || | ||
| - | | -b, --batch | non-interactive mode || | ||
| - | | -n NUM, --iter=NUM | number of iterations before ending [infinite] || | ||
| - | | -d SEC, --delay=SEC | delay between iterations [1 second] || | ||
| - | | -p PID, --pid=PID | processes/threads to monitor [all] || | ||
| - | | -u USER, --user=USER | users to monitor [all] || | ||
| - | | -P, --processes | only show processes, not all threads || | ||
| - | | -a, --accumulated | show accumulated I/O instead of bandwidth || | ||
| - | | -k, --kilobytes | use kilobytes instead of a human friendly unit || | ||
| - | | -t, --time | add a timestamp on each line (implies –batch) || | ||
| - | | -q, --quiet | suppress some lines of header (implies –batch) || | ||
| - | |||
| - | === Monitoring the behaviour === | ||
| - | |||
| - | Datafile: {{:ep:laboratoare:dummy.sh|dummy.sh}}. | ||
| - | |||
| - | * Run iotop (install it if you do not already have it) in a separate shell showing only processes or threads actually doing I/O. | ||
| - | * Inspect the script code (**dummy.sh**) to see what it does. | ||
| - | * Monitor the behaviour of the system with iotop while running the script. | ||
| - | * Identify the PID and PPID of the process running the dummy script and kill the process using command line from another shell (sending SIGINT signal to both parent & child processes). | ||
| - | * Hint - [[https://superuser.com/questions/150117/how-to-get-parent-pid-of-a-given-process-in-gnu-linux-from-command-line|How to get parent PID of a given process in GNU/Linux from command line?]] | ||
| - | |||
| - | <solution -hidden> | ||
| - | {{:ep:laboratoare:lab3-ex4.png?600}} | ||
| - | |||
| - | Find PPID from PID: ps -o ppid= -p PID | ||
| - | |||
| - | Send SIGINT signal: kill -SIGINT PID,PPID | ||
| - | </solution> | ||
| - | ==== 04. [20p] Monitor I/O with vmstat and iostat ==== | ||
| - | We said in the beginning that the disk I/O subsystems are the slowest part of any system. This is why the I/O monitoring is so important, maximizing the performance of the slowest part of a system resulting in an improvement of the performance of the entire system. | ||
| - | |||
| - | === [10p] Task A - Script === | ||
| - | |||
| - | Write a script that reads the data into memory and generates a text file 500 times larger, by concatenating the contents of the following novel {{:ep:labs:olivertwist.txt|olivertwist.txt}} to itself. | ||
| - | |||
| - | <solution -hidden> | ||
| - | <code> | ||
| - | if __name__ == '__main__': | ||
| - | text_file1 = open("OliverTwist.txt", "r") | ||
| - | text_file2 = open("OliverTwistLarge.txt", "w+") | ||
| - | lines_file1 = text_file1.readlines() | ||
| - | for x in range(0, 500): | ||
| - | text_file2.writelines(lines_file1) | ||
| - | </code> | ||
| - | </solution> | ||
| - | |||
| - | === [10p] Task B - Monitoring behaviour === | ||
| - | |||
| - | Now we want to analyze what is happening with the I/O subsystem during an expensive operation. Monitor the behavior of the system while running your script using **vmstat** and **iostat**. | ||
| - | |||
| - | <note tip> | ||
| - | Understanding vmstat IO section: | ||
| - | * **bi** - column reports the number of blocks received (or “blocks in”) from a disk per second. | ||
| - | * **bo** - column reports the number of blocks sent (“blocks out”) to a disk per second. | ||
| - | </note> | ||
