Differences
This shows you the differences between two versions of the page.
|
ep:labs:03:contents:tasks:ex5 [2020/08/05 20:45] gheorghe.petre2608 [05. [20p] Multitool Comparison] |
ep:labs:03:contents:tasks:ex5 [2022/10/24 00:11] (current) andrei.mirciu [03. [30p] RAM disk] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ==== 05. [20p] Multitool Comparison ==== | + | ==== 03. [30p] RAM disk ==== |
| - | Now we will see in a slightly different approach, with a source file and a final file of the same size, how the I/O mechanisms work as they are implemented by the compiler to be optimized. | + | |
| - | **Nu cred ca are sens chestia asta. Putem sa lasam direct cele 2 task-uri.** | + | Linux allows you to use part of your RAM as a block device, viewing it as a hard disk partition. The advantage of using a RAM disk is the **extremely low latency** (even when compared to SSDs). The disadvantage is that all contents will be lost after a reboot. |
| + | <note tip> | ||
| + | There are two main types of RAM disks: | ||
| + | * **ramfs** - cannot be limited in size and will continue to grow until you run out of RAM. Its size can not be determined precisely with tools like **df**. Instead, you have to estimate it by looking at the "cached" entry from **free**'s output. | ||
| + | * **tmpfs** - newer than **ramfs**. Can set a size limit. Behaves exactly like a hard disk partition but can't be monitored through conventional means (i.e. **iostat**). Size can be precisely estimated using **df**. | ||
| + | </note> | ||
| + | === [15p] Task A - Create RAM Disk === | ||
| + | Before getting started, let's find out the file system that our root partition uses. Run the following command (T - print file system type, h - human readable): | ||
| - | (Modify the script so that it reads one line at a time from the file with the novel from the previous exercise, and writes it in a resulting file 500 times each. Follow the process with a classic tool (vmstat, iostat). Why most of the time the number of **bytes read** is **0**? | + | <code bash> |
| + | $ df -Th | ||
| + | </code> | ||
| + | The result should look like this: | ||
| + | <code> | ||
| + | Filesystem Type Size Used Avail Use% Mounted on | ||
| + | udev devtmpfs 1.1G 0 1.1G 0% /dev | ||
| + | tmpfs tmpfs 214M 3.8M 210M 2% /run | ||
| + | /dev/sda1 ext4 218G 4.1G 202G 2% / <- root partition | ||
| + | tmpfs tmpfs 1.1G 252K 1.1G 1% /dev/shm | ||
| + | tmpfs tmpfs 5.0M 4.0K 5.0M 1% /run/lock | ||
| + | tmpfs tmpfs 1.1G 0 1.1G 0% /sys/fs/cgroup | ||
| + | /dev/sda2 ext4 923M 73M 787M 9% /boot | ||
| + | /dev/sda4 ext4 266G 62M 253G 1% /home | ||
| + | </code> | ||
| - | TODO COSMIN: Aici pui un hidden script cu rezolvarea pt asistenti cum e la 4, in python o pui, verifica sa fie astfel incat sa tina linia in RAM.) | + | From the results, we will assume in the following commands that the file system is **ext4**. If it's not your case, just replace with what you have: |
| - | |||
| - | |||
| - | === [10p] Task A - Different tool, same I/O === | ||
| - | |||
| - | For more special situations, a classic tool is not enough as long as we want to have a more detailed analysis of the behavior of I/O mechanisms. | ||
| - | An example would be to create an infinite loop that **copies endlessly** (simulation of a demanding process for a long time) a **large file** (use one of the files obtained previously). | ||
| - | |||
| - | Put the command below in a script: | ||
| <code bash> | <code bash> | ||
| - | $ while true; do cp original_file1 copied_file2; done | + | $ sudo mkdir /mnt/ramdisk |
| + | $ sudo mount -t tmpfs -o size=1G ext4 /mnt/ramdisk | ||
| </code> | </code> | ||
| - | Use several special monitoring tools for I/O to investigate the state of the system. | + | <note> |
| - | + | If you want the RAM disk to persist after a reboot, you can add the following line to ///etc/fstab//. Remember that its contents will still be lost. | |
| - | Choose 3-4 from here: | + | |
| - | [[https://www.golinuxcloud.com/monitor-disk-io-performance-statistics-linux/]] | + | |
| - | + | ||
| - | === [10p] Task B - Plot of Comparison === | + | |
| - | + | ||
| - | Plot a graph with the results obtained with iostat, iotop and one of the previously chosen tools. Interpret the graph and the values obtained using those tools. | + | |
| - | <note> | + | |
| - | * basic plot | + | |
| - | * the values used should represent the same metric (eg: kb written per second) | + | |
| - | * you can take the values manually or automatically | + | |
| - | * standardize the values (kb/s) | + | |
| - | * preferably use the matplotlib module in python | + | |
| + | <code> | ||
| + | tmpfs /mnt/ramdisk tmpfs rw,nodev,nosuid,size=1G 0 0 | ||
| + | </code> | ||
| </note> | </note> | ||
| - | <solution -hidden> | + | That's it. We just created a 1Gb **tmpfs** ramdisk with an **ext4** file system and mounted it at ///mnt/ramdisk//. Use **df** again to check this yourself. |
| - | <code python> | + | |
| - | #!/usr/bin/python3 | + | |
| - | import matplotlib.pyplot as plt | + | === [15p] Task B - Pipe View & RAM Disk === |
| + | As we mentioned before, you can't get I/O statistics regarding **tmpfs** since it is not a real partition. One solution to this problem is using **pv** to monitor the progress of data transfer through a pipe. This is a valid approach only if we consider the disk I/O being the bottleneck. | ||
| - | def main(): | + | Next, we will generate 512Mb of random data and place it in ///mnt/ramdisk/file// first and then in ///home/student/file//. The transfer is done using **dd** with 2048-byte blocks. |
| - | level = list(range(5)) | + | |
| - | iostat = [12538, 16918, 22001, 24112, 28796] | + | |
| - | iotop = [22138, 20111, 28011, 20819, 27940] | + | |
| - | nmon = [24912, 29112, 28091, 27812, 29012] | + | |
| - | # plot data | + | <code bash> |
| - | fig, axs = plt.subplots() | + | $ pv /dev/urandom | dd of=/mnt/ramdisk/rand bs=2048 count=$((512 * 1024 * 1024 / 2048)) |
| + | $ pv /dev/urandom | dd of=/home/student/rand bs=2048 count=$((512 * 1024 * 1024 / 2048)) | ||
| + | </code> | ||
| - | axs.plot(level, iotop, 'bo-') | + | Look at the elapsed time and average transfer speed. What conclusion can you draw? |
| - | axs.plot(level, iostat, 'ro-') | + | |
| - | axs.plot(level, nmon, 'go-') | + | |
| - | axs.set_xticks(level) | + | :!: Put one screenshot with the tmpfs partition in df output and one screenshot of both pv commands and write your conclusion. |
| - | axs.grid(True, which='both') | + | |
| - | axs.set_xlabel('Time [ms]') | ||
| - | axs.set_ylabel('Write [Kb/s]') | ||
| - | |||
| - | plt.show() | ||
| - | |||
| - | if __name__ == '__main__': | ||
| - | main() | ||
| - | </code> | ||
| - | </solution> | ||
