This shows you the differences between two versions of the page.
ep:labs:03:contents:tasks:ex5 [2020/08/10 23:09] cristian.marin0805 [05. [20p] Multitool Comparison] |
ep:labs:03:contents:tasks:ex5 [2022/10/24 00:11] (current) andrei.mirciu [03. [30p] RAM disk] |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | ==== 05. [20p] Multitool Comparison ==== | + | ==== 03. [30p] RAM disk ==== |
- | Now we will see in a slightly different approach, as for more special situations, a classic tool is not enough as long as we want to have a more detailed analysis of the behavior of I/O mechanisms. | + | |
+ | Linux allows you to use part of your RAM as a block device, viewing it as a hard disk partition. The advantage of using a RAM disk is the **extremely low latency** (even when compared to SSDs). The disadvantage is that all contents will be lost after a reboot. | ||
+ | <note tip> | ||
+ | There are two main types of RAM disks: | ||
+ | * **ramfs** - cannot be limited in size and will continue to grow until you run out of RAM. Its size can not be determined precisely with tools like **df**. Instead, you have to estimate it by looking at the "cached" entry from **free**'s output. | ||
+ | * **tmpfs** - newer than **ramfs**. Can set a size limit. Behaves exactly like a hard disk partition but can't be monitored through conventional means (i.e. **iostat**). Size can be precisely estimated using **df**. | ||
+ | </note> | ||
+ | === [15p] Task A - Create RAM Disk === | ||
- | === [10p] Task A - Different tool, same I/O === | + | Before getting started, let's find out the file system that our root partition uses. Run the following command (T - print file system type, h - human readable): |
- | + | ||
- | An example would be to create an infinite loop that **copies endlessly** (simulation of a demanding process for a long time) a **large file** (use one of the files obtained previously). | + | |
- | Put the command below in a script: | ||
<code bash> | <code bash> | ||
- | $ while true; do cp original_file1 copied_file2; done | + | $ df -Th |
+ | </code> | ||
+ | The result should look like this: | ||
+ | <code> | ||
+ | Filesystem Type Size Used Avail Use% Mounted on | ||
+ | udev devtmpfs 1.1G 0 1.1G 0% /dev | ||
+ | tmpfs tmpfs 214M 3.8M 210M 2% /run | ||
+ | /dev/sda1 ext4 218G 4.1G 202G 2% / <- root partition | ||
+ | tmpfs tmpfs 1.1G 252K 1.1G 1% /dev/shm | ||
+ | tmpfs tmpfs 5.0M 4.0K 5.0M 1% /run/lock | ||
+ | tmpfs tmpfs 1.1G 0 1.1G 0% /sys/fs/cgroup | ||
+ | /dev/sda2 ext4 923M 73M 787M 9% /boot | ||
+ | /dev/sda4 ext4 266G 62M 253G 1% /home | ||
</code> | </code> | ||
- | Use several special monitoring tools for I/O to investigate the state of the system. | + | From the results, we will assume in the following commands that the file system is **ext4**. If it's not your case, just replace with what you have: |
- | Choose 3-4 from here: | + | <code bash> |
- | [[https://www.golinuxcloud.com/monitor-disk-io-performance-statistics-linux/]] | + | $ sudo mkdir /mnt/ramdisk |
+ | $ sudo mount -t tmpfs -o size=1G ext4 /mnt/ramdisk | ||
+ | </code> | ||
- | === [10p] Task B - Plot of Comparison === | + | <note> |
- | + | If you want the RAM disk to persist after a reboot, you can add the following line to ///etc/fstab//. Remember that its contents will still be lost. | |
- | Plot a graph with the results obtained with iostat, iotop and one of the previously chosen tools. Interpret the graph and the values obtained using those tools. | + | |
- | <note> | + | |
- | * basic plot | + | |
- | * the values used should represent the same metric (eg: kb written per second) | + | |
- | * you can take the values manually or automatically | + | |
- | * standardize the values (kb/s) | + | |
- | * preferably use the matplotlib module in python | + | |
+ | <code> | ||
+ | tmpfs /mnt/ramdisk tmpfs rw,nodev,nosuid,size=1G 0 0 | ||
+ | </code> | ||
</note> | </note> | ||
- | Fill the data you obtained into the python3 script in {{:ep:labs:03:contents:tasks:plot.zip|}}. | + | That's it. We just created a 1Gb **tmpfs** ramdisk with an **ext4** file system and mounted it at ///mnt/ramdisk//. Use **df** again to check this yourself. |
- | Make sure you have **python3** and **python3-matplotlib** installed. | + | === [15p] Task B - Pipe View & RAM Disk === |
- | <solution -hidden> | + | As we mentioned before, you can't get I/O statistics regarding **tmpfs** since it is not a real partition. One solution to this problem is using **pv** to monitor the progress of data transfer through a pipe. This is a valid approach only if we consider the disk I/O being the bottleneck. |
- | <code python> | + | |
- | #!/usr/bin/python3 | + | |
- | import matplotlib.pyplot as plt | + | Next, we will generate 512Mb of random data and place it in ///mnt/ramdisk/file// first and then in ///home/student/file//. The transfer is done using **dd** with 2048-byte blocks. |
+ | <code bash> | ||
+ | $ pv /dev/urandom | dd of=/mnt/ramdisk/rand bs=2048 count=$((512 * 1024 * 1024 / 2048)) | ||
+ | $ pv /dev/urandom | dd of=/home/student/rand bs=2048 count=$((512 * 1024 * 1024 / 2048)) | ||
+ | </code> | ||
- | def main(): | + | Look at the elapsed time and average transfer speed. What conclusion can you draw? |
- | level = list(range(5)) | + | |
- | iostat = [12538, 16918, 22001, 24112, 28796] | + | |
- | iotop = [22138, 20111, 28011, 20819, 27940] | + | |
- | nmon = [24912, 29112, 28091, 27812, 29012] | + | |
- | # plot data | + | :!: Put one screenshot with the tmpfs partition in df output and one screenshot of both pv commands and write your conclusion. |
- | fig, axs = plt.subplots() | + | |
- | axs.plot(level, iotop, 'bo-') | ||
- | axs.plot(level, iostat, 'ro-') | ||
- | axs.plot(level, nmon, 'go-') | ||
- | |||
- | axs.set_xticks(level) | ||
- | axs.grid(True, which='both') | ||
- | |||
- | axs.set_xlabel('Time [ms]') | ||
- | axs.set_ylabel('Write [Kb/s]') | ||
- | |||
- | plt.show() | ||
- | |||
- | if __name__ == '__main__': | ||
- | main() | ||
- | </code> | ||
- | </solution> |