This shows you the differences between two versions of the page.
ep:labs:03:contents:tasks:ex5 [2020/08/03 20:04] cristian.marin0805 [05. [20p] Multitool Comparison] |
ep:labs:03:contents:tasks:ex5 [2022/10/24 00:11] (current) andrei.mirciu [03. [30p] RAM disk] |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | ==== 05. [20p] Multitool Comparison ==== | + | ==== 03. [30p] RAM disk ==== |
- | Now we will see in a slightly different approach, with a source file and a final file of the same size, how the I/O mechanisms work as they are implemented by the compiler to be optimized. | + | |
+ | Linux allows you to use part of your RAM as a block device, viewing it as a hard disk partition. The advantage of using a RAM disk is the **extremely low latency** (even when compared to SSDs). The disadvantage is that all contents will be lost after a reboot. | ||
+ | <note tip> | ||
+ | There are two main types of RAM disks: | ||
+ | * **ramfs** - cannot be limited in size and will continue to grow until you run out of RAM. Its size can not be determined precisely with tools like **df**. Instead, you have to estimate it by looking at the "cached" entry from **free**'s output. | ||
+ | * **tmpfs** - newer than **ramfs**. Can set a size limit. Behaves exactly like a hard disk partition but can't be monitored through conventional means (i.e. **iostat**). Size can be precisely estimated using **df**. | ||
+ | </note> | ||
+ | === [15p] Task A - Create RAM Disk === | ||
- | Modify the script so that it reads one line at a time from the file with the novel from the previous exercise, and writes it in a resulting file 500 times each. Follow the process with a classic tool (vmstat, iostat). Why most of the time the number of **bytes read** is **0**? | + | Before getting started, let's find out the file system that our root partition uses. Run the following command (T - print file system type, h - human readable): |
- | TODO COSMIN: Aici pui un hidden script cu rezolvarea pt asistenti cum e la 4, in python o pui, verifica sa fie astfel incat sa tina linia in RAM. | + | <code bash> |
+ | $ df -Th | ||
+ | </code> | ||
+ | The result should look like this: | ||
+ | <code> | ||
+ | Filesystem Type Size Used Avail Use% Mounted on | ||
+ | udev devtmpfs 1.1G 0 1.1G 0% /dev | ||
+ | tmpfs tmpfs 214M 3.8M 210M 2% /run | ||
+ | /dev/sda1 ext4 218G 4.1G 202G 2% / <- root partition | ||
+ | tmpfs tmpfs 1.1G 252K 1.1G 1% /dev/shm | ||
+ | tmpfs tmpfs 5.0M 4.0K 5.0M 1% /run/lock | ||
+ | tmpfs tmpfs 1.1G 0 1.1G 0% /sys/fs/cgroup | ||
+ | /dev/sda2 ext4 923M 73M 787M 9% /boot | ||
+ | /dev/sda4 ext4 266G 62M 253G 1% /home | ||
+ | </code> | ||
+ | From the results, we will assume in the following commands that the file system is **ext4**. If it's not your case, just replace with what you have: | ||
+ | <code bash> | ||
+ | $ sudo mkdir /mnt/ramdisk | ||
+ | $ sudo mount -t tmpfs -o size=1G ext4 /mnt/ramdisk | ||
+ | </code> | ||
- | === [10p] Task A - Different tool, same I/O === | + | <note> |
+ | If you want the RAM disk to persist after a reboot, you can add the following line to ///etc/fstab//. Remember that its contents will still be lost. | ||
- | For more special situations, a classic tool is not enough as long as we want to have a more detailed analysis on the behavior of I/O mechanisms. | + | <code> |
- | An example would be to create an infinite loop that **copies endlessly** (simulation of a demanding process for a long time) a **large file** (use one of the files obtained previously). | + | tmpfs /mnt/ramdisk tmpfs rw,nodev,nosuid,size=1G 0 0 |
- | + | ||
- | Put the command below in a script: | + | |
- | <code bash> | + | |
- | $ while true; do cp original_file1 copied_file2; done | + | |
</code> | </code> | ||
+ | </note> | ||
- | Use several special monitoring tools. | + | That's it. We just created a 1Gb **tmpfs** ramdisk with an **ext4** file system and mounted it at ///mnt/ramdisk//. Use **df** again to check this yourself. |
- | Choose 3-4 from here: | + | === [15p] Task B - Pipe View & RAM Disk === |
- | [[https://www.golinuxcloud.com/monitor-disk-io-performance-statistics-linux/]] | + | |
- | === [10p] Task B - Plot of Comparison === | + | As we mentioned before, you can't get I/O statistics regarding **tmpfs** since it is not a real partition. One solution to this problem is using **pv** to monitor the progress of data transfer through a pipe. This is a valid approach only if we consider the disk I/O being the bottleneck. |
- | Plot a graph with the results obtained with iostat, iotop and one of the previously chosen tools. | + | Next, we will generate 512Mb of random data and place it in ///mnt/ramdisk/file// first and then in ///home/student/file//. The transfer is done using **dd** with 2048-byte blocks. |
- | <note> | + | |
- | * basic plot | + | |
- | * you can take the values manually or automatically | + | |
- | * standardize the values (kb/s) | + | |
- | * preferably use the matplotlib module in python | + | |
- | </note> | + | <code bash> |
+ | $ pv /dev/urandom | dd of=/mnt/ramdisk/rand bs=2048 count=$((512 * 1024 * 1024 / 2048)) | ||
+ | $ pv /dev/urandom | dd of=/home/student/rand bs=2048 count=$((512 * 1024 * 1024 / 2048)) | ||
+ | </code> | ||
+ | |||
+ | Look at the elapsed time and average transfer speed. What conclusion can you draw? | ||
+ | |||
+ | :!: Put one screenshot with the tmpfs partition in df output and one screenshot of both pv commands and write your conclusion. | ||