Show page

Differences

This shows you the differences between two versions of the page.

--- ep:labs:08 [2021/10/05 16:01]
cezar.craciunoiu Change old archive
+++ ep:labs:08 [2025/02/12 00:01] (current)
cezar.craciunoiu
@@ Line 1: / Line 1: @@
-====== Lab 08 - I/O Monitoring (Windows) ======
+====== Lab 08 - TBD ======
-<note important>
-You can download the **Windows 10 VM** via [[https://ctipub-my.sharepoint.com/:u:/g/personal/radu_mantu_upb_ro/EXSrHQMCkWBEpGYseFEmnnABCA1hyb1oGWMUhnnHx8LIdQ?e=I0pxHg | OneDrive]].
-There is also the option to download as a torrent {{:ep:labs:ep_win10_vm.7z.torrent.txt}}.
-DokuWiki is not configured to accept //.torrent// files so remove the //.txt// extension.
-After that, you know what to do...
-Alternatively, you can install the following on your own Windows machine:
-  * **[[https://go.microsoft.com/fwlink/?linkid=2120254 | ADK]]** - make sure to check //**Windows Performance Analyser**// and //**Windows Performance Recorder**//.
-  * **[[https://visualstudio.microsoft.com/downloads/ | Visual Studio Community Edition]]** - select //C++ development//.
-  * **[[https://docs.microsoft.com/en-us/sysinternals/downloads/sysinternals-suite | Sysinternals suite]]**
-</note>
-<note important>
- If Visual Studio prompts you with an "Expired" message, you will have to log in with your (university) account.
-</note>
 ===== Objectives =====
-  * Offer an introduction to Windows I/O monitoring.
+  * TODO
-  * Get you acquainted with a few Windows standard monitoring tools like **Task Manager**, **Windows Performance Recorder**, **Process Monitor**, and **Process Explorer**.
-  * Learn how to monitor disk activity, identify what is generating it, and figure out what the issue is by looking at the pdbs and the code.
-  * Take a deeper look into how monitoring tools extract data from processes.
-===== Introduction =====
+===== Contents =====
-As you remember from the Linux Monitoring labs there is an endless list of tools for system analysis.
+TODO
-This is unfortunately not true for Windows.
-The system is closed-source and the development of tools is much slower.
-The first (and probably most popular), set of tools for system analysis is [[https://docs.microsoft.com/en-us/sysinternals/ | Sysinternals]].
-This was later aquired by Microsoft and it is now their recommended tool for analysis.
-The suite contains a wide variety of tools, but we will only concentrate on the widely used ones.
-==== 01. Task Manager ====
- Shows real time information about processes and the system.
- To start Task Manager you can use any of the following methods:
-  * //**Ctrl + Shift + Esc**//
-  * Right click the taskbar and choose Task Manager
-  * //**Ctrl + Alt + Del**// and select Task Manager
- **Tabs description:**
-   * **Processes** - shows all the running processes and their current resource usage in terms.
-   * **Performance** - shows the usage level of the computer's main resources in the last minute.
-   * **App history** - added with Windows 8, it shows the resource consumption of metro applications.
-   * **Startup** - shows all the applications that start at start-up and their impact on the boot time.
-   * **Users** - shows the resource consumption of every logged in user.
-   * **Details** - shows detailed information about each process. Right-clicking the column headers bar, offers the possibility to add or remove columns. The following columns: Handles, Threads, Image Path Name, and Command Line are useful for especially useful for this laboratory.
-   * **Services** - shows the service status for all services. A Windows service can be considered similar to a Linux daemon: a process without a visual interface, offering services to user-created processes.
- **Conclusions:**
-   * Task Manager can be used to identify which process uses a lot of RAM, CPU, accesses the disk many times or generates a lot of traffic on the network at a certain moment.
-   * It does offer some information for longer periods of time, in the Startup tab, which shows what process had higher impact at startup, but does not specify the area that was impacted.
-   * You can sort by I/O read or I/O Writes, but no there is no option to sort the results by Total I/O (combined Read & Write).
- <note>
- To overcome Task Manager’s limitations, and to perform a thorough analysis, use the Resource Monitor (Resmon) utility, which is built into Windows.
- </note>
-==== 02. Windows Performance Recorder & Analyzer ====
- Windows Performance Recorder (WPR) is used to record the whole activity of the system in a time frame.
- Compared to Task Manager, this tool only captures information, without displaying it.
- To inspect the captured data you will need to use another tool, Windows Performance Analyzer (WPA).
- This combination of tools is most useful when running tests that take hours and constantly watching Task Manager would be impossible.
-==== 03. Process Monitor ====
- Process Monitor is another troubleshooting tool from Windows Sysinternals that displays the files and registry keys that applications access in real-time.
- The results can be saved to a log file, which you can send it to an expert for analyzing a problem and troubleshooting it.
-**How to Use Process Monitor to Track Registry and File System Changes?**
- We want to write to the HOSTS file (C:\Windows\System32\drivers\etc\hosts) in order to add new rules.
- When we try to do this we encounter an error when saving the file.
- Following the steps below (or this [[https://drive.google.com/file/d/1ZYdtOq7QsY0nfYDS3e4foPFRVPgO4Qdb/view | video]]) we can record what causes the error.
- Afterwards we can send it to an expert or search for a fix ourselves.
-^ Part 1: Running Process Monitor & Configuring Filters ^^
-| **1.** Run the Process Monitor application. ||
-| **2.** Include the processes that you want to track the activity on. For this example, you want to include Notepad.exe in the (Include) Filters. ||
-| **3.** Click Add, and click OK. ||
-| **4.** From the Options menu, click Select Columns. ||
-| **5.** Under “Event Details”, enable Sequence Number, and click OK. ||
-<note>
- You can add multiple entries as well, in case you want to track more processes along with Notepad.exe.
- To keep this example simpler, let’s only track Notepad.exe.
- You’ll now see the Process Monitor main window tracking the list of registry and file accesses by processes real-time, as and when they occur.
-</note>
-^ Part 2: Capturing Events ^^
-| **6.** Open Notepad. ||
-| **7.** Switch to Process Monitor window. ||
-| **8.** Enable the “Capture” mode (if it’s not already ON). You can see the status of the “Capture” mode via the Process Monitor toolbar. ||
-| **9.** The highlighted button above is the “Capture” button, which is currently disabled. You need to click that button to enable capturing of events. ||
-| **10.** **Important**: Cleanup the existing events list using Ctrl + X key sequence and start afresh. ||
-| **11.** Switch back to Notepad. ||
-| **12.** To reproduce the problem, try writing to the HOSTS file and saving it. Windows offers to save the file with a different name, or in a different location. So, what happens under the hood when you save to HOSTS file? Process Monitor shows that exactly. ||
-| **13.** Switch to Process Monitor window, and turn off Capturing (Ctrl + E) as soon as you encountered the problem. **Important Note**: You need to do all that as quickly as you can in order to not record unneeded data. ||
-<note warning>
- The log file above tells us that Notepad encountered an ACCESS DENIED error when writing to the HOSTS file.
- The solution would be to simply run Notepad elevated (right-click and choose “Run as Administrator”) to be able to write to HOSTS file successfully.
-</note>
-^ Part 3: Saving the Output ^^
-| **14.** In the Process Monitor window, select the File menu and click Save. ||
-| **15.** Select Native Process Monitor Format (PML), mention the output file name and Path, save the file. ||
-| **16.** Right-click on the Logfile.PML file, click Send To, and choose Compressed (zipped) folder. You can now send it to an expert. ||
-<note>
- To recap, Task Manager shows what processes use the disk intensively at the current time.
- Windows Performance Recorder / Windows Performance Analyzer show who used the disk during a longer time period, although they were showing the activity as belonging to the System process instead of our process.
- Using Process Monitor we could identify our processes' entire activity and we could determine why one is slower than the other.
-</note>
-==== 04. Process Explorer ====
- Process Explorer is similar to Task Manager in many ways, as both serve the same purpose.
- Process Explorer is more verbose and shows much more information about different parts of the system.
- Even if it doesn't look as pretty as Task Manager, this tool was developed for Windows 2000 initially.
- The Task Manager of Windows 2000 offered much fewer options than the one for Windows 10.
- Now, Task Manager and Process Explorer are interchangeable in most cases.
-==== 05. Windows API ====
- The previous chapters cover most cases where we encounter an error, we diagnose it, and identify it.
- There is always the very rare case where a tool just doesn't cut it.
- In this case we can use the API offered by Windows to extract what information we want from a program and/or the system.
- More precisely, we are interested in the [[https://docs.microsoft.com/en-us/windows/win32/psapi/process-status-helper | Process Status API]] from Windows.
- This offers basic functionality to extract information from the system and its processes.
- We will only briefly go over the functions in the Task, so feel free to try more of the functionality of the API.
 ===== Tasks =====
-<note warning>
- The tasks can be found for the Windows sessions can be found here:
-  * New Tasks: {{:ep:laboratoare:lab08-tasks.zip|}}
-  * Old Tasks (before 2021): {{:ep:laboratoare:lab08-tasks-old.zip|}}
-</note>
-==== 01. Task Manager ====
- Go through the Task Manager introduction and/or watch this video containing a short [[https://drive.google.com/file/d/1z1J6lgoYfBOZF7acEzR8gEq1MH1OZgaf/view | tutorial]].
- :!: **Which process is constantly reading or writing to your hard disk?**
- <note tip>
-  - Open Task Manager and select the Details tab
-  - Right-click on the column header and click "Select columns"
-  - Enable the "I/O Read Bytes" and "I/O Write Bytes" checkboxes
-  - Sort the list by clicking on a column header to see the most read-intensive and/or write-intensive processes
-  - Identify the process and mention if and why the I/O traffic is justified
- </note>
- <note>
-  * **I/O Read Bytes** is the number of bytes read in input/output operations generated by a process, including file, network, and device I/Os.
-  * **I/O Write Bytes** is the number of bytes written in input/output operations by a process, including file, network, and device I/Os.
-  * **I/O Read Bytes** and **I/O Write Bytes** directed to console are not counted.
- </note>
-==== 02. Windows Performance Recorder & Analyzer ====
- <note warning>
- For this task we will use //**Logs.exe**// and //**GoodLog.exe**// inside the //**Task-02**// folder.
- </note>
- We want to capture the activity of both //**Logs.exe**// and //**GoodLog.exe**//.
- To do this we will start WPR and record an execution of Logs.exe and GoodLog.exe afterwards.
- We will then analyze the results using WPA, concentrating on the Disk activity and compare the results with Task Manager.
- :!: **To do this you can follow the steps below or follow the demo in this [[https://drive.google.com/file/d/1DTEnxhv9Tb5TORz1RFT7-v2ojLPW5l7A/view | video]].**
- The video is recommended as it is more visual and can be easily followed.
- <note tip>
-  - Open Windows Performance Recorder by searching for it in the search bar
-  - Click "More Options" and choose the following from Resource Analysis: CPU usage, Disk I/O activity, File I/O activity, Registry I/O activity, Networking I/O activity, Heap usage, Pool usage, and VirtualAlloc usage
-  - Using Command Prompt, navigate to the folders where the Logs/GoodLog executables are located
-  - Start the recording and run GoodLog & Logs afterwards
-  - Save the recording and open it with WPA
-  - Double click on Storage to analyze the storage operations
-  - Click on Utilization by Disk on the upper-left corner and select: Utilization by Process, Path Name, and Stack
-  - Run the executables again, this time observing them inside Task Manager
- </note>
- Can you guess why there is disk activity, but WPA does not show it?
- <spoiler>
- The way Windows Performance Recorder records activity is based on events generated by the Windows kernel.
- It registers to track the events, listens to them, and during the recording period it constantly samples which process uses which resource at the time of sampling.
- It sums up the number of times that a process was caught doing something.
- In our case, the two processes want to write to the disk, but they are not the ones that get to do the actual writing.
- They tell the system that they want to write, and the System process schedules the writing.
- The reason for this is targeting a more efficient disk writing, as the System process is trying to minimize the impact to the disk.
- This is why our process's writing is passed over to the System process.
- </spoiler>
-==== 03. Process Monitor ====
-== Task A - Checking Logging File ==
-<note warning>
- For this task we will use //**Logs.exe**// and //**GoodLog.exe**// inside the //**Task-03/Task-A**// folder.
-</note>
- We want to see what is the difference between //**Logs.exe**// and //**GoodLog.exe**//.
- After running both programs we can see that the output is identical.
- Still, **Logs.exe** takes much longer to complete compared to **GoodLog.exe**.
- We will use Process Monitor to determine the problem.
- :!: **Follow the steps to analyze the programs.**
- <note tip>
-  - Open Process Monitor and add a filter for **Logs.exe** (and **GoodLog.exe** afterwards). To do this create a filter of the format //'Process Name' 'is' 'Logs.exe'//.
-  - Open the Command Prompt and run Logs.exe, while Process Monitor is running.
-  - Save the capture and reset all events (Ctrl + X).
-  - Repeat the first 3 steps for **GoodLog.exe**.
-  - Open both saved PML files and notice the differences.
- </note>
- <spoiler>
- {{:ep:laboratoare:ep4_procmonlogscomparison.jpg?400|}}
- <note important>
- On the left-hand side it is shown the faster logging process, and on the right-hand side the slower one.
- Look in the red highlighted area to see the difference.
- On the left-hand side the logging file is opened, followed by continuous writing.
- On the right-hand side the file is opened and closed for every writing operation which explains the significant slowdown.
- </note>
- </spoiler>
- We now know why one program is faster than the other.
- <note>
- If we want to go more in depth to find what code generated an event we can still use Process Monitor.
-  - Go to Options → Configure Symbols. Here you can configure the path for the symbols and the source code.
-  - Now we can double click on events like CreateFile and check the code by going to the Stack tab and clicking on an entry from our program.
- </note>
-== Task B - Investigating a Handle Leak ==
- <note warning>
- For this task we will use //**bad.exe**// and //**good.exe**// inside the //**Task-03/Task-B**// folder.
- </note>
- Both executables have the same outcome.
- The difference is their running time.
- :!: **Identify the problem. Hint: Process Monitor**
- <spoiler>
- A handle leak consists of a process that opens files and does not close them.
- On modern computers if this action is performed millions of times, the system may become unresponsive and will either experience an overall slowdown or the application that causes this will eventually crash.
- You may think that millions of handles are impossible to reach, so it is not worth paying attention to this problem.
- However, imagine that there are services running on servers for years.
- As an example, having a handle leak every 2 seconds amounts for over 10 million handle leaks in a year.
- How should such problems be investigated?
- </spoiler>
- <note tip>
- Open up a terminal and run bad.exe. Check out the Details tab in Task Manager after adding the Handles column.
- </note>
-==== 04. Process Explorer ====
- <note warning>
- For this task we will use //**HandleLeak.exe**// inside the //**Task-04**// folder.
- </note>
- We want to check what is wrong with this HandleLeak.exe program (ignore its name :-)).
- For Task Manager, on the Details tab add the Handles column.
- Run the program, search for it inside Task Manager, and look at the Handles column.
- The number of handles keeps growing. This is clearly a problem.
- :!: **How do we investigate it? What is wrong with this file?**
- <note tip>
-  - Run Process Explorer as administrator. It is similar to Task Manager.
-  - Select the process you are interested in (HandleLeak) and press Ctrl + H. This will display all open Handles of the process.
-  - Notice that the leak happens on the leak.txt file.
-  - To find out more exact information about what causes this we will have to use Process Monitor with a filter for HandleLeak.exe.
- </note>
-==== 05. Windows API ====
- <note warning>
- For this task we will use the project skeleton inside the //**Task-05**// folder.
- </note>
- :!: We want to create a simple Console Task Manager with [[https://docs.microsoft.com/en-us/windows/win32/psapi/psapi-functions | functions from PSAPI]].
- This simple Task Manager will print two CSV tables.
- The first will contain system information provided by PSAPI.
- The second will print memory information for accesible process and its name.
- **We want to print information in kB instead of bytes for processes**.
- <note tip>
- This list of functions from PSAPI should catch your eye:
-  * [[https://docs.microsoft.com/en-us/windows/win32/api/psapi/nf-psapi-getperformanceinfo | GetPerformanceInfo]]
-  * [[https://docs.microsoft.com/en-us/windows/win32/api/psapi/nf-psapi-enumprocesses | EnumProcesses]]
-  * [[https://docs.microsoft.com/en-us/windows/win32/api/psapi/nf-psapi-getprocessimagefilenamea | GetProcessImageFileNameA]]
-  * [[https://docs.microsoft.com/en-us/windows/win32/api/psapi/nf-psapi-getprocessmemoryinfo | GetProcessMemoryInfo]]
- Follow the hints inside the skeleton to see how you will put them all together.
- </note>
- If everything goes smoothly you should see these two csv tables in your console:
-== System Data ==
- <code>
- CommitTotal, CommitLimit, CommitPeak, PhysicalTotal, PhysicalAvailable, SystemCache, KernelTotal, KernelPaged, KernelNonpaged, PageSize, HandleCount, ProcessCount, ThreadCount
-, 3211133, 876957, 1572733, 905287, 917590, 110538, 65666, 44872, 4096, 55952, 150, 1401
- </code>
-== Process Data ==
- <code>
- ProcessName, PageFaultCount, PeakWorkingSetSize, WorkingSetSize, QuotaPeakPagedPoolUsage, QuotaPagedPoolUsage, QuotaPeakNonPagedPoolUsage, QuotaNonPagedPoolUsage, PagefileUsage, PeakPagefileUsage
- sihost.exe, 11355, 24620, 14116, 250, 235, 19, 17, 5812, 6180
- svchost.exe, 17407, 16264, 8996, 160, 160, 14, 14, 5104, 5480
- svchost.exe, 17702, 29644, 15852, 278, 264, 26, 20, 6448, 7312
- taskhostw.exe, 9978, 16236, 9260, 183, 179, 30, 28, 6580, 7140
- explorer.exe, 363502, 143464, 116820, 1189, 1073, 124, 102, 62260, 85940
- svchost.exe, 13452, 20052, 15104, 238, 235, 16, 16, 3956, 4112
- StartMenuExperienceHost.exe, 24412, 65908, 22660, 574, 536, 32, 27, 19304, 23456
- RuntimeBroker.exe, 12519, 25888, 6088, 254, 213, 20, 16, 5320, 7440
- SearchUI.exe, 219211, 230984, 196976, 1070, 989, 119, 107, 140180, 155912
- RuntimeBroker.exe, 54907, 48524, 45580, 533, 440, 42, 32, 19536, 22592
- SkypeApp.exe, 13196, 42504, 7800, 453, 453, 37, 37, 14156, 14216
- SkypeBackgroundHost.exe, 3378, 11908, 1320, 124, 124, 8, 8, 2004, 2176
- RuntimeBroker.exe, 15080, 20336, 11592, 256, 193, 20, 15, 3340, 4936
- SecurityHealthSystray.exe, 2742, 8808, 2256, 149, 144, 10, 9, 1736, 1896
- vmtoolsd.exe, 66129, 39368, 28588, 316, 302, 30, 29, 19000, 30292
- OneDrive.exe, 26632, 73076, 23408, 527, 506, 67, 48, 21124, 34376
- RuntimeBroker.exe, 5798, 16256, 3384, 173, 162, 12, 11, 2440, 3088
- svchost.exe, 3619, 12040, 3236, 153, 152, 15, 14, 2940, 3304
- MyTaskManager.exe, 893, 3256, 3104, 26, 26, 5, 4, 1336, 1336
- msvsmon.exe, 5289, 20004, 20000, 464, 464, 14, 14, 141756, 141756
- ...
- </code>
-==== 06. Feedback ====
- :!: Please take a minute to fill in the **[[https://forms.gle/KHMVUhNfCPoR71Ew7 | feedback form]]** for this lab.
- {{ :ep:laboratoare:ep4_logo_bitd2.png?300 |}}
+TODO

General Information

Lectures

Labs

Assignments

Archived Labs

ep/labs/08.1633438893.txt.gz · Last modified: 2021/10/05 16:01 by cezar.craciunoiu

Show page Old revisions

Media Manager Back to top