Differences

This shows you the differences between two versions of the page.

Link to this comparison view

ep:labs:08 [2018/10/06 13:10]
emilian.radoi
ep:labs:08 [2020/12/02 18:12] (current)
radu.mantu [Lab 08 - I/O Monitoring (Windows)]
Line 1: Line 1:
 ====== Lab 08 - I/O Monitoring (Windows) ====== ====== Lab 08 - I/O Monitoring (Windows) ======
  
-Since computers started to surface, for many people it was a mystery what was happening behind the screen and it seemed magical when it wasn't working and even more magical when it was working. Since Linux is open-source,​ all sorts of tools appeared over time to analyse problems when they came up. On Windows on the other hand, the system being closed made it harder for tools to appear.+<note important>​
  
-The first tools were provided by SysinternalsThese were written by Mike Rusinovich, who chose to make public tools such as File monitor and Registry monitor, which were later combined into Process monitorThe tools were so good that even Microsoft'​s support teams were using themSeeing their usefulness and appreciating the know-how of their operating system, Microsoft decided ​to buy Sysinternals, ​so now the original website www.sysinternals.com redirects to https://technet.microsoft.com/en-us/sysinternals (outside Romania it probably redirects ​to a different link due to localization reasons that consider the language of the country where redirection is made).+You can download the **Windows 10 VM** via {{:​ep:​labs:​ep_win10_vm.7z.torrent.txt}}. 
 +DokuWiki is not configured ​to accept //​.torrent//​ files so remove ​the //.txt// extension. 
 +After that, you know what to do...
  
-On this website ​can be found some of the tools that will be used in this tutorial ​Process Monitor, Process Explorer, VMMap, Autoruns.+Alternatively,​ you can install ​the following on your own Windows machine: 
 +  * **[[https://​go.microsoft.com/​fwlink/?​linkid=2120254 | ADK]]** ​make sure to check //**Windows Performance Analyser**//​ and //**Windows Performance Recorder**//​. 
 +  * **[[https://​visualstudio.microsoft.com/​downloads/​ | Visual Studio Community Edition]]** - select //C++ development//​.  
 +  * **[[https://​docs.microsoft.com/​en-us/​sysinternals/​downloads/​sysinternals-suite | Sysinternals suite]]**
  
-Starting with Windows 7, Microsoft has begun to invest more and more in the performance of the system and in ways to monitor the system'​s performance. Some tools already existed since Windows 2000, but they were only used internally.+</​note>​
  
-From here on out the tutorial will present the Microsoft tools. 
  
 +===== Objectives =====
  
-===== 01 Task Manager ​=====+  * How to determine the root sources of having intensive disk usage/​RAM/​CPU and intensive networking using **Task Manager** (Processes, Performance,​ App history, Start-up, Users, Details, Services). 
 +  * Analyzing performance issues due to intensive disk use using **Windows Performance Recorder**, **Process Monitor** and **Process Explorer**. 
 +  * Monitor the disk activity, identify who is generating it and how to figure out the issue by looking at the pdbs and the code.
  
  
-Task Manager is present on windows systems since they had a graphical interface, which is for more than 20 years. However, this tool evolved significantly starting with Windows 8.1, providing more meaningful information. The tutorial is presenting the Task manager tool from Windows 10, since it represents the most evolved version.+===== Introduction =====
  
-To start Task manager use the shortcut: //Ctrl + Shift + Esc//.+Since computers started to surface, for many people it was a mystery what was happening behind ​the screen and it seemed magical when it wasn't working and even more magical when it was working. Since Linux is open-source,​ all sorts of tools appeared over time to analyse problems when they came up. On Windows on the other hand, the system being closed made it harder for tools to appear.
  
-The tools has several tabs:+The first tools were provided by Sysinternals. These were written by Mike Rusinovich, who chose to make public tools such as “File monitor” and “Registry monitor”, which were later combined into “Process monitor”. The tools were so good that even Microsoft'​s support teams were using them. Seeing their usefulness and appreciating the know-how of their operating system, Microsoft decided to buy Sysinternals,​ so now the original website redirects to https://​technet.microsoft.com/​en-us/​sysinternals (outside Romania it probably redirects to a different link due to localization reasons that consider the language of the country where redirection is made).
  
-==== Processes Tab ====+On this website can be found some of the tools that will be used in this tutorial - Process Monitor, Process Explorer, VMMap, Autoruns. Starting with Windows 7, Microsoft has begun to invest more and more in the performance of the system and in ways to monitor the system'​s performance. Some tools already existed since Windows 2000, but they were only used internally.
  
-Shows all the running processes and their current resource usage in terms of CPU, Memory, Disk and Network.+===== Tutorials & Tasks =====
  
-{{:​ep:​laboratoare:​ep4_taskmanager.jpg?​800|}}+===== Task Manager =====
  
-==== Performance Tab ====+Shows the process name responsible for constant disk thrashing either by reads or writes. ​
  
-Shows the usage level of the computer'​s main resources in the last 60 seconds.+To start Task manager use the shortcut: //**Ctrl + Shift + Esc**//.
  
-{{:ep:​laboratoare:​ep4_taskmanager-cpu.jpg?800|}} +**Tabs description:** 
-  +  * **Processes tab** shows all the running processes and their current resource usage in terms of CPU, Memory, Disk and Network
-==== App history ​Tab ==== +  * **Performance tab** shows the usage level of the computer'​s main resources in the last 60 seconds. 
-  +  * The **App history tab** was first added to Windows 8, and it shows the resource consumption of metro applications. Metro applications are touch-screen-friendly applications written especially for Microsoft'​s WinRT programming interfaces
-This tab was first added to Windows 8, and it shows the resource consumption of metro applications. Metro applications are touch-screen-friendly applications written especially for Microsoft'​s WinRT programming interfaces.+  * The **Start-up tab** shows all the applications that start at start-up, (or at least in Microsoft'​s vision - this will be further detailed in the Autoruns section), and their impact on the boot time. It is helpful to check this tab in case your computer takes a long to to start up. 
 +  * **Users tab** shows the resource consumption of every logged in user.  
 +  * **Details tab** shows details for each process - pid, status, the user under which it runs. Right-clicking the column headers bar, offers the possibility to add or remove columns. The following columns: Handles, Threads, Image Path Name and Command Line are very useful: the first one (Handles) when investigating a handle leak, the second one (Threads) in the case of investigating processes that create too many threads, the third one (Image Path Name) to find out the path from where the process was started, and the last one (Command Line) to find out the parameters with which it was started.  
 +  * **Services tab** shows the service status. A Windows service can be considered similar to a Linux daemon: a process without a visual interface, offering services to user-created processes.
  
-{{:ep:​laboratoare:​ep4_taskmanager-history.jpg?​800|}} +**Conclusion:**
-  +
-==== Start-up Tab ====+
  
-This tab shows all the applications that start at start-up(or at least in Microsoft'​s vision - this will be further detailed ​in the Autoruns section)and their impact on the boot time. It is helpful to check this tab in case your computer takes a long to to start up.+  * Task Manager can be used to identify which process uses a lot of RAMCPU, accesses the disk many times or generates a lot of traffic on the network ​at a certain moment (Services tab). However, it does not offer information if in the long runthat same process is the one that generated the slowdown of the system. It does offer some information for longer periods of time, in the Start-up tab, which shows what process had higher impact at start-up, but does not specify the area that was impacted (disk space, RAM, CPU). 
 +  * You can sort by I/O read or I/O Writes, but no option to sort the results by Total I/O (combined Read & Write).
  
-{{:​ep:​laboratoare:​ep4_taskmanager-startup.jpg?​800|}} +<​note>​ 
-  +To overcome Task manager’s limitation, and to perform a thorough analysis, use the excellent Resource Monitor (Resmon) utility, which is built-in to Windows
-==== Users Tab ====+</​note>​
  
-Shows the resource consumption of every logged in userThe screenshot below shows that there is only one user logged in.+Here [[https://​drive.google.com/​file/​d/​1z1J6lgoYfBOZF7acEzR8gEq1MH1OZgaf/​view]] you have a visual representation of the previous mentioned steps.
  
-{{:​ep:​laboratoare:​ep4_taskmanager-users.jpg?​800|}} +== 01. [20p] Task Manager ​==
-  +
-==== Details Tab ====+
  
-This tab shows details for each process ​pid, status, ​the user under which it runs. Right-clicking the column headers bar, offers the possibility to add or remove columns. In the screenshot presented below the following columns were added: //​Handles//,​ //​Threads//,​ //Image Path Name// and //Command Line//. These new columns are very useful: the first one (Handles) when investigating a handle leak, the second one (Threads) in the case of investigating processes that create too many threads, the third one (Image Path Name) to find out the path from where the process was started, and the last one (Command Line) to find out the parameters with which it was started.+  ​Watch the video and go through the tutorial 
 +  ​Which program is constantly reading ​or writing ​to your hard disk?
  
-{{:ep:laboratoare:ep4_taskmanager-details.jpg?​800|}} +:!: :!NON-DEMO TASK 
-  +
-==== Services Tab ==== +
-Shows the service status. A Windows service can be considered similar to a Linux daemona process without a visual interface, offering services to user-created processes. For more details you can can have a look here: https://​en.wikipedia.org/​wiki/​Windows_service.+
  
-{{:ep:​laboratoare:​ep4_taskmanager-services.jpg?​800|}} +**How to:**
- +
  
-So far, Task Manager ​can be used to identify which process uses a lot of RAMCPU, accesses ​the disk many times or generates a lot of traffic on the network at a certain moment (Services ​tab)However, it does not offer information if in the long run, that same process is the one that generated the slowdown of the system. It does offer some information for longer periods of time, in the Start-up tab, which shows what process had higher impact at start-up, but does not specify ​the area that was impacted ​(disk spaceRAMCPU).+  * Open Task Manager, ​and select ​the Details ​tab. 
 +  * Right-click on the column header ​(NamePIDStatus etcand click Select Columns. 
 +  * Enable the following checkboxes and click OK.
  
-This led Microsoft to develop ​//Windows Assessment ​and Deployment Kit// (Windows ADK) +<note tip> 
-https://www.microsoft.com/​en-US/​download/details.aspx?​id=39982 +I/O read bytes is the number of bytes read in input/output operations generated by a process, including file, network, ​and device I/Os. 
-You should download and install at least the //Windows Performance Toolkit// moduleThis module includes: +Whereas I/O write bytes is the number of bytes written in input/output operations by a process, including file, network, and device I/Os
-  * Windows Performance Recorder +I/O Read Bytes & I/O Write Bytes directed to CONSOLE (console input object) handles are not counted
-  * Windows Performance Analyzer +</​note>​
-  * Xperf+
  
-The part of the tutorial ​is focusing on how to determine ​the sources that cause the following problems: intensive ​disk usageintensive RAM usageintensive ​CPU usage, ​and intensive networking.+  * Next, sort the listings by I/O Read bytes and see which application ​is generating the maximum I/O (in bytes/sec). Similarly, sort by I/O Write bytes to see which program is writing ​to the hard disk continuously. 
 +  * Once you identify the programdecide if you need the program or not. Leave it as it is if the I/O operations are justified. Elseremove the program or consult its documentation to tweak the settings if any. For instance, one of your browser extensions may cause high disk or CPU usage. You need to isolate the extensionadd-on or the browser’s feature causing the trouble.
  
-===== 02 I/O Monitoring ​=====+===== Windows Performance Recorder ​=====
  
-This second part of the tutorial looks into **analysing performance issues due to intensive disk use**.+<note warning>​ 
 +Datafile: Logs.exe and GoodLog.exe 
 +</​note>​
  
-==== Windows Performance Recorder ​====+Installing Windows ADK will install ​Windows Performance Recorder. Check by clicking the windows button and typing “windows performance recorder”.
  
 +Start **Windows Performance Recorder** by pressing Enter. You will see the following:
  
-Installing Windows ADK will install Windows Performance RecorderCheck by clicking ​the windows ​button ​and typing "​windows performance recorder"​.+<​spoiler>​ 
 +{{:​ep:​laboratoare:​ep4_wpr-record.jpg?​400|}} ​  
 +</​spoiler> ​   
 +     
 +Click the **More options** ​button ​to get the list shown in the screenshot right below. 
 +  
 +<​spoiler>​ 
 +{{:​ep:​laboratoare:​ep4_wpr-select.jpg?​400|}} ​  
 +</​spoiler> ​   ​
  
-{{:​ep:​laboratoare:​ep4_startwindowsperformancerecorder.jpg?300|}}+Make sure that you select the same check boxes as in the screenshot, but do not click start just yet. Create a new directory and copy the **Logs.exe** and **GoodLog.exe** files into this directory. The behaviour of these two executables is similar to logging applications that write logs to the disk. Open a terminal and change the path to the directory where you copied the files.  
 +  
 +<​spoiler>​ 
 +{{:​ep:​laboratoare:​ep4_badlogs.jpg?400|}}   
 +</​spoiler>  ​
  
-Start Windows Performance Recorder ​by pressing Enter.+Start **Windows Performance Recorder** and right after run GoodLog.exe and then Logs.exe. Once the two applications finish running, click the Save button in Windows Performance Recorder.
  
-{{:​ep:​laboratoare:​ep4_wpr-record.jpg?800|}}+<​spoiler>​ 
 + {{:​ep:​laboratoare:​ep4_goodlogs.jpg?400|}}  
 +</​spoiler> ​     ​
  
-Click the //More options// button to get the list shown in the screenshot right below. 
  
-{{:​ep:​laboratoare:​ep4_wpr-select.jpg?​800|}} +After the capture is saved, the Open option will become available in **Windows Performance Analyzer**When clicking ​the Open button it should open window such as the one below
-  +
-Make sure that you select ​the same check boxes as in the screenshotbut do not click start just yet. Create a new directory and copy the //Logs.exe// and //​GoodLog.exe//​ files (from the resources attached to this tutorial that can be found at the end of this page) into this directory. The behaviour of these two executables is similar to logging applications that write logs to the disk. Open a terminal and change the path to the directory where you copied ​the files.+
  
-{{:​ep:​laboratoare:​ep4_badlogs.jpg?800|}} +<​spoiler>​ 
-  +{{:​ep:​laboratoare:​ep4_wpa-cpu1.jpg?400|}}   
-Start //Windows Performance Recorder// and right after run //GoodLog.exe// and then //​Logs.exe//​. Once the two applications finish running, click the //Save// button in //Windows Performance Recorder//.+</​spoiler>​ 
 +      
 +Double clicking on Storage should display the following windowAnalyse ​the resources.
  
-{{:​ep:​laboratoare:​ep4_goodlogs.jpg?800|}} +<​spoiler>​ 
-  +{{:​ep:​laboratoare:​ep4_wpa-cpu2.jpg?400|}}   
-After the capture is saved, the //Open// option will become available in //Windows Performance Analyzer//. When clicking the //Open// button it should open a window such as the one below.+</spoiler> ​      
  
-{{:​ep:​laboratoare:​ep4_wpa-cpu1.jpg?​800|}} +In the upper-left corner of the newly opened window it can select Disk Usage, Utilization by DiskClick on Utilization by Disk and select: Utilization by Process, Path Name, and Stack. This will generate ​the following ​output
-  +
-Double clicking ​on //Storage// should display ​the following ​window.+
  
-{{:​ep:​laboratoare:​ep4_wpa-cpu2.jpg?800|}} +<​spoiler>​ 
-  + {{:​ep:​laboratoare:​ep4_wpa-cpu3.jpg?400|}}   
-In the upper-left corner of the newly opened window it can select ​//Disk Usage//, //​Utilization by Disk//. Click on //​Utilization by Disk// and select: //​Utilization by Process//, //Path Name//, and //Stack//. This will generate the following output.+</spoiler> ​    
  
-{{:​ep:​laboratoare:​ep4_wpa-cpu3.jpg?​800|}} +The graph looks interesting ​in Task Manager. Processes can be selected for observing their activity on the disk. It can be noticed that our processes are not shown. Run Logs.exe again while keeping Task Manager on. 
-  +
-The graph looks interesting. Processes can be selected for observing their activity on the disk. It can be noticed that our processes are not shown. Run //Logs.exe// again while keeping Task Manager on.+
  
-{{:​ep:​laboratoare:​ep4_logstaskmanagerdisk.jpg?​800|}} +<​spoiler>​ 
-  +{{:​ep:​laboratoare:​ep4_logstaskmanagerdisk.jpg?​400|}}   
-This shows that there is activity on the disk. The question is why doesn'​t ​//Windows Performance Analyzer// show it.  +</spoiler> ​   ​
-The way //Windows Performance Recorder// records activity is based on events generated by the Windows kernel. It registers to track the events, listens to them, and during the recording period it constantly samples which process uses which resource at the time of sampling. It sums up the number of time that a process was caught doing something. +
-In our case, the two processes want to write to the disk, but they are not the ones that get to do the actual writing. They tell the system that they want to write, and the //System process// schedules the writing. The reason for this is targeting a more efficient disk writing, as the //System process// is trying to minimise the impact to the disk. This is why our process'​s writing is passed over to the //System process//.+
  
-This being the case, we are introducing a new tool that was mentioned at the beginning of the tutorial: //Process Monitor//. 
-  
  
-==== Process Monitor ====+**Conclusions:​**
  
-Process Monitor can be downloaded from:+  * This shows that there is activity on the disk. The question is why doesn'​t Windows Performance Analyzer show it. The way Windows Performance Recorder records activity is based on events generated by the Windows kernel. It registers to track the events, listens to them, and during the recording period it constantly samples which process uses which resource at the time of sampling. 
 +  * It sums up the number of time that a process was caught doing something. In our case, the two processes want to write to the disk, but they are not the ones that get to do the actual writing. They tell the system that they want to write, and the System process schedules the writing. The reason for this is targeting a more efficient disk writing, as the System process is trying to minimise the impact to the disk. This is why our process'​s writing is passed over to the System process.
  
-https://technet.microsoft.com/en-us/sysinternals/processmonitor.aspx+Here [[https://drive.google.com/file/d/​1DTEnxhv9Tb5TORz1RFT7-v2ojLPW5l7A/view]] you have a visual representation as well.
  
-Before running Process Monitor, which is a small tool without an installer, it is preferable to have the latest dbghelp.dll library. Windows 10 already has the latest version. If you are using another Windows, you can get the latest version by installing: 
  
-https://​developer.microsoft.com/​en-us/​windows/​downloads/​windows-8-1-sdk+== 02[20p] WPR and WPA  ==
  
-In the installation process choose to install at least the following:+  - Watch the video and go through ​the tutorial
  
-{{:​ep:​laboratoare:​ep4_wdk_install.jpg?​800|}} 
-  
-After the installation is complete, a newer version of //​dbghelp.dll//​ can be found in //​C:​\Program Files(x86)\WindowsKits\10\Debuggers//​. Keep this in mind as it will be useful later in the tutorial. 
  
-Looking at the logs created by the two apps - //​bad.log//,​ //​good.log//​ - they are identical, but //​Logs.exe//​ has a significantly longer running time compared to //​GoodLog.exe//​. Start Process Monitor.+===== Process Monitor ​=====
  
-{{:​ep:​laboratoare:​ep4_procmon.jpg?800|}}+//Process Monitor is an excellent troubleshooting tool from Windows Sysinternals that displays the files and registry keys that applications access in real-timeThe results can be saved to a log file, which you can send it to an expert for analyzing a problem and troubleshooting it.//
  
-  +**How to Use Process Monitor ​to Track Registry ​and File System Changes?**
-If the 4 buttons in the black area on the upper part of the window are selected, ​Process Monitor ​will display the activity (in this order) for: registry, files, networking, process ​and thread activity. By unchecking them, the corresponding events will be no longer displayed.+
  
-In the menu bar there is the //Filter// fieldIf selected, it will trigger a dropdown menu that contains another //Filter// fieldIf this second //Filter// field is selectedit will open the window shown belowReplicate this on your computer.+^ Step 1: Running Process Monitor & Configuring Filters ^^ 
 +| **1.** Download Process Monitor from Windows Sysinternals site. || 
 +| **2.** Extract ​the zip file contents to a folder of your choice. || 
 +| **3.** Run the Process Monitor application|| 
 +| **4.** Include the processes ​that you want to track the activity onFor this exampleyou want to include Notepad.exe in the (Include) Filters|| 
 +| **5.** Click Add, and click OK. || 
 +| **6.** From the Options menu, click Select Columns. || 
 +| **7.** Under “Event Details”, enable Sequence Number, and click OK||
  
-{{:​ep:​laboratoare:​ep4_procmon-filters.jpg?​800|}} +<​note>​ 
-  +You can add multiple entries as well, in case if you want to track few more processes along with Notepad.exe. To keep this example simpler, let’s only track Notepad.exe. (You’ll now see the Process Monitor main window tracking ​the list of registry ​and file accesses by processes real-timeas and whey they occur.) 
-From the two dropdown menus in the upper part of the context window, select "//​Process Name//"​ instead ​of "//​Architecture//" ​and "//​is//"​ instead of "//​contains//"​. In the text filed add //​Logs.exe//​click the //Add// button ​and then the //OK// button.+</note>
  
-Open the terminal and run //Logs.exe//After the program ​is done running, save the Process Monitor ​capture.+^ Step 2: Capturing Events ^^ 
 +| **8.** ​Open Notepad. || 
 +| **9.** Switch to Process Monitor window. || 
 +| **10.** Enable ​the “Capture” mode (if it’s not already ON)You can see the status of the “Capture” mode via the Process Monitor toolbar|| 
 +| **11.** The highlighted button above is the “Capture” button, which is current disabled. You need to click that button (or use Ctrl + E key sequence) to enable capturing of events. || 
 +| **12.** Cleanup the existing events list using Ctrl + X key sequence (Important) and start afresh. || 
 +| **13.** Now switch to Notepad and try to reproduce the problem. || 
 +| **14.** To reproduce the problem (for this example)try writing to HOSTS file (C:​\Windows\System32\Drivers\Etc\HOSTS) and saving it. Windows offers to save the file (by showing the Save As dialog) with a different name, or in a different location. So, what happens under the hood when you save to HOSTS file? Process Monitor ​shows that exactly. || 
 +| **15.** Switch to Process Monitor window, and turn off Capturing (Ctrl + E) as soon as you reproduce the problem. Important Note: Don’t take much time to reproduce the problem after enabling capturing. Similarly turn off capturing as soon as you finish reproducing the problem. This is to prevent Process Monitor from recording other unneeded data (which makes analysis part more difficult). You need to do all that as quickly as you can||
  
-Use //Ctrl + X// to reset all the events captured in Process MonitorGo to //Filter// -> //Filter area//, double-click ​on the filter that was just added and change //Logs.exe// with //​GoodLog.exe//,​ then click //Add// and //Ok//.+<note warning>​ 
 +The log file above tells us that Notepad encountered an ACCESS DENIED error when writing ​to the HOSTS fileThe solution would be to simply run Notepad elevated (right-click and choose “Run as Administrator”) to be able to write to HOSTS file successfully. 
 +</note>
  
-Start //GoodLog.exe// and save the capture once the program finishes running.+^ Step 3: Saving the Output ^^ 
 +| **16.** In the Process Monitor window, select the File menu and click Save. || 
 +| **17.** Select Native Process Monitor Format (PML), mention the output file name and Path, save the file. || 
 +| **18.** Right-click on the Logfile.PML file, click Send To, and choose Compressed (zipped) folder. This compresses the file by ~90%. Look at the graphic below. You certainly want to zip the log file before sending it to someone||
  
-Scroll down in the two capture-logs until you notice the activity for //bad.log// respectively //good.log//.+You can take also take a look at this video here: https://drive.google.com/file/d/1ZYdtOq7QsY0nfYDS3e4foPFRVPgO4Qdb/view.
  
-{{:​ep:​laboratoare:​ep4_procmonlogscomparison.jpg?​800|}} 
-  
-Notice the difference. On the left-hand side it is shown the faster logging process, and on the right-hand side the slower one. Look in the red highlighted area to see the difference. On the left-hand side the logging file is opened, followed by continuous writing, while on the right-hand side the file is opened and closed for every writing operation which explains the significant slowdown. 
  
-To recap, Task Manager shows what processes use the disk intensively at the current time, Windows Performance Recorder / Windows Performance Analyzer show who used the disc during a longer time period, although they were showing the activity as belonging to the //System// process instead of our processUsing Process Monitor ​we could identify our processes'​ entire activity and we could determine why one is slower than the other. +== 03[30p] Process Monitor ​==
-But what if we could find out which line in the code causes the problem?+
  
-Go back to Process Monitor. Use the window of the badly written logging program ​(//​Logs.exe//​). Go to //Options// -> //Configure Symbols//, which will open the window shown below.+<note warning>​ 
 +Download the archive {{:​ep:​laboratoare:​logs-final.7z|}} and check if you have “Process Monitor” installed ​(Windows 10).
  
-{{:​ep:​laboratoare:​ep4_procmon-symbols.jpg?400|}}+//logs-final.7z//: **parola**
  
-If you have an older version of //dbghelp.dll// in //C:\windows\system32/​dbghelp.dll//​ compared to the one that you have downloaded in  //​C:​\ProgramFiles(x86)\WindowsKits\10\Debuggers//,​ then change the path to point to the newer version. Otherwise it makes no difference.+//HandleLeak.7z//: **parola7**
  
-In the "//Symbol paths//" text box there is a path like this: +//Task.7z//: **parola17** 
-<​code>​ +</note>
-srv*https://​msdl.microsoft.com/​download/​symbols +
-</code>+
  
-This is the path to the Microsoft servers which are used to load the Microsoft binaries. Let's explain ​any application that runs on a machine, besides its executable and its libraries, it loads the Microsoft core libraries such as //​kernel32.dll//,​ //​ntdll.dll//​ and others. To rebuild the call stack for a specific event (similar to crash dumps), Process Monitor needs //pdb//s created during the compilation of the binaries, as they are a database containing the memory address where each loaded function can be found. In Microsoft'​s case, these symbols can be used from the mentioned path, but in our case they can used from where we have the binaries as there we should also have //pdb//s (e.g. d:\Logs\), so change the path to:+== [10p] Task A Checking logging file ==
  
-<​code>​ +  ​Looking at the logs created by the two apps in Process Monitor - bad.log, good.log - they are identical, but Logs.exe has a significantly longer running time compared to GoodLog.exe. Start Process Monitor. ​
-srv*https://​msdl.microsoft.com/​download/​symbols;​d:​\Logs+
-</​code>​+
  
-The same directory containing the binary also contains the source files, so for the "//​Source code paths//"​ add the path to the **.sln** of the two projects (in my case //d:\Logs//)After this you are ready to click //Ok//.+<​spoiler>​ 
 +{{:ep:​laboratoare:​ep4_procmon.jpg?​400|}} 
 +</spoiler>
  
-In the log (//​D:​\Logs\bad.log//​) go to //​CreateFile//​. Double-click to open the //Event Properties// ​window. Choose ​the //Stack// tabscroll down and you can notice that in the main function of //​main_bad_log.cpp//at line 12 the opening takes placeClick the "//​Source//"​ button to view the source code containing ​the issue.+  * If the 4 buttons in the black area on the upper part of the window ​are selected, Process Monitor will display ​the activity (in this order) for: registryfiles, networking, process ​and thread activityBy unchecking them, the corresponding events will be no longer displayedIn the menu bar there is the Filter field. If selected, it will trigger a dropdown menu that contains another Filterfield. If this second Filter field is selected, it will open the window shown below. Replicate this on your computer
  
-{{:​ep:​laboratoare:​ep4_prcomoncode.jpg?800|}}+<​spoiler>​ 
 +{{:​ep:​laboratoare:​ep4_procmon-filters.jpg?400|}} 
 +</​spoiler>​
  
-In "​Task.7z"​ you have another example of two executables:​ //good.exe// and //bad.exe//They both have the same outcome, the only difference being their running ​time (one of them is significantly slower) +  * From the two dropdown menus in the upper part of the context window, select ”Process Name” instead of ”Architecture” and ”is” instead of ”contains”. In the text filed add Logs.exe, click the Add button ​and then the OKbutton. Open the terminal and run Logs.exe. After the program is done runningsave the Process Monitor capture. Use Ctrl + X to reset all the events captured in Process Monitor. Go to Filter → Filter area, double-click on the filter that was just added and change Logs.exe with GoodLog.exe,​ then click Add and Ok. Start GoodLog.exe and save the capture once the program finishes ​running. ​Scroll down in the two capture-logs until you notice the activity for bad.log respectively good.log.
-Identify ​the problem.+
  
-<solution -hidden+<spoiler
-//bad.exe// - flushes the buffer after each writing +{{:​ep:​laboratoare:​ep4_procmonlogscomparison.jpg?400|}} 
-</solution>+</spoiler>
  
 +<note important>​
 +  * Notice the difference. On the left-hand side it is shown the faster logging process, and on the right-hand side the slower one. Look in the red highlighted area to see the difference. On the left-hand side the logging file is opened, followed by continuous writing, while on the right-hand side the file is opened and closed for every writing operation which explains the significant slowdown. ​
 +</​note>​
  
-So far you have seen how to monitor ​the disk activityhow to identify ​who is generating it and how to figure ​out the issue by looking at the //pdb//s and the code+  * To recap, Task Manager shows what processes use the disk intensively at the current timeWindows Performance Recorder / Windows Performance Analyzer show who used the disc during a longer time period, although they were showing the activity as belonging ​to the System process instead of our process. Using Process Monitor we could identify ​our processes'​ entire activity ​and we could determine why one is slower than the other. But what if we could find out which line in the code causes ​the problem? Go back to Process Monitor. Use the window of the badly written logging program (Logs.exe). Go to Options → Configure Symbols, which will open the window shown below
  
-Let's consider a new probleminvestigating a handle leak. A handle leak consists of a process that opens files and does not close them. On modern computers if this action is performed millions of times, the system may become unresponsive and will either experience an overall slowdown or the application that causes this will eventually crash+<​spoiler>​ 
-You may think that millions of handles are impossible to reach, so it is not worth paying attention to this problem. However, imagine that there are services running on servers for years. As an example, having a handle leak every 2 seconds amounts for over 10 million handle leaks in a year. How should such problems be investigated?​+{{:ep:​laboratoare:​ep4_procmon-symbols.jpg?400|}} 
 +</​spoiler>​
  
-Open up a terminal and run //​HandleLeak.exe//Check out the "//​Details//"​ tab in Task Manager after adding ​the "//​Handles//"​ column.+  * In the log (D:​\Logs\bad.log) go to CreateFileDouble-click to open the Event Properties window. Choose the Stacktab, scroll down and you can notice that in the main function of main_bad_log.cpp,​ at line 12 the opening takes place. Click the ”Source” button to view the source code containing the issue
  
-{{:​ep:​laboratoare:​ep4_taskmanagerhandles.jpg?800|}} +<​spoiler>​ 
-  +{{:​ep:​laboratoare:​ep4_prcomoncode.jpg?400|}}  
-It can be noticed that the number of handles keeps growing. This is clearly a problem, but how do we investigate it? +</spoiler>
-We will consider a new tool, //Process Explorer//.+
  
-==== Process Explorer ==== 
  
-https://​technet.microsoft.com/​en-us/​sysinternals/​processexplorer.aspx+== [20p] Task B Investigating a handle leak ==
  
-Run it as //​administrator//​. It is similar to Task Manager. Select the process that you are interested in, namely //​HandleLeak//,​ and press //Ctrl + H//. The output should be similar to the one in the screenshot below. +:!: :!: NON-DEMO TASK 
- +
-{{:ep:laboratoare:ep4_procexplorerhandles.jpg?​800|}} +
-  +
-//Ctrl + H// opens a window under the "//​Process//"​ section that displays all open handles along with information about them. Thus it will display file handles, registry handles, threads handles, and so on. There is another view (//Ctrl + D//) that displays all the loaded //dll//s.+
  
-So it can be noticed that the leaks are on the following file//D:\Logs\HandleLeak\leak.txt//. This is very useful information,​ but it would be better to find out who is responsible for the leak in code. Run Process Monitor with a filter on //​HandleLeak.exe// and to notice ​the stack where the leakage ​is happens.+In {{:ep:laboratoare:​logs-final.7z|}} you have another example of two executables:​ **good.exe** and **bad.exe**. Both have the same outcome, ​the only difference being their running time (one of them is significantly slower). **Identify the problem**.
  
-==== Feedback ====+**How to:**
  
-Please take minute ​to fill in the **[[https://​goo.gl/​forms/​B9WLG5IYOfMu2ByJ2 | feedback form]]** for this lab.+  * A handle leak consists of process that opens files and does not close them. On modern computers if this action is performed millions of times, the system may become unresponsive and will either experience an overall slowdown or the application that causes this will eventually crash. You may think that millions of handles are impossible ​to reach, so it is not worth paying attention to this problem. However, imagine that there are services running on servers for years. As an example, having a handle leak every 2 seconds amounts for over 10 million handle leaks in a year. How should such problems be investigated?​ 
 +  ​* **Hint:** Open up a terminal and run HandleLeak.exe. Check out the ”Details” tab in Task Manager after adding the ”Handles” column.
  
  
-====== Resources ======+== 04. [30p] Process Explorer ​==
  
-{{:ep:laboratoare:logs-final.7z|}}+:!: :!: NON-DEMO TASK 
  
-<​hidden>​+  * It can be noticed that the number of handles keeps growing. This is clearly a problem, but how do we investigate it?
  
-//​logs-final.7z//​ password: ​**parola**+**How to:**
  
-Passwords for the other two archives within //​logs-final.7z//): +  * Run it as administrator. It is similar to Task Manager. Select ​the process that you are interested in, namely HandleLeak, and press “Ctrl + H”
-  * //​HandleLeak.7z//: **parola7** +  * “Ctrl + H” opens a window under the ”Process” section that displays all open handles along with information about them. Thus it will display file handles, registry handles, threads handles, and so on. There is another view (Ctrl + D) that displays all the loaded dlls
-  * //Task.7z//**parola17**+  * So it can be noticed that the leaks are on the following fileD:​\Logs\HandleLeak\leak.txt. This is very useful information,​ but it would be better to find out who is responsible for the leak in code. Run Process Monitor with a filter on HandleLeak.exe and to notice the stack where the leakage is happens.
  
-The point of having the passwords is to not let the students have access to the code before they observe the behaviour of the programs. 
  
-</​hidden>​+== 05. [10p] Feedback ==
  
-{{ :ep:laboratoare:ep4_logo_bitd2.png?​250|}}+:!: :!: NON-DEMO TASK 
  
 +  * Please take a minute to fill in the **[[https://​docs.google.com/​forms/​d/​e/​1FAIpQLSfsMBl2EFu10jJG2qHEiSsR-qYr3wkzQPfDwjhChKnjRtDT_w/​viewform | feedback form]]** for this lab.
  
 +{{ :​ep:​laboratoare:​ep4_logo_bitd2.png?​300 |}}
  
ep/labs/08.1538820611.txt.gz · Last modified: 2018/10/06 13:10 by emilian.radoi
CC Attribution-Share Alike 3.0 Unported
www.chimeric.de Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0