Differences
This shows you the differences between two versions of the page.
|
ep:laboratoare:02 [2016/10/26 12:04] emilian.radoi |
ep:laboratoare:02 [2017/10/02 20:27] (current) emilian.radoi |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== Tutorial 02 ====== | ====== Tutorial 02 ====== | ||
| - | |||
| The material for this tutorial was taken from Darren Hoch’s “Linux System and Performance Monitoring”. You can access it at: http://ufsdump.org/papers/oscon2009-linux-monitoring.pdf. | The material for this tutorial was taken from Darren Hoch’s “Linux System and Performance Monitoring”. You can access it at: http://ufsdump.org/papers/oscon2009-linux-monitoring.pdf. | ||
| Line 50: | Line 49: | ||
| - Convert 1 of 166 to decimal (1/166 = 0.006 seconds per Rotation) | - Convert 1 of 166 to decimal (1/166 = 0.006 seconds per Rotation) | ||
| - Multiply the seconds per rotation by 1000 milliseconds (6 MS per rotation) | - Multiply the seconds per rotation by 1000 milliseconds (6 MS per rotation) | ||
| - | - Divide the total in half (6/2 = 3 MS) or RD | + | - Divide the total in half (6/2 = 3 MS) (RD is considered half a revolution around a disk) |
| - Add an average of 3 MS for seek time (3 MS + 3 MS = 6 MS) | - Add an average of 3 MS for seek time (3 MS + 3 MS = 6 MS) | ||
| - Add 2 MS for latency (internal transfer) (6 MS + 2 MS = 8MS) | - Add 2 MS for latency (internal transfer) (6 MS + 2 MS = 8MS) | ||
| Line 81: | Line 80: | ||
| To see the effect the swapping to disk is having on the system, check the swap partition on the drive using **iostat**. | To see the effect the swapping to disk is having on the system, check the swap partition on the drive using **iostat**. | ||
| - | {{ :ep:laboratoare:ep2_poz2.png?600 |}} | + | {{ :ep:laboratoare:ep2_poz2.png?650 |}} |
| Both the swap device (///dev/sda1//) and the file system device (///dev/sda3//) are contending for I/O. Both have high amounts of write requests per second (//w/s//) and high wait time (//await//) to low service time ratios (//svctm//). This indicates that there is contention between the two partitions, causing both to underperform. | Both the swap device (///dev/sda1//) and the file system device (///dev/sda3//) are contending for I/O. Both have high amounts of write requests per second (//w/s//) and high wait time (//await//) to low service time ratios (//svctm//). This indicates that there is contention between the two partitions, causing both to underperform. | ||
| Line 95: | Line 94: | ||
| ===== 03 Introducing Network Monitoring ===== | ===== 03 Introducing Network Monitoring ===== | ||
| + | |||
| + | Out of all the subsystems to monitor, networking is the hardest to monitor. This is due primarily to the fact that the network is abstract. There are many factors that are beyond a system’s control when it comes to monitoring and performance. These factors include latency, collisions, congestion and packet corruption to name a few. | ||
| + | |||
| + | This section focuses on how to check the performance of Ethernet, IP and TCP. | ||
| + | |||
| + | ==== Ethernet Configuration Settings ==== | ||
| + | |||
| + | Unless explicitly changed, all Ethernet networks are auto negotiated for speed. The benefit of this is largely historical when there were multiple devices on a network at different speeds and duplexes. | ||
| + | |||
| + | Most enterprise Ethernet networks run at either 100 or 1000BaseTX. Use **ethtool** to ensure that a specific system is synced at this speed. | ||
| + | |||
| + | In the following example, a system with a 100BaseTX card is running auto negotiated in 10BaseT. | ||
| + | |||
| + | {{ :ep:laboratoare:ep2_poz4.png?450 |}} | ||
| + | |||
| + | The following command can be used to force the card into 100BaseTX: //# ethtool -s eth0 speed 100 duplex full autoneg off//. | ||
| + | |||
| + | ==== Monitoring Network Throughput ==== | ||
| + | |||
| + | It is impossible to control or tune the switches, wires, and routers that sit in between two host systems. The best way to test network throughput is to send traffic between two systems and measure statistics like latency and speed. | ||
| + | |||
| + | === Using iptraf for Local Throughput === | ||
| + | |||
| + | The **iptraf** utility (http://iptraf.seul.org) provides a dashboard of throughput per Ethernet interface. (Use: //# iptraf –d eth0//) | ||
| + | |||
| + | === Using netperf for Endpoint Throughput === | ||
| + | |||
| + | Unlike **iptraf** which is a passive interface that monitors traffic, the **netperf** utility enables a system administrator to perform controlled tests of network throughput. This is extremely helpful in determining the throughput from a client workstation to a heavily utilised server such as a file or web server. The **netperf** utility runs in a client/server mode. | ||
| + | |||
| + | To perform a basic controlled throughput test, the **netperf** server must be running on the server system (//server# netserver//). | ||
| + | |||
| + | There are multiple tests that the **netperf** utility may perform. The most basic test is a standard throughput test. The following test initiated from the client performs a 30 second test of TCP based throughput on a LAN. | ||
| + | The output shows that that the throughput on the network is around 89 mbps. The server (192.168.1.215) is on the same LAN. This is exceptional performance for a 100 mbps network. | ||
| + | |||
| + | {{ :ep:laboratoare:ep2_poz5.png?430 |}} | ||
| + | |||
| + | Another useful test using **netperf** is to monitor the amount of TCP request and response transactions taking place per second. The test accomplishes this by creating a single TCP connection and then sending multiple request/response sequences over that connection (ack packets back and forth with a byte size of 1). This behavior is similar to applications such as RDBMS executing multiple transactions or mail servers piping multiple messages over one connection. | ||
| + | |||
| + | The following example simulates TCP request/response over the duration of 30 seconds. | ||
| + | |||
| + | {{ :ep:laboratoare:ep2_poz6.png?450 |}} | ||
| + | |||
| + | In the previous output, the network supported a transaction rate of 4453 psh/ack per second using 1 byte payloads. This is somewhat unrealistic due to the fact that most requests, especially responses, are greater than 1 byte. | ||
| + | |||
| + | In a more realistic example, a **netperf** uses a default size of 2K for requests and 32K for responses. | ||
| + | |||
| + | {{ :ep:laboratoare:ep2_poz7.png?470 |}} | ||
| + | |||
| + | The transaction rate reduces significantly to 222 transactions per second. | ||
| + | |||
| + | === Using iperf to Measure Network Efficiency === | ||
| + | |||
| + | The **iperf** tool is similar to the **netperf** tool in that it checks connections between two endpoints. The difference with **iperf** is that it has more in-depth checks around TCP/UDP efficiency such as window sizes and QoS settings. The tool is designed for administrators who specifically want to tune TCP/IP stacks and then test the effectiveness of those stacks. The **iperf** tool is a single binary that can run in either server or client mode. The tool runs on port 5001 by default. In addition to TCP tests, **iperf** also has UDP tests to measure packet loss and jitter. | ||
| + | |||
| + | ==== Individual Connections with tcptrace ==== | ||
| + | |||
| + | The **tcptrace** utility provides detailed TCP based information about specific connections. The utility uses **libpcap** based files to perform an analysis of specific TCP sessions. The utility provides information that is at times difficult to catch in a TCP stream. This information includes: | ||
| + | * TCP Retransmissions – the amount of packets that needed to be sent again and the total data size | ||
| + | * TCP Window Sizes – identify slow connections with small window sizes | ||
| + | * Total throughput of the connection | ||
| + | * Connection duration | ||
| + | |||
| + | For more information refer to pages 34-37 from Darren Hoch’s “Linux System and Performance Monitoring” - http://ufsdump.org/papers/oscon2009-linux-monitoring.pdf. | ||
| + | |||
| + | |||
| + | ==== Conclusion ==== | ||
| + | |||
| + | Takeaways for network performance monitoring: | ||
| + | * Check to make sure all Ethernet interfaces are running at proper rates. | ||
| + | * Check total throughput per network interface and be sure it is inline with network speeds. | ||
| + | * Monitor network traffic types to ensure that the appropriate traffic has precedence on the system. | ||
| + | |||
| + | |||
| Line 107: | Line 179: | ||
| * Run //iostat –x 1 5// | * Run //iostat –x 1 5// | ||
| - | * Considering the last two outputs outputs provided by the previous command, calculate the efficiency of IOPS for each of them. Does the amount of data written per I/O increase or decrease? | + | * Considering the last two outputs provided by the previous command, calculate the efficiency of IOPS for each of them. Does the amount of data written per I/O increase or decrease? |
| Hint | Hint | ||
| - | * Divide the reads per second (//r/s//) and the writes per second (//w/s//) by the kilobytes read (//rkB/s//) and written (//wkB/s//) per second. | + | * Divide the kilobytes read (//rkB/s//) and written (//wkB/s//) per second by the reads per second (//r/s//) and the writes per second (//w/s//). |
| ==== Ex02 ==== | ==== Ex02 ==== | ||
