Lab 06 - Network Monitoring

tcpdump is a network traffic monitoring tool. At its core, it uses libpcap which in turn uses a technology called Extended Berkley Packet Filter (eBPF).

BPF was first proposed around 1995 when filtering mechanisms (and firewalls) were still a novel concept and were based on interpreters. BPF (now referred to as Classic BPF - cBPF) was the initial version of a Motorola inspired virtual ISA (i.e.: had no hardware implementation – think CHIP-8). eBPF is basically still BPF but more compatible with 64-bit architectures so that Just In Time (JIT) translators have an easier time running the code.

At first, the whole idea was to compile packet filtering programs and attach them to sockets in kernelspace. These programs would filter out packets that userspace processes would not be interested in. Consequently, this would reduce the quantity of data copied over the kernelspace/userspace boundary, only to ultimately be discarded.

Today, eBPF is used heavily for system profiling by companies such as Netflix and Facebook. Linux has had a kernel VM capable of running and statically analyzing eBPF code since around 2006. tcpdump is one of the few examples that still use it for its original purpose.

iptables is a configuration tool for the kernel packet filter.

The system as a whole provides many functionalities that are grouped by tables: filter, nat, mangle, raw, security. If you want to alter a packet header, you place a rule in the mangle table. If you want to mask the private IP address of an internal host with the external IP address of the default gateway, you place a rule in the nat table. Depending on the table you choose, you will gain or lose access to some chains. If not specified, the default is the filter table.

Chains are basically lists of rules. The five built-in chains are PREROUTING, FORWARD, POSTROUTING, INPUT, OUTPUT. Each of these corresponds to certain locations in the network stack where packets trigger Netfilter hooks (here is the PREROUTING kernel hook as an example – not that hard to add one, right?) For a selected chain, the order in which the rules are evaluated is determined primarily by the priority of their tables and secondarily by the user's discretionary arrangement (i.e.: order in which rules are inserted).

A rule consists of two entities: a sequence of match criteria and a jump target.

The jump target represents an action to be taken. You are most likely familiar with the built-in actions such as ACCEPT or DROP. These actions decide the ultimate fate of the packet and are final (i.e.: rule iteration stops when these are invoked). However, there are also extended actions (see man iptables-extensions(8)) that are not terminal verdicts and can be used for various tasks such as auditing, forced checksum recalculation or removal of Explicit Congestion Notification (ECN) bits.

The match criteria of every rule are checked to determine if the jump target is applied. The way this is designed is very elegant: every type of feature (e.g.: Layer 3 IP address vs Layer 4 port) that you can check has a match callback function defined in the kernel. If you want, you can write your own such function in a Linux Kernel Module (LKM) and thus extend the functionality of iptables (Writing Netfilter Modules with code example). However, you will need to implement a userspace shared library counterpart. When you start an iptables process, it searches in /usr/lib/xtables/ and automatically loads certain shared libraries (note: this path can be overwritten or extended using the XTABLES_LIBDIR environment variable). Each library there must do three things:

• define iptables flags for the new criteria that you want to include.
• define help messages for when iptables --help is called (its help message is an amalgamation of each library's help snippet).
• provide an initialization function for the structure containing the rule parameters; this structure will end up in the kernel's rule chain.

So when you want to test the efficiency of the iptables rule evaluation process, keep in mind that each rule may imply the invocation of multiple callbacks such as this.

The Address Resolution Protocol (ARP) resolves layer 2 addresses (MAC) from layer 3 addresses (e.g.: IP). Normally, all hosts are compelled to reply to ARP requests, but this can be fiddled with using tools such as arptables. You can show the currently known neighbors using iproute2.

$ ip -c neigh show

# alias for iproute2 color output
alias ip='ip -c'

The Internet Control Message Protocol (ICMP) is an ancillary protocol meant mainly to report errors between hosts. Sometimes it can also be used to perform measurements (ping) or to inform network participants of better routes (Redirect Messages). There are many ICMP functionalities, most of which are now deprecated. Note that some network equipment may not be capable of understanding new and officially recognized protocols, while other may not even recognize experimental ICMP codepoints (i.e.: type=253,254) and simply drop the packet. Because ICMP can be used to stage attacks in a network, some operating systems (e.g.: Windows ≥7) went so far as to disable Echo Replies by default.

Hint: click on the file name to download the snippet below.

localnet-ping.sh

#!/bin/bash
 
# localnet-ping.sh - performs differential ARP / ICMP scan
#   $1 : [required] interface name  
 
if [ "$#" -ne 1 ]; then
    echo "Usage: ./localnet-ping.sh <interface>"
    exit 1
fi
 
# generate list of IPs and hostnames in local network for given interface
localnet_hosts=$(sudo arp-scan                                      \
                    --interface=$1           `# scanned network`    \
                    --localnet               `# only local network` \
                | head -n -3                 `# hide footer lines`  \
                | tail -n +3                 `# hide header lines`  \
                | awk '{$2=""; print $0}'    `# hide MAC address`   \
                )
 
# process generated list, one item at a time
while read -r it; do
    # separate IP from hostname
    current_ip=$(awk '{print $1}' <<< $it)
    current_host=$(awk '{$1=""; print $0}' <<< $it)
 
    printf '\033[1;33m%15s   %-35s \033[0;33m==>  \033[0m' \
        $current_ip "$current_host"
 
    # ping current host
    ping -c 1           `# only one ping` \
         -W 1           `# 1s timeout`    \
         $current_ip    `# target host`   \
         1>/dev/null 2>&1
 
    # evaluate ping success
    if [ $? -eq 0 ]; then
        printf '\033[1;32mok\n\033[0m'
    else
        printf '\033[1;31mfail\n\033[0m'
    fi
done <<< "$localnet_hosts"

nmap is a network exploration tool and a port scanner. Today, we will look only at a specific functionality that it shares with the traceroute utility.

Route discovery is simple in principle: IPv4 packets have a Time to Live (TTL) field that is decremented by 1 with each hop, thus ensuring a limited packet lifespan (imagine routing loops without TTL). Even if the TTL is 0, the layer 3 network equipment must process the received packet (the destination host can accept a packet with TTL=0). Routers may check the TTL field only if they are to forward the packet. If the TTL is already 0, the packet is dropped and a ICMP Time-To-Live Exceeded message is issued to the source IP. By sending packets with incrementally larger TTL values, it is possible to obtain the IP of each router on the path (at least in theory).