03. [50p] Protocol Options

As you've probably already seen in the previous exercise, TCP uses protocol extensions (called options) in order to negotiate session parameters and improve overall performance. Note that this mechanism has existed since the very inception of the protocol approx. 30 years ago (RFC793 - Transmission Control Protocol), with many being added post-factum.

While the same could be said for IP options (RFC791 - Internet Protocol), there have always been… issues. In 2005 it was decided that IP Options are not an option. Middleboxes (i.e.: network equipment in the Internet – routers, NATs, firewalls, etc.) would sometimes implement abridged versions of the protocol specifications. For IP options, this meant that the IHL field would be ignored and the header would always be considered to be 20 bytes in length. As a result, if a packet carried IP options, cheap network equipment would wrongly assume that the layer 4 header would start at a 20 byte offset and drop it due to erroneously perceived malformations.

The authors of the 2005 report discovered that only a fraction (15%) of edge Autonomous Systems (AS – huge, ISP-grade networks with a unified routing policy) were responsible for most packet drops. This made them optimistic towards a speedy resolution of this issue, but things haven't changed much in the past 15 or so years. Today IP options usually work just fine in local networks. As for the wider Internet, there are specific paths and networks where IP options can pass unthwarted, but only if the layer 4 protocol is ICMP. The logic may be that most IP options are used for path measurement anyway, so why use it in conjunction with anything other than ICMP (a pretty dumb argument, I admit… but it's not mine).

Luckily, one of these compliant networks is RoEduNet. If you're not working from the university's network, then start a VM instance on OpenStack. ISPs like RDS tend to drop IP Options. As a target, we will use DigitalOcean. Based on some personal tests, I can guarantee that they don't drop any options, regardless of region.

Figure 2: Layout of a layer 3 (IP) header with options. Their presence is confirmed by an Internet Header Length (IHL) value strictly larger than 5. Most options are Type-Length-Value (TLV) encoded. When the length is constant or there is no information to be included (e.g.: No Operation, End of Options List), deviations from this format can be accepted for the purpose of saving space.

Overarching Goal

The first task is to modify outgoing traffic and include a Record Route option in an ICMP Echo Request. Check its description in RFC791 to understand what it does (and no, not “Loose/Strict Source and Record Route”… keep hitting that Find key). The packet structure will resemble that in the picture above. Don't worry; for this step, we'll give you a little help ;)

Your second task will be to write a bash script that extracts the IP addresses from the ICMP Echo Response's Record Route option from a packet capture and perform an AS Lookup. In other words, you will determine the names of all the networks that the packet traverses on its way from the university to DigialOcean and back.

But let's take it step-by-step.

[10p] Task A - Injecting IP Options

Remember talking about iptables extensions earlier? Netfilter Queue is one of them and will be relevant for this task. What it is, is an iptables target. What it does, it redirects each matched packet to a userspace process for evaluation and optionally, modification.

The userspace process receives each packet by polling a Unix Domain Socket. After obtaining one, it can perform any type of analysis that it wants (e.g.: deep packet inspection) in order to reach a verdict. The verdict can be the already known built-ins (i.e.: ACCEPT, DROP, etc.) or it can redirect the packet to another queue, with another process listening. When setting the verdict, a modified packet can be provided to replace the original on its datapath through the kernel's network stack.

Enter ops-inject. This is a tool (that's still under development, mind you) that allows the annotation of matched packets with IP/TCP/UDP options. Why is this tool simple to use: all you have to do is provide a sequence of bytes representing the codepoints of the options that you want to append. This byte stream is passed to an internal decoder that expands each byte into a fully-fledged option, albeit in accordance to an arbitrary implementation. Once you clone the repo, you should look over two sources in particular:

  • src/main.cpp : here, just understand what libnetfilterqueue library calls are made in order to set up the Unix socket, to receive the packet and to set its verdict.
  • src/ops_ip.c : this is where all available IP Options are implemented; take a look at the ip_decoders vtable at the bottom to associate options and codepoints.

First of all, let's fetch the tool and compile it.

$ git clone https://github.com/RaduMantu/ops-inject.git
$ cd !$:t:r
$ make -j $(nproc)

Pro tip #3: Bash Modifiers and Word Designators

  • !$ is substituted with the final argument of previously executed command
  • :t removes the leading pathname components, leaving ops-inject.git
  • :r removes exactly one trailing suffix, leaving ops-inject


  • IPPROTO_ETHERNET error : if your netinet/in.h is missing this definition, you can delete line 36 from src/str_proto.c.
  • libnetfilter-queue missing : consider installing libnetfilter-queue-dev and libnetfilter-queue1.

Next, let's insert an iptables rule that matches all outgoing ICMP packets. Take note of --queue-num 0 for when we'll need to tell the userspace process which Netfilter Queue to subscribe. Also, --queue-bypass tells the iptables module to disable enqueuing packets if there's no process listening. Otherwise, the queue's buffer will fill up and overflowing packets will be dropped by default until some space is created.

$ sudo iptables -I OUTPUT -p icmp -j NFQUEUE --queue-num 0 --queue-bypass

Finally, run ops-inject while telling it to append the Record Route option (0x07). Because the tool takes a file as input, and because we give it the PseudoTerminal Slave of a subshell, things can get messy if we simply run it with sudo. As a result, it's easier to just switch to root. To get a feel of what the other options do, just run ops-inject --help once.

$ sudo su
# ./bin/ops-inject -p ip -q 0 -w <(printf '\x07')

Having completed the setup, let's generate some traffic!

$ ping -c 3 $(dig +short digitalocean.com | head -n 1)
    PING ( 56(84) bytes of data.
    64 bytes from icmp_seq=1 ttl=57 time=46.7 ms
    64 bytes from icmp_seq=2 ttl=57 time=14.2 ms     (same route)
    64 bytes from icmp_seq=3 ttl=57 time=18.2 ms     (same route)

Normally, we would need wireshark or tcpdump to see the result but fortunately, ping is able to understand the Record Route option. The reason for this is that it can generate it itself (see -R option). Should it have wondered that it received a Record Route option in response to a normal ICMP Echo Request? Apparently not…

If your ISP is blocking IP options, try ping-ing your default gateway.

Normally, that should work, but there is really no guarantee. A TP-Link router usually runs Linux 2.6 (at least) and does its job well. A Tenda router, however, most likely runs some garbage proprietary firmware and won't even reply to an ICMP Echo Request with IP options.

From this point onward, it's all you! :)

[5p] Task B - Traffic capture

Run the same experiment with ICMP Echo Request again, but this time capture the traffic using tcpdump and write it to a pcap capture file.

Consider using the -U option in tcpdump to avoid buffering packets if you plan to suddenly stop it with Ctrl^C.

If you had reachability problems at task A and IP options just can't get through, use this pcap starting with Task C. It contains pretty much what you were supposed to get.

As for Task B, show us that you can capture traffic correctly by targeting the default gateway again.

[5p] Task C - Route extraction

Use tshark to extract the Record Route payload of ICMP Echo Replies from the created pcap.
You only need the IPs of the intermediary hops; these will be further processed in your script at Task D.

  • Check out the -Y option in man tshark(1).
  • Look for the appropriate IPv4 Display Filter.
  • Test filter expressions in wireshark before applying them in tshark.

[25p] Task D - AS lookup

Write a bash script starting from your tshark command. The script must perform an AS lookup and display information about each registered hop (e.g.: IP, AS name, etc.), for all packets in the pcap. Run the script. What do you notice?

The output should look something like this ± a few info.

Click to display ⇲

Click to hide ⇱

Figure 3: Packet sent from to In green are enumerated middleboxes (e.g.: routers) that added the IP of their outgoing interface to the buffer of the RR option. Not all may do so, depending on the implementation of their networking stack! In blue we have part of the publicly available information regarding the Autonomous System that said IP addresses belong to.

In order to get the required information, use the whois tool.

$ whois ${SOME_IP}
$ whois -h whois.cymru.com -- -v ${SOME_IP}

Want to make your script's output look pretty? Remember that you have ANSI color escape codes :)

[5p] Task E - AS lookup (part II)

As you might have noticed during the previous task, even DigitalOcean uses CloudFlare. This archive contains a pcap with ICMP Echo Request/Replies sent from the university to four VMs hosted on DigitalOcean. Run your script again, on this pcap and see if you can spot any interesting organization names. Then, look them up.

Not really relevant, but here are the IP addresses of the VMs involved:

New York 
UPB (localhost)
ep/labs/04/contents/tasks/ex3.txt · Last modified: 2022/09/13 13:26 by radu.mantu
CC Attribution-Share Alike 3.0 Unported
www.chimeric.de Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0