Earlier in Ex. 1, we mentioned that eBPF is used for more than traffic filtering. Some of you may have heard of the eXpress Data Path (XDP) or the more recent eXpress Resubmission Path (XRP). Both of these are eBPF-powered shunts of kernel data paths that are used to optimize the system for very specific types of workloads. We'll return to these in a future lecture (and maybe a lab as well) since they can be considered advanced topics. For now, we'll focus on the third purpose eBPF can serve: execution tracing.
pwru is a tool created by Cilium to help trace network packets in the kernel's network stack and debug network connectivity issues. It does this by attaching simple eBPF programs to certain function entry points. These programs can report back to a userspace process different kinds of information, including the function that was reached, the arguments that were passed, and a CPU clock timestamp. The method used for instrumenting kernel code is based on kprobes. Ask your assistant for more information.
Installation — build from source
Pre-built packages are no longer maintained for most distributions, so you'll build pwru from source. All you need is a Go compiler and make.
# Install Go if you don't have it $ sudo apt install golang-go # Ubuntu/Debian # or follow https://go.dev/dl/ for the latest version # Clone and build $ git clone https://github.com/cilium/pwru.git $ cd pwru $ make $ sudo mv pwru /usr/local/bin/ The build takes about a minute on first run (Go downloads dependencies). The result is a statically linked binary with no runtime dependencies. **Minimum requirements** (check before running): * Linux kernel ≥ 5.5 (for BTF support): ''uname -r'' * BTF enabled: ''ls /sys/kernel/btf/vmlinux'' — file must exist * ''bpf'' filesystem mounted: ''mount | grep bpf'' If BTF is missing, ''pwru'' will fail immediately with a clear error message.
Now, trace all outgoing DNS queries to the Google DNS (i.e.: 8.8.8.8) and perform one using dig. Add relative timestamps to the individual trace entries, to get an idea of the computational cost of each operation.
Finally, insert an iptables rule on the OUTPUT chain that drops DNS queries to 8.8.8.8 and redo the experiment. Check where the packet's path is cut short (the reason should be obvious :p).
systemd-resolved may intercept your query before it reaches the network. If pwru shows nothing, try:
$ sudo systemd-resolve --flush-caches
or target 127.0.0.53 to confirm caching is the issue.
Analyze the call path in the kernel network stack for the first scenario (when the packet actually made it out). Explain each step of the packet's journey.
To structure your analysis, answer these questions in order:
nf_hook_slow in the trace. Which Netfilter hook point does it correspond to (refer back to Figure 1 from Task 01)?