Add advanced eBPF tutorials 51–54 and tutorial agent skills (#220)

* Add targeted TCP connection quarantine tool

* Document TCP quarantine workflow

* Refresh package indexes before CI install

* Document TCP quarantine namespace scope

* Require BPF events for TCP iterator

* Add production fsession latency tracer

* Document fsession latency workflow

* Reject empty fsession CLI values

* Preserve successful fsession exit status

* Document fsession deadline regression test

* Drain fsession events before summary

* Document fsession PID and shutdown semantics

* Simplify fsession polling control flow

* Add bounded BPF egress pacer tutorial

* Test queued packet cleanup on signal

* Document the BPF egress pacer

* Polish and review the BPF egress pacer

* Refactor egress pacer option parsing

* Add executable image inspector lesson

* Build the exec image inspector in CI

* Document executable image inspection

* Handle re-exec chains in the image inspector

* Polish exec image inspector guidance

* Reduce exec inspector wait complexity

* Rewrite the egress pacer tutorial

* Rewrite the exec image inspector tutorial

* Polish the exec image inspector tutorial

* Correct the egress pacer tutorial

* Make the exec tutorial Chinese natural

* Fix egress tutorial prose checks

* Finish the exec tutorial style pass

* Clarify exec tutorial runtime evidence

* Polish exec tutorial scope wording

* Rewrite the egress pacer tutorial for readers

* Regenerate tutorial indexes

* Teach the egress pacer component by component

* Use standard Markdown in the egress tutorial

* Strengthen the egress pacer tutorial narrative

* jiji

* Use positive framing in the egress tutorial

* Rewrite the exec image inspector tutorial

* Rewrite the TCP quarantine tutorial

* Add the egress pacer lifecycle diagram

* Refresh TCP tutorial indexes

* Add the exec image inspection flow diagram

* Use one source directory link

* Rewrite the fsession latency tutorial

* Add fsession feature background

* Add BPF qdisc feature background

* Add TCP iterator feature background

* Add exec inspection feature background

* Require feature background in tutorial openings

* Polish fsession background prose

* Simplify repository tutorial skills

* Update exec inspector tooling revisions

* Restructure tutorials 51-54 following advanced guideline

- Update SKILL.md with strict section ordering and "why" requirements
- Rewrite README.zh.md for tutorials 51-54:
  - Add background sections explaining why traditional approaches fail
  - Move compilation/execution sections after code analysis
  - Add high-level mechanism explanation before code
  - Use paragraph style instead of bullet lists

* Consolidate skill guidelines and remove duplicates

- Update write-bpf-production-tutorial skill to reference 47, 48, 49
- Add structure requirements section with "Why" section emphasis
- Remove duplicate advanced-tutorial-guidelines.md (use scripts/guideline_advance.md instead)

* Rewrite English READMEs 51-54 to match Chinese structure

Restructure all four English tutorials to match their Chinese counterparts:
- Add "Why this approach is needed" sections explaining traditional limitations
- Add high-level mechanism sections before code
- Move compilation/execution sections after code analysis
- Add environment requirements tables
- Use consistent section ordering across all tutorials

* Regenerate TOC after tutorial title changes

Run generate_toc.py to sync SUMMARY and README files with
updated tutorial titles from the restructuring.

* Regenerate compatibility tables after title changes

---------

Co-authored-by: LinuxDev9002 <i@littlefisher.me>
This commit is contained in:
云微
2026-07-19 19:34:09 -07:00
committed by GitHub
parent d055a0833f
commit ca1267d82a
57 changed files with 9014 additions and 385 deletions

View File

@@ -0,0 +1,10 @@
level=Depth
type=Security
kernel_min=6.5
kernel_min_basis=documented
architectures=x86_64
btf=required
kernel_config=CONFIG_BPF=y,CONFIG_BPF_SYSCALL=y,CONFIG_BPF_JIT=y,CONFIG_BPF_EVENTS=y,CONFIG_DEBUG_INFO_BTF=y,CONFIG_INET=y,CONFIG_PROC_FS=y
hardware=none
root=required
test_status=ci-runtime

12
src/51-tcp-quarantine/.gitignore vendored Normal file
View File

@@ -0,0 +1,12 @@
# Build artifacts
.output/
*.o
*.skel.h
# Generated binaries
tcp_quarantine
# Editor files
*.swp
*~
.vscode/

View File

@@ -0,0 +1,92 @@
# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
OUTPUT := .output
CLANG ?= clang
LIBBPF_SRC := $(abspath ../third_party/libbpf/src)
BPFTOOL_SRC := $(abspath ../third_party/bpftool/src)
LIBBPF_OBJ := $(abspath $(OUTPUT)/libbpf.a)
BPFTOOL_OUTPUT ?= $(abspath $(OUTPUT)/bpftool)
BPFTOOL ?= $(BPFTOOL_OUTPUT)/bootstrap/bpftool
ARCH ?= $(shell uname -m | sed 's/x86_64/x86/' \
| sed 's/arm.*/arm/' \
| sed 's/aarch64/arm64/' \
| sed 's/ppc64le/powerpc/' \
| sed 's/mips.*/mips/' \
| sed 's/riscv64/riscv/' \
| sed 's/loongarch64/loongarch/')
VMLINUX := ../third_party/vmlinux/$(ARCH)/vmlinux.h
INCLUDES := -I$(OUTPUT) -I../third_party/libbpf/include/uapi -I$(dir $(VMLINUX)) -I.
CFLAGS := -g -Wall
ALL_LDFLAGS := $(LDFLAGS) $(EXTRA_LDFLAGS)
APPS = tcp_quarantine
CLANG_BPF_SYS_INCLUDES ?= $(shell $(CLANG) -v -E - </dev/null 2>&1 \
| sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }')
ifeq ($(V),1)
Q =
msg =
else
Q = @
msg = @printf ' %-8s %s%s\n' "$(1)" \
"$(patsubst $(abspath $(OUTPUT))/%,%,$(2))" \
"$(if $(3), $(3))";
MAKEFLAGS += --no-print-directory
endif
define allow-override
$(if $(or $(findstring environment,$(origin $(1))),\
$(findstring command line,$(origin $(1)))),,\
$(eval $(1) = $(2)))
endef
$(call allow-override,CC,$(CROSS_COMPILE)cc)
$(call allow-override,LD,$(CROSS_COMPILE)ld)
.PHONY: all clean test
all: $(APPS)
clean:
$(call msg,CLEAN)
$(Q)rm -rf $(OUTPUT) $(APPS)
test: all
python3 tests/test_tcp_quarantine.py ./tcp_quarantine
$(OUTPUT) $(OUTPUT)/libbpf $(BPFTOOL_OUTPUT):
$(call msg,MKDIR,$@)
$(Q)mkdir -p $@
$(LIBBPF_OBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)/libbpf
$(call msg,LIB,$@)
$(Q)$(MAKE) -C $(LIBBPF_SRC) BUILD_STATIC_ONLY=1 \
OBJDIR=$(dir $@)/libbpf DESTDIR=$(dir $@) \
INCLUDEDIR= LIBDIR= UAPIDIR= install
$(BPFTOOL): | $(BPFTOOL_OUTPUT)
$(call msg,BPFTOOL,$@)
$(Q)$(MAKE) ARCH= CROSS_COMPILE= OUTPUT=$(BPFTOOL_OUTPUT)/ -C $(BPFTOOL_SRC) bootstrap
$(OUTPUT)/%.bpf.o: %.bpf.c $(LIBBPF_OBJ) $(wildcard %.h) $(VMLINUX) | $(OUTPUT) $(BPFTOOL)
$(call msg,BPF,$@)
$(Q)$(CLANG) -g -O2 -target bpf -D__TARGET_ARCH_$(ARCH) \
$(INCLUDES) $(CLANG_BPF_SYS_INCLUDES) \
-c $(filter %.c,$^) -o $(patsubst %.bpf.o,%.tmp.bpf.o,$@)
$(Q)$(BPFTOOL) gen object $@ $(patsubst %.bpf.o,%.tmp.bpf.o,$@)
$(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(OUTPUT) $(BPFTOOL)
$(call msg,GEN-SKEL,$@)
$(Q)$(BPFTOOL) gen skeleton $< > $@
$(patsubst %,$(OUTPUT)/%.o,$(APPS)): %.o: %.skel.h
$(OUTPUT)/%.o: %.c $(wildcard %.h) | $(OUTPUT)
$(call msg,CC,$@)
$(Q)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
$(APPS): %: $(OUTPUT)/%.o $(LIBBPF_OBJ) | $(OUTPUT)
$(call msg,BINARY,$@)
$(Q)$(CC) $(CFLAGS) $^ $(ALL_LDFLAGS) -lelf -lz -o $@
.DELETE_ON_ERROR:
.SECONDARY:

View File

@@ -0,0 +1,419 @@
# eBPF Tutorial: Precisely Isolating Established TCP Connections
Suppose an outbound destination address has just been added to a threat-intelligence blocklist. The firewall has updated its rules to block future connections. However, `netstat` shows that a server already has an active TCP session to that address -- you need to close that exact connection while preserving the process and all unrelated traffic.
This tutorial demonstrates how to achieve this using a BPF iterator and the `bpf_sock_destroy` kfunc. The tool traverses the kernel's TCP socket table, finds established connections that exactly match an IPv4 destination address and port, reports match counts in dry-run mode, and destroys the selected sockets in `--apply` mode.
> Complete source: <https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/51-tcp-quarantine>
## Why Kernel-Level Connection Isolation Is Needed
Traditional approaches to terminating connections all have significant limitations.
**Killing the process** is the most direct approach, but a single process often maintains multiple connections. Killing it disrupts all business traffic, and the process may immediately reconnect to the same destination after restart.
**Firewall rules** can block new connections but are powerless against already-established ones. The `-m state --state ESTABLISHED` rule in `iptables` or `nftables` can only match state -- it cannot actively sever existing sessions.
**User-space tools** such as `ss --kill` or `tcpkill` rely on traversing `/proc/net/tcp` and injecting RST packets. This approach has a race condition: between reading the socket list and sending the RST, the socket state may have already changed. Furthermore, injecting RST requires the correct sequence number and may be ineffective against encrypted connections or certain protocol stack configurations.
**A kernel-level approach** is needed to truly solve this problem. BPF iterators can traverse the kernel's socket table while holding appropriate locks. The `bpf_sock_destroy` kfunc directly invokes the kernel's socket destruction path. The entire process is atomic with no race window. This is the significance of Linux 6.5 introducing `bpf_sock_destroy`: it allows BPF programs to forcibly close sockets from an iterator context without relying on user-space RST injection.
## BPF Iterator and bpf_sock_destroy
A BPF TCP iterator is a special type of BPF program that drives typed callbacks for each TCP socket in the loading process's network namespace. User space triggers the traversal by reading the iterator file descriptor, and the kernel invokes the BPF callback once per socket during traversal.
The `bpf_sock_destroy` kfunc was introduced in Linux 6.5 (commit `4ddbcb886268af8d12a23e6640b39d1d9c652b1b`), allowing BPF programs to forcibly close sockets from an iterator context. The kernel executes the protocol-specific destroy path. This is a synchronous operation -- the socket is destroyed by the time the call returns.
The `tcp_quarantine` tool combines these two capabilities. The user specifies a target IPv4 address and port, and the BPF program traverses the socket table, filtering for `ESTABLISHED` connections with an exact destination match. Dry-run mode only counts matches; apply mode calls `bpf_sock_destroy` to destroy matching sockets. The entire process completes in kernel space. User space only needs to read the iterator file descriptor to drive the traversal, then read the statistics counters from the BSS section afterward.
## Code Implementation
This tool consists of three files: a shared header defining the statistics structure, a BPF program that traverses the socket table and performs destruction, and a user-space loader that manages the lifecycle and prints results.
### Shared Header
`tcp_quarantine.h` defines the statistics structure shared between BPF and user space, residing in the BSS section as a result channel carrying five counters.
```c
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __TCP_QUARANTINE_H
#define __TCP_QUARANTINE_H
struct quarantine_stats {
unsigned long long scanned;
unsigned long long established;
unsigned long long matched;
unsigned long long destroyed;
unsigned long long failed;
};
#endif /* __TCP_QUARANTINE_H */
```
The five counters record: total sockets scanned, IPv4 sockets in `ESTABLISHED` state, sockets with exact destination address and port matches, sockets successfully destroyed in apply mode, and `bpf_sock_destroy` calls that returned nonzero.
### BPF Program
`tcp_quarantine.bpf.c` uses `SEC("iter/tcp")` to declare itself as a TCP iterator type. When user space reads the iterator file descriptor, the kernel calls the callback once for each TCP socket in the loading process's network namespace.
```c
// SPDX-License-Identifier: GPL-2.0
#include "vmlinux.h"
#include <bpf/bpf_core_read.h>
#include <bpf/bpf_endian.h>
#include <bpf/bpf_helpers.h>
#include "tcp_quarantine.h"
#define AF_INET 2
#define TCP_ESTABLISHED 1
char LICENSE[] SEC("license") = "GPL";
const volatile __u32 target_addr;
const volatile __u16 target_port;
const volatile bool apply;
struct quarantine_stats stats;
extern int bpf_sock_destroy(struct sock_common *sock) __ksym;
SEC("iter/tcp")
int quarantine_tcp(struct bpf_iter__tcp *ctx)
{
struct sock_common *sk = ctx->sk_common;
__u32 dst_addr;
__u16 dst_port;
__u16 family;
__u8 state;
int err;
if (!sk)
return 0;
stats.scanned++;
family = BPF_CORE_READ(sk, skc_family);
state = BPF_CORE_READ(sk, skc_state);
if (family != AF_INET || state != TCP_ESTABLISHED)
return 0;
stats.established++;
dst_addr = BPF_CORE_READ(sk, skc_daddr);
dst_port = BPF_CORE_READ(sk, skc_dport);
if (dst_addr != target_addr || dst_port != bpf_htons(target_port))
return 0;
stats.matched++;
if (!apply)
return 0;
err = bpf_sock_destroy(sk);
if (err)
stats.failed++;
else
stats.destroyed++;
return 0;
}
```
The program structure is straightforward. Three `const volatile` variables reside in the `.rodata` section. User space writes the target address, port, and apply flag after `open()` but before `load()`, and the verifier treats them as compile-time constants. `bpf_sock_destroy` is declared via `extern ... __ksym` as a strong kfunc symbol, which the kernel resolves at load time.
The callback logic uses three-level filtering. First, for each non-null `sk`, it increments the `scanned` counter, then selects sockets that are `AF_INET` and in `TCP_ESTABLISHED` state for the `established` count. The `BPF_CORE_READ` macro uses BTF information to read `sock_common` structure fields. CO-RE relocations ensure the same compiled artifact runs across different kernel versions and field layouts.
The second filter checks destination address and port. The destination address is compared directly in network byte order (the result stored by `inet_pton` is already in network byte order). Port comparison uses `bpf_htons(target_port)` because `skc_dport` is stored in network byte order.
Matching sockets increment `matched`. Dry-run mode returns at this point. Apply mode calls `bpf_sock_destroy(sk)`, incrementing `destroyed` on success or `failed` on nonzero return.
### User-Space Loader
`tcp_quarantine.c` parses command-line arguments, configures BPF constants, runs the iterator, and prints results.
```c
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <arpa/inet.h>
#include <errno.h>
#include <getopt.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <bpf/bpf.h>
#include <bpf/libbpf.h>
#include "tcp_quarantine.h"
#include "tcp_quarantine.skel.h"
static struct env {
const char *destination;
unsigned int port;
bool apply;
bool verbose;
} env;
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s --destination IPv4 --port PORT [--apply] [--verbose]\n"
"\n"
"Find established TCP client connections to an exact destination.\n"
"The default is a safe dry run; --apply destroys matching sockets.\n"
"\n"
"Options:\n"
" -d, --destination IPv4 exact remote IPv4 address\n"
" -p, --port PORT exact remote TCP port (1-65535)\n"
" -a, --apply destroy matching sockets\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_port(const char *value, unsigned int *port)
{
char *end = NULL;
unsigned long parsed;
errno = 0;
parsed = strtoul(value, &end, 10);
if (errno || !end || *end || parsed == 0 || parsed > 65535)
return -EINVAL;
*port = parsed;
return 0;
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "destination", required_argument, NULL, 'd' },
{ "port", required_argument, NULL, 'p' },
{ "apply", no_argument, NULL, 'a' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
int option;
while ((option = getopt_long(argc, argv, "d:p:avh", options, NULL)) != -1) {
switch (option) {
case 'd':
env.destination = optarg;
break;
case 'p':
if (parse_port(optarg, &env.port)) {
fprintf(stderr, "invalid TCP port: %s\n", optarg);
return -EINVAL;
}
break;
case 'a':
env.apply = true;
break;
case 'v':
env.verbose = true;
break;
case 'h':
usage(argv[0]);
exit(0);
default:
return -EINVAL;
}
}
if (!env.destination || !env.port || optind != argc)
return -EINVAL;
return 0;
}
static int run_iterator(struct bpf_program *program)
{
struct bpf_link *link;
char buffer[256];
int iter_fd, length, err;
link = bpf_program__attach_iter(program, NULL);
err = libbpf_get_error(link);
if (err) {
fprintf(stderr, "failed to attach TCP iterator: %s\n", strerror(-err));
return err;
}
iter_fd = bpf_iter_create(bpf_link__fd(link));
if (iter_fd < 0) {
err = iter_fd;
fprintf(stderr, "failed to create TCP iterator: %s\n", strerror(-err));
goto cleanup;
}
while ((length = read(iter_fd, buffer, sizeof(buffer))) > 0)
;
if (length < 0) {
err = -errno;
fprintf(stderr, "failed while scanning TCP sockets: %s\n", strerror(errno));
}
close(iter_fd);
cleanup:
bpf_link__destroy(link);
return err;
}
int main(int argc, char **argv)
{
struct tcp_quarantine_bpf *skel = NULL;
struct in_addr destination;
int err;
err = parse_args(argc, argv);
if (err) {
usage(argv[0]);
return 1;
}
if (inet_pton(AF_INET, env.destination, &destination) != 1) {
fprintf(stderr, "invalid IPv4 destination: %s\n", env.destination);
return 1;
}
libbpf_set_print(libbpf_print_fn);
skel = tcp_quarantine_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return 1;
}
skel->rodata->target_addr = destination.s_addr;
skel->rodata->target_port = env.port;
skel->rodata->apply = env.apply;
err = tcp_quarantine_bpf__load(skel);
if (err) {
fprintf(stderr,
"failed to load TCP quarantine program: %s\n"
"This tool requires Linux 6.5+ with BTF, TCP BPF iterators, "
"BPF JIT, and the bpf_sock_destroy kfunc.\n",
strerror(-err));
goto cleanup;
}
err = run_iterator(skel->progs.quarantine_tcp);
if (err)
goto cleanup;
printf("mode=%s destination=%s:%u scanned=%llu established=%llu "
"matched=%llu destroyed=%llu failed=%llu\n",
env.apply ? "apply" : "dry-run", env.destination, env.port,
skel->bss->stats.scanned, skel->bss->stats.established,
skel->bss->stats.matched, skel->bss->stats.destroyed,
skel->bss->stats.failed);
if (skel->bss->stats.failed)
err = -EIO;
cleanup:
tcp_quarantine_bpf__destroy(skel);
return err != 0;
}
```
The loader follows a standard flow. After parsing command-line arguments, it validates the IPv4 address with `inet_pton` and converts it to network byte order. After opening the skeleton, it writes three parameters to the `.rodata` section, then calls `load` to complete BPF program loading. The load failure diagnostic provides both the kernel errno and a prerequisite checklist to help users troubleshoot.
The `run_iterator` function is the core driver for traversal. It first attaches the BPF program with `bpf_program__attach_iter`, then creates an iterator file descriptor with `bpf_iter_create`. The loop reads that fd until EOF, with each `read` driving the kernel to invoke the BPF callback for a batch of TCP sockets. The content of the read buffer is irrelevant in this scenario -- it only serves to drive traversal. The actual results are conveyed through the BSS section counters. After iteration completes, the loader reads counters from `skel->bss->stats` and prints a single-line summary.
## Network Namespaces
The TCP BPF iterator scans the network namespace of the process that loads the BPF program. This is an important boundary: the tool can only see and operate on connections within its own network namespace.
To isolate connections inside a container or another network namespace, first enter the target namespace with `nsenter`:
```bash
sudo nsenter --net=/proc/<target-pid>/ns/net -- ./tcp_quarantine \
--destination 203.0.113.99 --port 443 --apply
```
`<target-pid>` is the PID of any process in the target network namespace. `nsenter --net` changes only the network namespace; the caller's mount namespace is retained, so the binary and working directory remain accessible.
## bpf_sock_destroy Semantics
`bpf_sock_destroy` directly invokes the kernel's socket destruction path, which is fundamentally different from user-space RST packet injection. The kernel executes protocol-specific cleanup logic, removes the socket from hash tables, and releases associated resources.
The error seen by the application depends on which endpoint is destroyed and what operation it attempts next. The upstream TCP selftest expects `ECONNABORTED` on the client side and `ECONNRESET` on the server side. This tutorial's test also accepts `EPIPE` on the send side, as the exact error code may vary depending on operation order and kernel version.
## Compilation and Execution
Build from source:
```bash
cd src/51-tcp-quarantine
make clean
make -j2
```
Dry-run mode reports matches; matching connections remain established:
```bash
sudo ./tcp_quarantine --destination 127.0.0.1 --port 42063
```
Apply mode destroys matching sockets:
```bash
sudo ./tcp_quarantine --destination 127.0.0.1 --port 42063 --apply
```
Command-line options:
```text
Usage: ./tcp_quarantine --destination IPv4 --port PORT [--apply] [--verbose]
Options:
-d, --destination IPv4 exact remote IPv4 address
-p, --port PORT exact remote TCP port (1-65535)
-a, --apply destroy matching sockets
-v, --verbose print libbpf diagnostics
-h, --help show this help
```
The following output was captured on an x86_64 environment running kernel `7.0.0-rc2+`. The automated test creates two loopback TCP connection pairs: one target port and one unrelated control port. It performs request/response round trips before running the tool to verify behavior:
```console
mode=dry-run destination=127.0.0.1:42063 scanned=6 established=4 matched=1 destroyed=0 failed=0
mode=apply destination=127.0.0.1:42063 scanned=6 established=4 matched=1 destroyed=1 failed=0
PASS: dry-run preserved both connections; apply destroyed only the target
```
Socket table counts and ephemeral port numbers vary with each run. The test assertions focus on: match count is 1, dry-run preserves connections, apply destroys the target, destroy failures are 0, and the control connection survives.
Run the test:
```bash
sudo make test
```
### Environment Requirements
| Requirement | Details |
|---|---|
| Kernel | Linux 6.5+ (when `bpf_sock_destroy` was first introduced) |
| BTF | Required (`CONFIG_DEBUG_INFO_BTF=y`) |
| Kernel configs | `CONFIG_BPF=y`, `CONFIG_BPF_SYSCALL=y`, `CONFIG_BPF_JIT=y`, `CONFIG_BPF_EVENTS=y`, `CONFIG_DEBUG_INFO_BTF=y`, `CONFIG_INET=y`, `CONFIG_PROC_FS=y` |
| BPF JIT | Must be enabled at runtime. Programs calling kfuncs use the JIT path; `CONFIG_BPF_JIT_ALWAYS_ON` kernels require no sysctl setting. |
| Architecture | Tested on x86_64 |
| Privileges | root |
## Summary
This tutorial demonstrated how to use a BPF iterator and the `bpf_sock_destroy` kfunc to implement precise TCP connection isolation. Compared to killing processes, firewall rules, or user-space RST injection, this kernel-level approach is atomic with no race window and no dependency on correct sequence numbers.
The tool is designed for single-pass traversal with exact matching on one IPv4 destination tuple. Future extensions could add IPv6 support, continuous policy input, process/cgroup attribution, multi-namespace orchestration, authorization controls, and audit logging.
> To learn more about eBPF, check out our tutorial repository at <https://github.com/eunomia-bpf/bpf-developer-tutorial> or visit our website at <https://eunomia.dev/tutorials/>.
## References
- [bpf_sock_destroy kernel commit](https://github.com/torvalds/linux/commit/4ddbcb886268af8d12a23e6640b39d1d9c652b1b)
- [Upstream sock_destroy selftest (BPF program)](https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/sock_destroy_prog.c)
- [Upstream sock_destroy selftest (userspace test)](https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/prog_tests/sock_destroy.c)

View File

@@ -0,0 +1,419 @@
# eBPF 教程:精准隔离已建立的 TCP 连接
假设某个出站目标地址刚被添加到威胁情报黑名单,防火墙已更新规则阻止后续连接。然而 `netstat` 显示某台服务器上已经存在一条到该地址的活跃 TCP 会话——你需要精准关闭这条连接,同时保留进程和其他无关流量。
本教程展示如何使用 eBPF 迭代器和 `bpf_sock_destroy` kfunc 实现这一目标。工具遍历内核的 TCP 套接字表,找出精确匹配 IPv4 目标地址和端口的已建立连接,dry-run 模式统计匹配数量,`--apply` 模式销毁选中的套接字。
> 完整源码:<https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/51-tcp-quarantine>
## 为什么需要内核级别的连接隔离
传统的连接终止方案都有明显的局限性。
**杀掉进程**是最直接的想法,但一个进程往往维护着多条连接,杀进程会中断所有业务流量,而且进程重启后可能立刻重建到同一目标的连接。
**防火墙规则**可以阻止新连接,但对已建立的连接无能为力。`iptables` 或 `nftables` 的 `-m state --state ESTABLISHED` 规则只能匹配状态,不能主动断开现有会话。
**用户态工具**如 `ss --kill` 或 `tcpkill` 依赖 `/proc/net/tcp` 遍历和注入 RST 报文,但这种方式有竞态问题:在你读取套接字列表和发送 RST 之间,套接字状态可能已经变化。而且注入 RST 需要正确的序列号,对加密连接或某些协议栈配置可能无效。
**内核态方案**才能真正解决这个问题。BPF 迭代器可以在持有适当锁的情况下遍历内核的套接字表,`bpf_sock_destroy` kfunc 直接调用内核的套接字销毁路径,整个过程是原子的,不存在竞态窗口。这就是 Linux 6.5 引入 `bpf_sock_destroy` 的意义:让 BPF 程序能够在迭代器上下文中强制关闭套接字,无需依赖用户态的 RST 注入。
## BPF 迭代器与 bpf_sock_destroy
BPF TCP 迭代器是一种特殊的 BPF 程序类型,它为加载进程所在网络命名空间内的每个 TCP 套接字驱动类型化回调。用户态程序通过读取迭代器文件描述符来触发遍历,内核在遍历过程中为每个套接字调用一次 BPF 回调。
`bpf_sock_destroy` kfunc 由 Linux 6.5 引入(commit `4ddbcb886268af8d12a23e6640b39d1d9c652b1b`),允许 BPF 程序在迭代器上下文中强制关闭套接字。内核执行协议特定的销毁路径,这是一个同步操作,调用返回时套接字已经被销毁。
`tcp_quarantine` 工具把这两个能力组合起来。用户指定目标 IPv4 地址和端口,BPF 程序遍历套接字表,筛选出 `ESTABLISHED` 状态且目标地址完全匹配的连接。dry-run 模式只统计匹配数量,apply 模式调用 `bpf_sock_destroy` 销毁匹配的套接字。整个过程在内核中完成,用户态只需要读取迭代器文件描述符来驱动遍历,最后从 BSS 段读取统计计数器。
## 代码实现
本工具由三个文件组成:共享头文件定义统计结构、BPF 程序遍历套接字表并执行销毁、用户空间加载器管理生命周期并打印结果。
### 共享头文件
`tcp_quarantine.h` 定义了 BPF 和用户空间共享的统计结构,位于 BSS 段作为结果通道传递五个计数器。
```c
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __TCP_QUARANTINE_H
#define __TCP_QUARANTINE_H
struct quarantine_stats {
unsigned long long scanned;
unsigned long long established;
unsigned long long matched;
unsigned long long destroyed;
unsigned long long failed;
};
#endif /* __TCP_QUARANTINE_H */
```
五个计数器分别记录:扫描的套接字总数、处于 `ESTABLISHED` 状态的 IPv4 套接字数、目标地址和端口完全匹配的数量、apply 模式下成功销毁的数量、以及 `bpf_sock_destroy` 返回非零的数量。
### BPF 程序
`tcp_quarantine.bpf.c` 使用 `SEC("iter/tcp")` 声明为 TCP 迭代器类型。当用户态读取迭代器文件描述符时,内核会为加载进程所在网络命名空间中的每个 TCP 套接字调用一次回调。
```c
// SPDX-License-Identifier: GPL-2.0
#include "vmlinux.h"
#include <bpf/bpf_core_read.h>
#include <bpf/bpf_endian.h>
#include <bpf/bpf_helpers.h>
#include "tcp_quarantine.h"
#define AF_INET 2
#define TCP_ESTABLISHED 1
char LICENSE[] SEC("license") = "GPL";
const volatile __u32 target_addr;
const volatile __u16 target_port;
const volatile bool apply;
struct quarantine_stats stats;
extern int bpf_sock_destroy(struct sock_common *sock) __ksym;
SEC("iter/tcp")
int quarantine_tcp(struct bpf_iter__tcp *ctx)
{
struct sock_common *sk = ctx->sk_common;
__u32 dst_addr;
__u16 dst_port;
__u16 family;
__u8 state;
int err;
if (!sk)
return 0;
stats.scanned++;
family = BPF_CORE_READ(sk, skc_family);
state = BPF_CORE_READ(sk, skc_state);
if (family != AF_INET || state != TCP_ESTABLISHED)
return 0;
stats.established++;
dst_addr = BPF_CORE_READ(sk, skc_daddr);
dst_port = BPF_CORE_READ(sk, skc_dport);
if (dst_addr != target_addr || dst_port != bpf_htons(target_port))
return 0;
stats.matched++;
if (!apply)
return 0;
err = bpf_sock_destroy(sk);
if (err)
stats.failed++;
else
stats.destroyed++;
return 0;
}
```
程序结构很直接。三个 `const volatile` 变量位于 `.rodata` 段,用户态在 `open()` 之后、`load()` 之前写入目标地址、端口和 apply 标志,验证器将它们视为编译期常量。`bpf_sock_destroy` 通过 `extern ... __ksym` 声明为强 kfunc 符号,内核在加载时解析该符号。
回调的处理逻辑分为三层过滤。首先对每个非空 `sk` 递增 `scanned` 计数器,然后选出 `AF_INET` 且状态为 `TCP_ESTABLISHED` 的套接字计入 `established`。`BPF_CORE_READ` 宏利用 BTF 信息读取 `sock_common` 结构体的字段,CO-RE 重定位保证同一编译产物可以在不同内核版本和字段布局上运行。
第二层过滤检查目标地址和端口。目标地址直接比较网络字节序的值(`inet_pton` 存储的结果已经是网络字节序),端口比较使用 `bpf_htons(target_port)` 因为 `skc_dport` 以网络字节序存储。
匹配的套接字递增 `matched`,dry-run 模式到此返回。apply 模式调用 `bpf_sock_destroy(sk)`,成功时递增 `destroyed`,返回非零时递增 `failed`。
### 用户空间加载器
`tcp_quarantine.c` 解析命令行参数、配置 BPF 常量、运行迭代器并打印结果。
```c
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <arpa/inet.h>
#include <errno.h>
#include <getopt.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <bpf/bpf.h>
#include <bpf/libbpf.h>
#include "tcp_quarantine.h"
#include "tcp_quarantine.skel.h"
static struct env {
const char *destination;
unsigned int port;
bool apply;
bool verbose;
} env;
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s --destination IPv4 --port PORT [--apply] [--verbose]\n"
"\n"
"Find established TCP client connections to an exact destination.\n"
"The default is a safe dry run; --apply destroys matching sockets.\n"
"\n"
"Options:\n"
" -d, --destination IPv4 exact remote IPv4 address\n"
" -p, --port PORT exact remote TCP port (1-65535)\n"
" -a, --apply destroy matching sockets\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_port(const char *value, unsigned int *port)
{
char *end = NULL;
unsigned long parsed;
errno = 0;
parsed = strtoul(value, &end, 10);
if (errno || !end || *end || parsed == 0 || parsed > 65535)
return -EINVAL;
*port = parsed;
return 0;
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "destination", required_argument, NULL, 'd' },
{ "port", required_argument, NULL, 'p' },
{ "apply", no_argument, NULL, 'a' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
int option;
while ((option = getopt_long(argc, argv, "d:p:avh", options, NULL)) != -1) {
switch (option) {
case 'd':
env.destination = optarg;
break;
case 'p':
if (parse_port(optarg, &env.port)) {
fprintf(stderr, "invalid TCP port: %s\n", optarg);
return -EINVAL;
}
break;
case 'a':
env.apply = true;
break;
case 'v':
env.verbose = true;
break;
case 'h':
usage(argv[0]);
exit(0);
default:
return -EINVAL;
}
}
if (!env.destination || !env.port || optind != argc)
return -EINVAL;
return 0;
}
static int run_iterator(struct bpf_program *program)
{
struct bpf_link *link;
char buffer[256];
int iter_fd, length, err;
link = bpf_program__attach_iter(program, NULL);
err = libbpf_get_error(link);
if (err) {
fprintf(stderr, "failed to attach TCP iterator: %s\n", strerror(-err));
return err;
}
iter_fd = bpf_iter_create(bpf_link__fd(link));
if (iter_fd < 0) {
err = iter_fd;
fprintf(stderr, "failed to create TCP iterator: %s\n", strerror(-err));
goto cleanup;
}
while ((length = read(iter_fd, buffer, sizeof(buffer))) > 0)
;
if (length < 0) {
err = -errno;
fprintf(stderr, "failed while scanning TCP sockets: %s\n", strerror(errno));
}
close(iter_fd);
cleanup:
bpf_link__destroy(link);
return err;
}
int main(int argc, char **argv)
{
struct tcp_quarantine_bpf *skel = NULL;
struct in_addr destination;
int err;
err = parse_args(argc, argv);
if (err) {
usage(argv[0]);
return 1;
}
if (inet_pton(AF_INET, env.destination, &destination) != 1) {
fprintf(stderr, "invalid IPv4 destination: %s\n", env.destination);
return 1;
}
libbpf_set_print(libbpf_print_fn);
skel = tcp_quarantine_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return 1;
}
skel->rodata->target_addr = destination.s_addr;
skel->rodata->target_port = env.port;
skel->rodata->apply = env.apply;
err = tcp_quarantine_bpf__load(skel);
if (err) {
fprintf(stderr,
"failed to load TCP quarantine program: %s\n"
"This tool requires Linux 6.5+ with BTF, TCP BPF iterators, "
"BPF JIT, and the bpf_sock_destroy kfunc.\n",
strerror(-err));
goto cleanup;
}
err = run_iterator(skel->progs.quarantine_tcp);
if (err)
goto cleanup;
printf("mode=%s destination=%s:%u scanned=%llu established=%llu "
"matched=%llu destroyed=%llu failed=%llu\n",
env.apply ? "apply" : "dry-run", env.destination, env.port,
skel->bss->stats.scanned, skel->bss->stats.established,
skel->bss->stats.matched, skel->bss->stats.destroyed,
skel->bss->stats.failed);
if (skel->bss->stats.failed)
err = -EIO;
cleanup:
tcp_quarantine_bpf__destroy(skel);
return err != 0;
}
```
加载器的流程比较标准。解析命令行后,用 `inet_pton` 验证 IPv4 地址并转换为网络字节序。打开 skeleton 后把三个参数写入 `.rodata` 段,然后调用 `load` 完成 BPF 程序加载。加载失败时的诊断同时给出内核 errno 和前置条件列表,方便用户排查问题。
`run_iterator` 函数是驱动遍历的核心。它先用 `bpf_program__attach_iter` 挂载 BPF 程序,然后用 `bpf_iter_create` 创建迭代器文件描述符。循环读取该 fd 直到 EOF,每次 `read` 驱动内核为若干 TCP 套接字调用 BPF 回调。读取缓冲区的内容在这个场景下不重要,它只是用于驱动遍历,真正的结果通过 BSS 段的计数器传递。迭代完成后,加载器从 `skel->bss->stats` 读取计数器并打印单行汇总。
## 网络命名空间
TCP BPF 迭代器扫描的是加载 BPF 程序的进程所在的网络命名空间。这是一个重要的边界:工具只能看到和操作自己网络命名空间中的连接。
要隔离容器或其他网络命名空间中的连接,先用 `nsenter` 进入目标命名空间:
```bash
sudo nsenter --net=/proc/<target-pid>/ns/net -- ./tcp_quarantine \
--destination 203.0.113.99 --port 443 --apply
```
`<target-pid>` 是目标网络命名空间中任意进程的 PID。`nsenter --net` 只改变网络命名空间,调用者的 mount 命名空间保持不变,因此二进制文件和当前工作目录仍可访问。
## bpf_sock_destroy 语义
`bpf_sock_destroy` 直接调用内核的套接字销毁路径,这与用户态注入 RST 报文有本质区别。内核执行协议特定的清理逻辑,套接字从哈希表中移除,相关资源被释放。
应用层看到的错误取决于被销毁的端点和下一个操作。上游 TCP selftest 预期客户端收到 `ECONNABORTED`,服务端收到 `ECONNRESET`。本教程的测试还接受发送端的 `EPIPE`,因为具体错误码可能因操作顺序和内核版本而异。
## 编译与运行
从源码构建:
```bash
cd src/51-tcp-quarantine
make clean
make -j2
```
Dry-run 模式报告匹配结果,匹配的连接保持已建立状态:
```bash
sudo ./tcp_quarantine --destination 127.0.0.1 --port 42063
```
Apply 模式销毁匹配的套接字:
```bash
sudo ./tcp_quarantine --destination 127.0.0.1 --port 42063 --apply
```
命令行参数:
```text
Usage: ./tcp_quarantine --destination IPv4 --port PORT [--apply] [--verbose]
Options:
-d, --destination IPv4 精确的远程 IPv4 地址
-p, --port PORT 精确的远程 TCP 端口 (1-65535)
-a, --apply 销毁匹配的套接字
-v, --verbose 打印 libbpf 诊断信息
-h, --help 显示帮助
```
以下输出在运行内核 `7.0.0-rc2+` 的 x86_64 环境中采集。自动化测试创建两组回环 TCP 连接对:一个目标端口和一个无关的控制端口,先执行请求/响应往返,再运行工具验证行为:
```console
mode=dry-run destination=127.0.0.1:42063 scanned=6 established=4 matched=1 destroyed=0 failed=0
mode=apply destination=127.0.0.1:42063 scanned=6 established=4 matched=1 destroyed=1 failed=0
PASS: dry-run preserved both connections; apply destroyed only the target
```
套接字表计数和临时端口号随每次运行而变化,测试断言关注的是:匹配数为 1、dry-run 保留连接、apply 销毁目标、销毁失败数为 0、控制连接存活。
运行测试:
```bash
sudo make test
```
### 环境要求
| 要求 | 详情 |
|---|---|
| 内核版本 | Linux 6.5+(`bpf_sock_destroy` 首次引入) |
| BTF | 必须启用 (`CONFIG_DEBUG_INFO_BTF=y`) |
| 内核配置 | `CONFIG_BPF=y`, `CONFIG_BPF_SYSCALL=y`, `CONFIG_BPF_JIT=y`, `CONFIG_BPF_EVENTS=y`, `CONFIG_DEBUG_INFO_BTF=y`, `CONFIG_INET=y`, `CONFIG_PROC_FS=y` |
| BPF JIT | 运行时必须启用。调用 kfunc 的程序使用 JIT 路径;`CONFIG_BPF_JIT_ALWAYS_ON` 内核无需 sysctl 设置。 |
| 架构 | 已在 x86_64 上测试 |
| 权限 | root |
## 总结
本教程展示了如何使用 BPF 迭代器和 `bpf_sock_destroy` kfunc 实现精确的 TCP 连接隔离。相比杀进程、防火墙规则或用户态 RST 注入,这种内核态方案是原子的,不存在竞态窗口,也不依赖正确的序列号。
工具的设计是单次遍历、精确匹配一个 IPv4 目标元组。后续扩展可以添加 IPv6 支持、持续策略输入、进程/cgroup 归属、多命名空间编排、授权控制和审计记录。
> 如果你想深入了解 eBPF,请查看我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或访问我们的网站 <https://eunomia.dev/tutorials/>。
## 参考资料
- [bpf_sock_destroy 内核 commit](https://github.com/torvalds/linux/commit/4ddbcb886268af8d12a23e6640b39d1d9c652b1b)
- [上游 sock_destroy selftest(BPF 程序)](https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/sock_destroy_prog.c)
- [上游 sock_destroy selftest(用户态测试)](https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/prog_tests/sock_destroy.c)

View File

@@ -0,0 +1,57 @@
// SPDX-License-Identifier: GPL-2.0
#include "vmlinux.h"
#include <bpf/bpf_core_read.h>
#include <bpf/bpf_endian.h>
#include <bpf/bpf_helpers.h>
#include "tcp_quarantine.h"
#define AF_INET 2
#define TCP_ESTABLISHED 1
char LICENSE[] SEC("license") = "GPL";
const volatile __u32 target_addr;
const volatile __u16 target_port;
const volatile bool apply;
struct quarantine_stats stats;
extern int bpf_sock_destroy(struct sock_common *sock) __ksym;
SEC("iter/tcp")
int quarantine_tcp(struct bpf_iter__tcp *ctx)
{
struct sock_common *sk = ctx->sk_common;
__u32 dst_addr;
__u16 dst_port;
__u16 family;
__u8 state;
int err;
if (!sk)
return 0;
stats.scanned++;
family = BPF_CORE_READ(sk, skc_family);
state = BPF_CORE_READ(sk, skc_state);
if (family != AF_INET || state != TCP_ESTABLISHED)
return 0;
stats.established++;
dst_addr = BPF_CORE_READ(sk, skc_daddr);
dst_port = BPF_CORE_READ(sk, skc_dport);
if (dst_addr != target_addr || dst_port != bpf_htons(target_port))
return 0;
stats.matched++;
if (!apply)
return 0;
err = bpf_sock_destroy(sk);
if (err)
stats.failed++;
else
stats.destroyed++;
return 0;
}

View File

@@ -0,0 +1,188 @@
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <arpa/inet.h>
#include <errno.h>
#include <getopt.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <bpf/bpf.h>
#include <bpf/libbpf.h>
#include "tcp_quarantine.h"
#include "tcp_quarantine.skel.h"
static struct env {
const char *destination;
unsigned int port;
bool apply;
bool verbose;
} env;
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s --destination IPv4 --port PORT [--apply] [--verbose]\n"
"\n"
"Find established TCP client connections to an exact destination.\n"
"The default is a safe dry run; --apply destroys matching sockets.\n"
"\n"
"Options:\n"
" -d, --destination IPv4 exact remote IPv4 address\n"
" -p, --port PORT exact remote TCP port (1-65535)\n"
" -a, --apply destroy matching sockets\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_port(const char *value, unsigned int *port)
{
char *end = NULL;
unsigned long parsed;
errno = 0;
parsed = strtoul(value, &end, 10);
if (errno || !end || *end || parsed == 0 || parsed > 65535)
return -EINVAL;
*port = parsed;
return 0;
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "destination", required_argument, NULL, 'd' },
{ "port", required_argument, NULL, 'p' },
{ "apply", no_argument, NULL, 'a' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
int option;
while ((option = getopt_long(argc, argv, "d:p:avh", options, NULL)) != -1) {
switch (option) {
case 'd':
env.destination = optarg;
break;
case 'p':
if (parse_port(optarg, &env.port)) {
fprintf(stderr, "invalid TCP port: %s\n", optarg);
return -EINVAL;
}
break;
case 'a':
env.apply = true;
break;
case 'v':
env.verbose = true;
break;
case 'h':
usage(argv[0]);
exit(0);
default:
return -EINVAL;
}
}
if (!env.destination || !env.port || optind != argc)
return -EINVAL;
return 0;
}
static int run_iterator(struct bpf_program *program)
{
struct bpf_link *link;
char buffer[256];
int iter_fd, length, err;
link = bpf_program__attach_iter(program, NULL);
err = libbpf_get_error(link);
if (err) {
fprintf(stderr, "failed to attach TCP iterator: %s\n", strerror(-err));
return err;
}
iter_fd = bpf_iter_create(bpf_link__fd(link));
if (iter_fd < 0) {
err = iter_fd;
fprintf(stderr, "failed to create TCP iterator: %s\n", strerror(-err));
goto cleanup;
}
while ((length = read(iter_fd, buffer, sizeof(buffer))) > 0)
;
if (length < 0) {
err = -errno;
fprintf(stderr, "failed while scanning TCP sockets: %s\n", strerror(errno));
}
close(iter_fd);
cleanup:
bpf_link__destroy(link);
return err;
}
int main(int argc, char **argv)
{
struct tcp_quarantine_bpf *skel = NULL;
struct in_addr destination;
int err;
err = parse_args(argc, argv);
if (err) {
usage(argv[0]);
return 1;
}
if (inet_pton(AF_INET, env.destination, &destination) != 1) {
fprintf(stderr, "invalid IPv4 destination: %s\n", env.destination);
return 1;
}
libbpf_set_print(libbpf_print_fn);
skel = tcp_quarantine_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return 1;
}
skel->rodata->target_addr = destination.s_addr;
skel->rodata->target_port = env.port;
skel->rodata->apply = env.apply;
err = tcp_quarantine_bpf__load(skel);
if (err) {
fprintf(stderr,
"failed to load TCP quarantine program: %s\n"
"This tool requires Linux 6.5+ with BTF, TCP BPF iterators, "
"BPF JIT, and the bpf_sock_destroy kfunc.\n",
strerror(-err));
goto cleanup;
}
err = run_iterator(skel->progs.quarantine_tcp);
if (err)
goto cleanup;
printf("mode=%s destination=%s:%u scanned=%llu established=%llu "
"matched=%llu destroyed=%llu failed=%llu\n",
env.apply ? "apply" : "dry-run", env.destination, env.port,
skel->bss->stats.scanned, skel->bss->stats.established,
skel->bss->stats.matched, skel->bss->stats.destroyed,
skel->bss->stats.failed);
if (skel->bss->stats.failed)
err = -EIO;
cleanup:
tcp_quarantine_bpf__destroy(skel);
return err != 0;
}

View File

@@ -0,0 +1,13 @@
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __TCP_QUARANTINE_H
#define __TCP_QUARANTINE_H
struct quarantine_stats {
unsigned long long scanned;
unsigned long long established;
unsigned long long matched;
unsigned long long destroyed;
unsigned long long failed;
};
#endif /* __TCP_QUARANTINE_H */

View File

@@ -0,0 +1,109 @@
#!/usr/bin/env python3
"""End-to-end smoke test for the TCP quarantine tool."""
from __future__ import annotations
import errno
import pathlib
import socket
import subprocess
import sys
import time
BROKEN_ERRORS = {errno.ECONNABORTED, errno.ECONNRESET, errno.EPIPE}
def open_connection() -> tuple[socket.socket, socket.socket, socket.socket, int]:
listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listener.bind(("127.0.0.1", 0))
listener.listen(1)
client = socket.create_connection(listener.getsockname(), timeout=2)
server, _ = listener.accept()
client.settimeout(1)
server.settimeout(1)
return listener, client, server, listener.getsockname()[1]
def round_trip(client: socket.socket, server: socket.socket, payload: bytes) -> None:
client.sendall(payload)
assert server.recv(len(payload)) == payload
server.sendall(payload)
assert client.recv(len(payload)) == payload
def run_tool(binary: pathlib.Path, port: int, apply: bool) -> str:
command = [str(binary), "--destination", "127.0.0.1", "--port", str(port)]
if apply:
command.append("--apply")
result = subprocess.run(command, check=True, text=True, capture_output=True)
print(result.stdout, end="")
return result.stdout
def wait_for_destroy(client: socket.socket) -> None:
deadline = time.monotonic() + 2
while time.monotonic() < deadline:
try:
client.send(b"x")
except OSError as error:
if error.errno in BROKEN_ERRORS:
return
raise
error = client.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR)
if error in BROKEN_ERRORS:
return
time.sleep(0.05)
raise AssertionError("quarantined TCP socket remained usable")
def main() -> int:
if len(sys.argv) != 2:
raise SystemExit(f"usage: {sys.argv[0]} /path/to/tcp_quarantine")
if not hasattr(socket, "SO_ERROR"):
raise SystemExit("SO_ERROR is unavailable")
binary = pathlib.Path(sys.argv[1]).resolve()
if not binary.is_file():
raise SystemExit(f"missing binary: {binary}")
invalid = subprocess.run(
[str(binary), "--destination", "not-an-ip", "--port", "1"],
text=True,
capture_output=True,
)
assert invalid.returncode != 0
assert "invalid IPv4 destination" in invalid.stderr
target = open_connection()
control = open_connection()
all_sockets = target[:3] + control[:3]
try:
round_trip(target[1], target[2], b"before-target")
round_trip(control[1], control[2], b"before-control")
dry_run = run_tool(binary, target[3], apply=False)
assert "mode=dry-run" in dry_run
assert "matched=1" in dry_run
assert "destroyed=0" in dry_run
round_trip(target[1], target[2], b"after-dry-run")
round_trip(control[1], control[2], b"control-after-dry-run")
applied = run_tool(binary, target[3], apply=True)
assert "mode=apply" in applied
assert "matched=1" in applied
assert "destroyed=1" in applied
assert "failed=0" in applied
wait_for_destroy(target[1])
round_trip(control[1], control[2], b"control-survives")
finally:
for sock in all_sockets:
sock.close()
print("PASS: dry-run preserved both connections; apply destroyed only the target")
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,10 @@
level=Depth
type=Tracing
kernel_min=7.0
kernel_min_basis=documented
architectures=x86_64
btf=required
kernel_config=CONFIG_BPF=y,CONFIG_BPF_SYSCALL=y,CONFIG_BPF_JIT=y,CONFIG_BPF_EVENTS=y,CONFIG_DEBUG_INFO_BTF=y,CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y
hardware=none
root=required
test_status=ci-build

15
src/52-fsession-latency/.gitignore vendored Normal file
View File

@@ -0,0 +1,15 @@
# Build artifacts
.output/
*.o
*.skel.h
# Generated binary
fsession_latency
# Python cache
__pycache__/
# Editor files
*.swp
*~
.vscode/

View File

@@ -0,0 +1,92 @@
# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
OUTPUT := .output
CLANG ?= clang
LIBBPF_SRC := $(abspath ../third_party/libbpf/src)
BPFTOOL_SRC := $(abspath ../third_party/bpftool/src)
LIBBPF_OBJ := $(abspath $(OUTPUT)/libbpf.a)
BPFTOOL_OUTPUT ?= $(abspath $(OUTPUT)/bpftool)
BPFTOOL ?= $(BPFTOOL_OUTPUT)/bootstrap/bpftool
ARCH ?= $(shell uname -m | sed 's/x86_64/x86/' \
| sed 's/arm.*/arm/' \
| sed 's/aarch64/arm64/' \
| sed 's/ppc64le/powerpc/' \
| sed 's/mips.*/mips/' \
| sed 's/riscv64/riscv/' \
| sed 's/loongarch64/loongarch/')
VMLINUX := ../third_party/vmlinux/$(ARCH)/vmlinux.h
INCLUDES := -I$(OUTPUT) -I../third_party/libbpf/include/uapi -I$(dir $(VMLINUX)) -I.
CFLAGS := -g -Wall -Wextra
ALL_LDFLAGS := $(LDFLAGS) $(EXTRA_LDFLAGS)
APPS = fsession_latency
CLANG_BPF_SYS_INCLUDES ?= $(shell $(CLANG) -v -E - </dev/null 2>&1 \
| sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }')
ifeq ($(V),1)
Q =
msg =
else
Q = @
msg = @printf ' %-8s %s%s\n' "$(1)" \
"$(patsubst $(abspath $(OUTPUT))/%,%,$(2))" \
"$(if $(3), $(3))";
MAKEFLAGS += --no-print-directory
endif
define allow-override
$(if $(or $(findstring environment,$(origin $(1))),\
$(findstring command line,$(origin $(1)))),,\
$(eval $(1) = $(2)))
endef
$(call allow-override,CC,$(CROSS_COMPILE)cc)
$(call allow-override,LD,$(CROSS_COMPILE)ld)
.PHONY: all clean test
all: $(APPS)
clean:
$(call msg,CLEAN)
$(Q)rm -rf $(OUTPUT) $(APPS)
test: all
python3 tests/test_fsession_latency.py ./fsession_latency
$(OUTPUT) $(OUTPUT)/libbpf $(BPFTOOL_OUTPUT):
$(call msg,MKDIR,$@)
$(Q)mkdir -p $@
$(LIBBPF_OBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)/libbpf
$(call msg,LIB,$@)
$(Q)$(MAKE) -C $(LIBBPF_SRC) BUILD_STATIC_ONLY=1 \
OBJDIR=$(dir $@)/libbpf DESTDIR=$(dir $@) \
INCLUDEDIR= LIBDIR= UAPIDIR= install
$(BPFTOOL): | $(BPFTOOL_OUTPUT)
$(call msg,BPFTOOL,$@)
$(Q)$(MAKE) ARCH= CROSS_COMPILE= OUTPUT=$(BPFTOOL_OUTPUT)/ -C $(BPFTOOL_SRC) bootstrap
$(OUTPUT)/%.bpf.o: %.bpf.c $(LIBBPF_OBJ) $(wildcard %.h) $(VMLINUX) | $(OUTPUT) $(BPFTOOL)
$(call msg,BPF,$@)
$(Q)$(CLANG) -g -O2 -target bpf -D__TARGET_ARCH_$(ARCH) \
$(INCLUDES) $(CLANG_BPF_SYS_INCLUDES) \
-c $(filter %.c,$^) -o $(patsubst %.bpf.o,%.tmp.bpf.o,$@)
$(Q)$(BPFTOOL) gen object $@ $(patsubst %.bpf.o,%.tmp.bpf.o,$@)
$(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(OUTPUT) $(BPFTOOL)
$(call msg,GEN-SKEL,$@)
$(Q)$(BPFTOOL) gen skeleton $< > $@
$(patsubst %,$(OUTPUT)/%.o,$(APPS)): %.o: %.skel.h
$(OUTPUT)/%.o: %.c $(wildcard %.h) | $(OUTPUT)
$(call msg,CC,$@)
$(Q)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
$(APPS): %: $(OUTPUT)/%.o $(LIBBPF_OBJ) | $(OUTPUT)
$(call msg,BINARY,$@)
$(Q)$(CC) $(CFLAGS) $^ $(ALL_LDFLAGS) -lelf -lz -o $@
.DELETE_ON_ERROR:
.SECONDARY:

View File

@@ -0,0 +1,515 @@
# eBPF Tutorial: Tracing Slow vfs_read Calls with fsession
Suppose a file-backed service shows read-latency spikes. Application-level timing tells you that requests slowed down, but it cannot distinguish whether the kernel read path blocked or user-space logic took too long. The useful questions are: which thread issued the read, how many bytes did it request, what did the call return, and how long did that single `vfs_read` invocation take?
This tutorial demonstrates how to measure `vfs_read` call latency using the fsession mechanism introduced in Linux 7.0. The tool timestamps function entry, computes latency at return, filters by TGID and threshold, and reports slow-read events.
> Complete source: <https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/52-fsession-latency>
## Why fsession Is Needed
Traditional approaches to measuring function latency all have notable limitations.
**Separate fentry and fexit programs** are the most common approach. You write two BPF programs: one attached to function entry, one to function return. The two programs pass timestamps through a per-thread hash map: entry stores a mapping from `bpf_get_current_pid_tgid()` to timestamp, and return looks up and deletes the entry. This pattern has several problems. First, you must maintain an extra hash map, incurring lookup and deletion overhead on every call. Second, if a thread dies between entry and return (for example, if it is killed), the hash-map entry leaks. Third, the two programs have no intrinsic relationship; state passing depends entirely on an external data structure.
**kprobe and kretprobe** also require two separate programs and an external hash map. Compared to fentry/fexit, kprobes have higher overhead because they work through a breakpoint mechanism, whereas fentry hooks directly into the function prologue using JIT-optimized call sequences on x86_64.
**User-space sampling** such as `perf record` or periodic stack sampling with `bpftrace` provides only a statistical profile and cannot precisely measure the latency of each call. For scenarios that need to capture tail-latency events, sampling may completely miss critical slow calls.
**The fsession mechanism** solves this problem fundamentally. Linux 7.0 introduced fsession, which executes the same BPF program once at function entry and once at return, providing an 8-byte per-invocation scratch area through `bpf_session_cookie(ctx)`. This cookie persists from entry to return, completely replacing external hash maps. The program uses `bpf_session_is_return(ctx)` to distinguish the two phases and handles all logic within a single program.
## How fsession Works
fsession is a special ftracing program type. When you declare a program with `SEC("fsession/vfs_read")`, the kernel calls your BPF program once at `vfs_read` entry and once at return.
`bpf_session_is_return(ctx)` returns false during the entry phase and true during the return phase. The program uses this helper to determine which phase it is in and execute the appropriate logic.
`bpf_session_cookie(ctx)` returns a pointer to an 8-byte scratch area. This scratch area is unique to each invocation and persists from entry to return. You can write data (such as a timestamp) during entry and read it during return.
The return-phase context includes not only the original function arguments but also the function's return value. For `vfs_read`, you can see the `file`, `buf`, `count`, and `pos` arguments at entry; at return you can also see the `ret` return value. This means you do not need to store arguments in a map - the return phase can access them directly.
The `fsession_latency` tool leverages these capabilities for latency measurement. At entry, it checks whether the TGID matches the target process; matching calls store a `bpf_ktime_get_ns()` timestamp in the cookie. At return, it reads the timestamp from the cookie to compute latency, increments aggregate counters, and reports detailed events through a ring buffer for calls exceeding the threshold.
## Code Implementation
This tool consists of three files: a shared header defining event and statistics structures, a BPF program that measures latency and submits events, and a user-space loader that manages the lifecycle and prints results.
### Shared Header
`fsession_latency.h` defines the event and statistics structures shared between BPF and user space. `latency_event` contains a 32-bit PID and TGID, 64-bit requested bytes, a signed 64-bit return value, nanosecond-precision latency, and the process name. `latency_stats` contains four counters for aggregate statistics.
```c
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __FSESSION_LATENCY_H
#define __FSESSION_LATENCY_H
#define FSESSION_COMM_LEN 16
struct latency_event {
unsigned int pid;
unsigned int tgid;
unsigned long long requested;
long long result;
unsigned long long latency_ns;
char comm[FSESSION_COMM_LEN];
};
struct latency_stats {
unsigned long long calls;
unsigned long long slow;
unsigned long long errors;
unsigned long long dropped;
};
#endif /* __FSESSION_LATENCY_H */
```
The four counters record: `calls` for total invocations, `slow` for calls reaching the threshold, `errors` for calls returning negative values, and `dropped` for ring-buffer reservation failures. `FSESSION_COMM_LEN` is set to 16 to match the kernel's `TASK_COMM_LEN`.
### BPF Program
`fsession_latency.bpf.c` uses `SEC("fsession/vfs_read")` to declare itself as an fsession type. The kernel calls this program once at `vfs_read` entry and once at return.
```c
// SPDX-License-Identifier: GPL-2.0
#define bpf_session_is_return bpf_session_is_return_vmlinux_snapshot
#define bpf_session_cookie bpf_session_cookie_vmlinux_snapshot
#include "vmlinux.h"
#undef bpf_session_is_return
#undef bpf_session_cookie
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include "fsession_latency.h"
char LICENSE[] SEC("license") = "GPL";
const volatile __u64 threshold_ns;
const volatile __u32 target_tgid;
struct latency_stats stats;
struct {
__uint(type, BPF_MAP_TYPE_RINGBUF);
__uint(max_entries, 256 * 1024);
} events SEC(".maps");
/*
* The repository vmlinux.h snapshot predates the ctx argument on these
* kfunc prototypes. Rename those stale declarations while including the
* snapshot, then provide the Linux 7.0 signatures below.
*/
extern bool bpf_session_is_return(void *ctx) __ksym;
extern __u64 *bpf_session_cookie(void *ctx) __ksym;
SEC("fsession/vfs_read")
int BPF_PROG(measure_vfs_read, struct file *file, char *buf, size_t count,
loff_t *pos, ssize_t ret)
{
__u64 pid_tgid = bpf_get_current_pid_tgid();
__u64 *started = bpf_session_cookie(ctx);
struct latency_event *event;
__u64 latency;
if (!bpf_session_is_return(ctx)) {
if (target_tgid && pid_tgid >> 32 != target_tgid) {
*started = 0;
return 0;
}
*started = bpf_ktime_get_ns();
return 0;
}
if (!*started)
return 0;
latency = bpf_ktime_get_ns() - *started;
__sync_fetch_and_add(&stats.calls, 1);
if (ret < 0)
__sync_fetch_and_add(&stats.errors, 1);
if (latency < threshold_ns)
return 0;
__sync_fetch_and_add(&stats.slow, 1);
event = bpf_ringbuf_reserve(&events, sizeof(*event), 0);
if (!event) {
__sync_fetch_and_add(&stats.dropped, 1);
return 0;
}
event->pid = (__u32)pid_tgid;
event->tgid = pid_tgid >> 32;
event->requested = count;
event->result = ret;
event->latency_ns = latency;
bpf_get_current_comm(event->comm, sizeof(event->comm));
bpf_ringbuf_submit(event, 0);
return 0;
}
```
The program is structured into entry and return phases. The two `const volatile` variables reside in the `.rodata` section; user space writes the threshold and target TGID after `open()` but before `load()`, and the verifier treats them as compile-time constants. `bpf_session_is_return` and `bpf_session_cookie` are declared as kfunc symbols via `extern ... __ksym`, resolved by the kernel at load time.
At entry, the program calls `bpf_get_current_pid_tgid()` to obtain the caller's PID and TGID. If `target_tgid` is set and the call comes from a different process, the program writes 0 to the cookie to indicate this invocation should be skipped. Matching calls store the monotonic timestamp returned by `bpf_ktime_get_ns()`.
At return, the program first checks whether the cookie is 0. A zero cookie indicates the entry phase already filtered this call, so it returns immediately. Otherwise it computes latency as the current timestamp minus the saved entry timestamp, then increments the `calls` counter. If `ret < 0`, indicating the read operation returned an error, it increments the `errors` counter. Calls with latency below `threshold_ns` stop here, updating only aggregate counters without reporting events.
Calls reaching the threshold increment the `slow` counter, then attempt to reserve a fixed-size record in the ring buffer. A failed reservation increments the `dropped` counter, allowing the final statistics to expose pressure in the event path. On successful reservation, the record is filled with PID, TGID, requested bytes, return value, latency, and process name, then submitted to the ring buffer.
The macros at the beginning of the file provide a local compatibility bridge. The repository's `vmlinux.h` snapshot has `bpf_session_is_return` and `bpf_session_cookie` declarations that predate the `ctx` argument added in Linux 7.0. The macros rename the old declarations, include the snapshot, undef, then redeclare with the correct Linux 7.0 signatures. A freshly regenerated `vmlinux.h` from kernel 7.0 or later can be used directly without this bridge.
### User-Space Loader
`fsession_latency.c` parses the command line, configures BPF constants, manages the observation window, and prints results.
```c
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <errno.h>
#include <getopt.h>
#include <signal.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <bpf/libbpf.h>
#include "fsession_latency.h"
#include "fsession_latency.skel.h"
static volatile sig_atomic_t exiting;
static struct env {
unsigned long long threshold_us;
unsigned int duration;
unsigned int pid;
bool verbose;
} env = {
.threshold_us = 1000,
.duration = 10,
};
static unsigned long long events_printed;
static void handle_signal(int signal)
{
(void)signal;
exiting = 1;
}
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s [--threshold-us USEC] [--duration SEC] [--pid TGID] "
"[--verbose]\n\n"
"Report vfs_read calls at or above a latency threshold.\n\n"
"Options:\n"
" -t, --threshold-us USEC slow-read threshold (default: 1000)\n"
" -d, --duration SEC trace duration, 1-86400 (default: 10)\n"
" -p, --pid TGID trace one process ID (default: all)\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_u64(const char *value, unsigned long long maximum,
unsigned long long *result)
{
char *end = NULL;
unsigned long long parsed;
errno = 0;
parsed = strtoull(value, &end, 10);
if (errno || end == value || *end || parsed > maximum)
return -EINVAL;
*result = parsed;
return 0;
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "threshold-us", required_argument, NULL, 't' },
{ "duration", required_argument, NULL, 'd' },
{ "pid", required_argument, NULL, 'p' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
unsigned long long parsed;
int option;
while ((option = getopt_long(argc, argv, "t:d:p:vh", options, NULL)) != -1) {
switch (option) {
case 't':
if (parse_u64(optarg, UINT64_MAX / 1000, &env.threshold_us)) {
fprintf(stderr, "invalid threshold in microseconds: %s\n", optarg);
return -EINVAL;
}
break;
case 'd':
if (parse_u64(optarg, 86400, &parsed) || parsed == 0) {
fprintf(stderr, "invalid duration in seconds: %s\n", optarg);
return -EINVAL;
}
env.duration = parsed;
break;
case 'p':
if (parse_u64(optarg, UINT32_MAX, &parsed) || parsed == 0) {
fprintf(stderr, "invalid process ID: %s\n", optarg);
return -EINVAL;
}
env.pid = parsed;
break;
case 'v':
env.verbose = true;
break;
case 'h':
usage(argv[0]);
exit(0);
default:
return -EINVAL;
}
}
if (optind != argc)
return -EINVAL;
return 0;
}
static long long monotonic_milliseconds(void)
{
struct timespec timestamp;
if (clock_gettime(CLOCK_MONOTONIC, &timestamp))
return -errno;
return timestamp.tv_sec * 1000LL + timestamp.tv_nsec / 1000000;
}
static int handle_event(void *context, void *data, size_t size)
{
const struct latency_event *event = data;
(void)context;
if (size < sizeof(*event))
return 0;
printf("EVENT comm=%-16s tgid=%u pid=%u requested=%llu result=%lld "
"latency_us=%llu\n",
event->comm, event->tgid, event->pid, event->requested,
event->result, event->latency_ns / 1000);
events_printed++;
return 0;
}
static int poll_until_deadline(struct ring_buffer *ring, long long deadline)
{
while (!exiting) {
long long now = monotonic_milliseconds();
int consumed;
if (now < 0)
return (int)now;
if (now >= deadline)
break;
consumed = ring_buffer__poll(ring,
deadline - now > 100 ? 100 : deadline - now);
if (consumed == -EINTR)
continue;
if (consumed < 0) {
fprintf(stderr, "ring buffer poll failed: %s\n",
strerror(-consumed));
return consumed;
}
}
return 0;
}
int main(int argc, char **argv)
{
struct fsession_latency_bpf *skel = NULL;
struct ring_buffer *ring = NULL;
long long deadline, now;
int err;
err = parse_args(argc, argv);
if (err) {
usage(argv[0]);
return 1;
}
libbpf_set_print(libbpf_print_fn);
signal(SIGINT, handle_signal);
signal(SIGTERM, handle_signal);
skel = fsession_latency_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return 1;
}
skel->rodata->threshold_ns = env.threshold_us * 1000;
skel->rodata->target_tgid = env.pid;
err = fsession_latency_bpf__load(skel);
if (err) {
fprintf(stderr,
"failed to load fsession program: %s\n"
"This tool requires Linux 7.0+, BTF, BPF JIT, and x86_64 "
"fsession support.\n",
strerror(-err));
goto cleanup;
}
err = fsession_latency_bpf__attach(skel);
if (err) {
fprintf(stderr, "failed to attach to vfs_read: %s\n", strerror(-err));
goto cleanup;
}
ring = ring_buffer__new(bpf_map__fd(skel->maps.events), handle_event, NULL, NULL);
err = libbpf_get_error(ring);
if (err) {
ring = NULL;
fprintf(stderr, "failed to create ring buffer: %s\n", strerror(-err));
goto cleanup;
}
printf("Tracing vfs_read for %u seconds; threshold=%llu us; pid=%s\n",
env.duration, env.threshold_us, env.pid ? "selected" : "all");
fflush(stdout);
now = monotonic_milliseconds();
if (now < 0) {
err = now;
goto cleanup;
}
deadline = now + env.duration * 1000LL;
err = poll_until_deadline(ring, deadline);
if (err)
goto cleanup;
fsession_latency_bpf__detach(skel);
err = ring_buffer__consume(ring);
if (err < 0) {
fprintf(stderr, "ring buffer drain failed: %s\n", strerror(-err));
goto cleanup;
}
err = 0;
printf("SUMMARY calls=%llu slow=%llu errors=%llu dropped=%llu events=%llu\n",
skel->bss->stats.calls, skel->bss->stats.slow,
skel->bss->stats.errors, skel->bss->stats.dropped, events_printed);
cleanup:
ring_buffer__free(ring);
fsession_latency_bpf__destroy(skel);
return err != 0;
}
```
The loader follows a standard flow. After parsing the command line, it opens the skeleton and writes `threshold_ns` and `target_tgid` to the read-only BPF configuration. The user-provided threshold is in microseconds; it is multiplied by 1000 to convert to nanoseconds before writing. Calling `load` completes BPF program loading, then attaches to `vfs_read`. The failure diagnostic provides both the kernel errno and a prerequisite list to help users troubleshoot.
`ring_buffer__new` creates the ring-buffer consumer and registers the `handle_event` callback to process each event. The callback prints comm, TGID, PID, requested bytes, result, and latency in microseconds, incrementing `events_printed`.
The `poll_until_deadline` function drives the event consumption loop. It calculates a monotonic deadline and polls for up to 100 ms per iteration until the deadline or SIGINT/SIGTERM. A `-EINTR` return continues the loop; other negative values indicate errors.
After polling ends, the loader first calls `fsession_latency_bpf__detach` to detach the fsession program, stopping changes to aggregate counters. It then calls `ring_buffer__consume` to deliver records submitted before detach but not yet consumed. This ordering is part of observation-window boundary correctness: stop new event production first, drain existing events, then read stable counters. On exit, it frees the ring buffer and destroys the skeleton.
## Compilation and Execution
Build from source (the repository vendors libbpf 1.7.0 and bpftool v7.7.0):
```bash
cd src/52-fsession-latency
make clean
make -j2
```
Trace a specific service process for 30 seconds, reporting reads of 10 ms or longer:
```bash
SERVICE_PID=$(pgrep -n my-service)
sudo ./fsession_latency --pid "$SERVICE_PID" --threshold-us 10000 --duration 30
```
`--pid` compares against the TGID returned by `bpf_get_current_pid_tgid()`, which is the TGID in the host (initial) PID namespace. The `pgrep` example above works for processes running directly on the host; targets inside containers or nested PID namespaces need their host-visible TGID resolved and passed in.
Command-line arguments:
```text
Usage: ./fsession_latency [--threshold-us USEC] [--duration SEC] [--pid TGID] [--verbose]
Options:
-t, --threshold-us USEC slow-read threshold (default: 1000 microseconds)
-d, --duration SEC trace duration, 1-86400 (default: 10 seconds)
-p, --pid TGID trace a specific process ID (default: all processes)
-v, --verbose print libbpf diagnostics
-h, --help show this help
```
Defaults are 1000 microsecond threshold, 10 second duration, and all processes. `--duration` accepts 1 through 86400 seconds, `--pid` accepts 1 through 2^32-1, and `--threshold-us` accepts 0 through `UINT64_MAX / 1000` microseconds (user space converts to nanoseconds before load). Empty values and trailing characters are rejected by `parse_u64`.
The following output was captured on x86_64 with kernel `7.0.0-rc2+`. The integration test performs 64 ordinary file reads plus one pipe read delayed by 50 ms; the pipe read exceeds the 10 ms threshold and appears in the event stream:
```console
Tracing vfs_read for 1 seconds; threshold=10000 us; pid=selected
EVENT comm=python3 tgid=1257 pid=1257 requested=1 result=1 latency_us=50160
SUMMARY calls=66 slow=1 errors=0 dropped=0 events=1
TEST-SUMMARY miss_calls=0 high_threshold_calls=66 high_threshold_slow=0 threshold_calls=66 threshold_slow=1 events=1 dropped=0
PASS: PID filtering, threshold miss, and slow-read reporting behaved as expected
```
Exact call counts, PID values, and latency numbers vary by run.
Running the tests:
```bash
sudo make test
```
The integration test validates four runtime paths: target TGID of `UINT32_MAX` produces zero calls and events; a 10,000,000 microsecond threshold causes ordinary reads to produce zero slow events; 64 ordinary file reads plus one 50 ms delayed pipe read under a 10,000 microsecond threshold produces a slow event; and threshold 0 verifies `slow == events + dropped`, proving every threshold-matching call is either delivered or accounted as a ring-buffer reservation failure. CLI tests cover invalid duration, empty duration, empty threshold, and empty PID.
### Environment Requirements
| Requirement | Details |
|---|---|
| Kernel version | Linux 7.0+ (fsession first introduced) |
| BTF | Must be enabled (`CONFIG_DEBUG_INFO_BTF=y`) |
| Kernel config | `CONFIG_BPF=y`, `CONFIG_BPF_SYSCALL=y`, `CONFIG_BPF_JIT=y`, `CONFIG_BPF_EVENTS=y`, `CONFIG_DEBUG_INFO_BTF=y`, `CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y` |
| BPF JIT | Must be enabled at runtime |
| Architecture | Tested on x86_64 |
| Privilege | root |
Runtime behavior was tested on x86_64 with kernel `7.0.0-rc2+`. The upstream merge commit is `f17b474e36647c23801ef8fdaf2255ab66dd2973`.
## Observation Scope
This tool measures time inside `vfs_read`, covering read paths that reach that function and filtering by exact host-visible TGID. A more comprehensive tracer could add cgroup, path, inode, mount, stack, or long-running metrics dimensions while keeping the same fsession correlation pattern; the existing `dropped` counter remains the event-loss signal under load.
## Summary
This tutorial demonstrated how to measure kernel function latency using the fsession mechanism in Linux 7.0. Compared to the traditional approach of two fentry/fexit programs with a hash map, fsession handles both entry and return in a single program. The 8-byte per-invocation session cookie replaces the external correlation map for carrying timestamps, resulting in cleaner code, no state-leak risk, and lower performance overhead.
The tool's design demonstrates the core fsession pattern: use `bpf_session_is_return` to distinguish phases, use `bpf_session_cookie` to pass per-invocation state, and access function arguments and return values directly in the return phase. This pattern applies to any scenario requiring correlation between function entry and return.
> To learn more about eBPF, check out our tutorial repository at <https://github.com/eunomia-bpf/bpf-developer-tutorial> or visit <https://eunomia.dev/tutorials/>.
## References
- [Linux fsession merge commit](https://github.com/torvalds/linux/commit/f17b474e36647c23801ef8fdaf2255ab66dd2973)
- [Upstream fsession program selftest](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/progs/fsession_test.c)
- [Upstream fsession runner selftest](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/prog_tests/fsession_test.c)
- [libbpf 1.7.0](https://github.com/libbpf/libbpf/releases/tag/v1.7.0)
- [bpftool v7.7.0](https://github.com/libbpf/bpftool/releases/tag/v7.7.0)

View File

@@ -0,0 +1,515 @@
# eBPF 教程:使用 fsession 追踪慢速 vfs_read 调用
假设一个文件服务出现了读延迟尖峰,应用层计时只能告诉你请求变慢了,却无法区分是内核读路径阻塞还是用户态逻辑耗时。有用的问题是:哪个线程发起了读操作、请求了多少字节、调用返回了什么、这一次 `vfs_read` 花了多长时间。
本教程展示如何使用 Linux 7.0 引入的 fsession 机制测量 `vfs_read` 调用延迟。工具在函数进入时记录时间戳、返回时计算延迟,按 TGID 和阈值过滤后上报慢读事件。
> 完整源码:<https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/52-fsession-latency>
## 为什么需要 fsession
传统的函数延迟测量方案都有明显的局限性。
**独立的 fentry 和 fexit 程序**是最常见的做法。你需要编写两个 BPF 程序,一个附加到函数入口,一个附加到函数返回。两个程序之间需要通过一个按线程索引的哈希表来传递时间戳:进入时存入 `bpf_get_current_pid_tgid()` 到时间戳的映射,返回时查找并删除。这个模式有几个问题:首先需要维护额外的哈希表,每次调用都有查找和删除的开销;其次如果进入后没有正常返回(比如线程被杀死),哈希表中的条目会泄漏;最后两个程序之间没有天然的关联,状态传递完全依赖外部数据结构。
**kprobe 和 kretprobe**同样需要两个独立的程序和外部哈希表。相比 fentry/fexit,kprobe 的开销更大,因为它通过断点机制工作,而 fentry 直接挂钩到函数序言,在 x86_64 上使用 JIT 优化的调用序列。
**用户态采样**如 `perf record` 或 `bpftrace` 的周期性堆栈采样只能提供统计概貌,无法精确测量每次调用的延迟。对于需要捕获尾部延迟事件的场景,采样可能完全错过关键的慢调用。
**fsession 机制**从根本上解决了这个问题。Linux 7.0 引入了 fsession,它让同一个 BPF 程序在被追踪函数的进入和返回阶段各执行一次,并通过 `bpf_session_cookie(ctx)` 提供一个 8 字节的调用级暂存区。这个 cookie 从进入到返回期间持续有效,完全替代了外部哈希表。程序用 `bpf_session_is_return(ctx)` 区分两个阶段,在同一个程序内处理所有逻辑。
## fsession 的工作原理
fsession 是一种特殊的 ftracing 程序类型。当你用 `SEC("fsession/vfs_read")` 声明一个程序时,内核会在 `vfs_read` 的进入和返回阶段各调用一次你的 BPF 程序。
`bpf_session_is_return(ctx)` 返回 false 表示进入阶段,返回 true 表示返回阶段。程序可以用这个 helper 函数区分当前处于哪个阶段,执行相应的逻辑。
`bpf_session_cookie(ctx)` 返回一个指向 8 字节暂存区的指针。这个暂存区是每次调用独立的,从进入阶段到返回阶段持续有效。你可以在进入阶段写入数据(比如时间戳),在返回阶段读取并使用它。
返回阶段的上下文不仅包含原始函数参数,还包含函数的返回值。对于 `vfs_read`,进入时能看到 `file`、`buf`、`count`、`pos` 四个参数,返回时还能看到 `ret` 返回值。这意味着你不需要把参数存到 map 里,返回阶段可以直接访问。
`fsession_latency` 工具利用这些能力实现延迟测量。进入阶段,它检查 TGID 是否匹配目标进程,匹配的调用在 cookie 中存入 `bpf_ktime_get_ns()` 时间戳。返回阶段,它从 cookie 读取时间戳计算延迟,递增聚合计数器,超过阈值的调用通过 ring buffer 上报详细事件。
## 代码实现
本工具由三个文件组成:共享头文件定义事件和统计结构、BPF 程序测量延迟并提交事件、用户空间加载器管理生命周期并打印结果。
### 共享头文件
`fsession_latency.h` 定义了 BPF 和用户空间共享的事件与统计结构。`latency_event` 包含 32 位 PID 和 TGID、64 位请求字节数、有符号 64 位返回值、纳秒精度延迟和进程名。`latency_stats` 包含四个计数器用于聚合统计。
```c
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __FSESSION_LATENCY_H
#define __FSESSION_LATENCY_H
#define FSESSION_COMM_LEN 16
struct latency_event {
unsigned int pid;
unsigned int tgid;
unsigned long long requested;
long long result;
unsigned long long latency_ns;
char comm[FSESSION_COMM_LEN];
};
struct latency_stats {
unsigned long long calls;
unsigned long long slow;
unsigned long long errors;
unsigned long long dropped;
};
#endif /* __FSESSION_LATENCY_H */
```
四个计数器分别记录:`calls` 是总调用数、`slow` 是达到阈值的调用数、`errors` 是返回负值的调用数、`dropped` 是 ring buffer 预留失败的计数。`FSESSION_COMM_LEN` 设为 16,与内核 `TASK_COMM_LEN` 一致。
### BPF 程序
`fsession_latency.bpf.c` 使用 `SEC("fsession/vfs_read")` 声明为 fsession 类型。内核会在 `vfs_read` 的进入和返回阶段各调用一次这个程序。
```c
// SPDX-License-Identifier: GPL-2.0
#define bpf_session_is_return bpf_session_is_return_vmlinux_snapshot
#define bpf_session_cookie bpf_session_cookie_vmlinux_snapshot
#include "vmlinux.h"
#undef bpf_session_is_return
#undef bpf_session_cookie
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include "fsession_latency.h"
char LICENSE[] SEC("license") = "GPL";
const volatile __u64 threshold_ns;
const volatile __u32 target_tgid;
struct latency_stats stats;
struct {
__uint(type, BPF_MAP_TYPE_RINGBUF);
__uint(max_entries, 256 * 1024);
} events SEC(".maps");
/*
* The repository vmlinux.h snapshot predates the ctx argument on these
* kfunc prototypes. Rename those stale declarations while including the
* snapshot, then provide the Linux 7.0 signatures below.
*/
extern bool bpf_session_is_return(void *ctx) __ksym;
extern __u64 *bpf_session_cookie(void *ctx) __ksym;
SEC("fsession/vfs_read")
int BPF_PROG(measure_vfs_read, struct file *file, char *buf, size_t count,
loff_t *pos, ssize_t ret)
{
__u64 pid_tgid = bpf_get_current_pid_tgid();
__u64 *started = bpf_session_cookie(ctx);
struct latency_event *event;
__u64 latency;
if (!bpf_session_is_return(ctx)) {
if (target_tgid && pid_tgid >> 32 != target_tgid) {
*started = 0;
return 0;
}
*started = bpf_ktime_get_ns();
return 0;
}
if (!*started)
return 0;
latency = bpf_ktime_get_ns() - *started;
__sync_fetch_and_add(&stats.calls, 1);
if (ret < 0)
__sync_fetch_and_add(&stats.errors, 1);
if (latency < threshold_ns)
return 0;
__sync_fetch_and_add(&stats.slow, 1);
event = bpf_ringbuf_reserve(&events, sizeof(*event), 0);
if (!event) {
__sync_fetch_and_add(&stats.dropped, 1);
return 0;
}
event->pid = (__u32)pid_tgid;
event->tgid = pid_tgid >> 32;
event->requested = count;
event->result = ret;
event->latency_ns = latency;
bpf_get_current_comm(event->comm, sizeof(event->comm));
bpf_ringbuf_submit(event, 0);
return 0;
}
```
程序结构分为进入阶段和返回阶段两部分。两个 `const volatile` 变量位于 `.rodata` 段,用户态在 `open()` 之后、`load()` 之前写入阈值和目标 TGID,验证器将它们视为编译期常量。`bpf_session_is_return` 和 `bpf_session_cookie` 通过 `extern ... __ksym` 声明为 kfunc 符号,内核在加载时解析。
进入阶段,程序调用 `bpf_get_current_pid_tgid()` 获取调用者的 PID 和 TGID。如果设置了 `target_tgid` 且调用来自其他进程,程序在 cookie 中写入 0 表示这次调用应该被跳过。匹配的调用则存入 `bpf_ktime_get_ns()` 返回的单调时间戳。
返回阶段,程序首先检查 cookie 是否为 0。cookie 为 0 表示进入阶段已经过滤了这次调用,直接返回。否则计算延迟为当前时间戳减去保存的进入时间戳,然后递增 `calls` 计数器。如果 `ret < 0` 表示读操作返回错误,递增 `errors` 计数器。延迟低于 `threshold_ns` 的调用到此为止,只更新聚合计数器不上报事件。
达到阈值的调用递增 `slow` 计数器,然后尝试在 ring buffer 中预留固定大小的记录。预留失败时递增 `dropped` 计数器,这让最终统计能够暴露事件路径的压力。预留成功后填入 PID、TGID、请求字节数、返回值、延迟和进程名,最后提交到 ring buffer。
文件开头的宏处理是一个本地兼容桥接。仓库的 `vmlinux.h` 快照中 `bpf_session_is_return` 和 `bpf_session_cookie` 的声明早于 Linux 7.0 添加的 `ctx` 参数。宏先把旧声明重命名,include 之后再 undef,然后用正确的 Linux 7.0 签名重新声明。从 7.0 以上内核重新生成的 `vmlinux.h` 可以直接使用,不需要这个桥接。
### 用户空间加载器
`fsession_latency.c` 解析命令行、配置 BPF 常量、管理观察窗口并打印结果。
```c
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <errno.h>
#include <getopt.h>
#include <signal.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <bpf/libbpf.h>
#include "fsession_latency.h"
#include "fsession_latency.skel.h"
static volatile sig_atomic_t exiting;
static struct env {
unsigned long long threshold_us;
unsigned int duration;
unsigned int pid;
bool verbose;
} env = {
.threshold_us = 1000,
.duration = 10,
};
static unsigned long long events_printed;
static void handle_signal(int signal)
{
(void)signal;
exiting = 1;
}
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s [--threshold-us USEC] [--duration SEC] [--pid TGID] "
"[--verbose]\n\n"
"Report vfs_read calls at or above a latency threshold.\n\n"
"Options:\n"
" -t, --threshold-us USEC slow-read threshold (default: 1000)\n"
" -d, --duration SEC trace duration, 1-86400 (default: 10)\n"
" -p, --pid TGID trace one process ID (default: all)\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_u64(const char *value, unsigned long long maximum,
unsigned long long *result)
{
char *end = NULL;
unsigned long long parsed;
errno = 0;
parsed = strtoull(value, &end, 10);
if (errno || end == value || *end || parsed > maximum)
return -EINVAL;
*result = parsed;
return 0;
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "threshold-us", required_argument, NULL, 't' },
{ "duration", required_argument, NULL, 'd' },
{ "pid", required_argument, NULL, 'p' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
unsigned long long parsed;
int option;
while ((option = getopt_long(argc, argv, "t:d:p:vh", options, NULL)) != -1) {
switch (option) {
case 't':
if (parse_u64(optarg, UINT64_MAX / 1000, &env.threshold_us)) {
fprintf(stderr, "invalid threshold in microseconds: %s\n", optarg);
return -EINVAL;
}
break;
case 'd':
if (parse_u64(optarg, 86400, &parsed) || parsed == 0) {
fprintf(stderr, "invalid duration in seconds: %s\n", optarg);
return -EINVAL;
}
env.duration = parsed;
break;
case 'p':
if (parse_u64(optarg, UINT32_MAX, &parsed) || parsed == 0) {
fprintf(stderr, "invalid process ID: %s\n", optarg);
return -EINVAL;
}
env.pid = parsed;
break;
case 'v':
env.verbose = true;
break;
case 'h':
usage(argv[0]);
exit(0);
default:
return -EINVAL;
}
}
if (optind != argc)
return -EINVAL;
return 0;
}
static long long monotonic_milliseconds(void)
{
struct timespec timestamp;
if (clock_gettime(CLOCK_MONOTONIC, &timestamp))
return -errno;
return timestamp.tv_sec * 1000LL + timestamp.tv_nsec / 1000000;
}
static int handle_event(void *context, void *data, size_t size)
{
const struct latency_event *event = data;
(void)context;
if (size < sizeof(*event))
return 0;
printf("EVENT comm=%-16s tgid=%u pid=%u requested=%llu result=%lld "
"latency_us=%llu\n",
event->comm, event->tgid, event->pid, event->requested,
event->result, event->latency_ns / 1000);
events_printed++;
return 0;
}
static int poll_until_deadline(struct ring_buffer *ring, long long deadline)
{
while (!exiting) {
long long now = monotonic_milliseconds();
int consumed;
if (now < 0)
return (int)now;
if (now >= deadline)
break;
consumed = ring_buffer__poll(ring,
deadline - now > 100 ? 100 : deadline - now);
if (consumed == -EINTR)
continue;
if (consumed < 0) {
fprintf(stderr, "ring buffer poll failed: %s\n",
strerror(-consumed));
return consumed;
}
}
return 0;
}
int main(int argc, char **argv)
{
struct fsession_latency_bpf *skel = NULL;
struct ring_buffer *ring = NULL;
long long deadline, now;
int err;
err = parse_args(argc, argv);
if (err) {
usage(argv[0]);
return 1;
}
libbpf_set_print(libbpf_print_fn);
signal(SIGINT, handle_signal);
signal(SIGTERM, handle_signal);
skel = fsession_latency_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return 1;
}
skel->rodata->threshold_ns = env.threshold_us * 1000;
skel->rodata->target_tgid = env.pid;
err = fsession_latency_bpf__load(skel);
if (err) {
fprintf(stderr,
"failed to load fsession program: %s\n"
"This tool requires Linux 7.0+, BTF, BPF JIT, and x86_64 "
"fsession support.\n",
strerror(-err));
goto cleanup;
}
err = fsession_latency_bpf__attach(skel);
if (err) {
fprintf(stderr, "failed to attach to vfs_read: %s\n", strerror(-err));
goto cleanup;
}
ring = ring_buffer__new(bpf_map__fd(skel->maps.events), handle_event, NULL, NULL);
err = libbpf_get_error(ring);
if (err) {
ring = NULL;
fprintf(stderr, "failed to create ring buffer: %s\n", strerror(-err));
goto cleanup;
}
printf("Tracing vfs_read for %u seconds; threshold=%llu us; pid=%s\n",
env.duration, env.threshold_us, env.pid ? "selected" : "all");
fflush(stdout);
now = monotonic_milliseconds();
if (now < 0) {
err = now;
goto cleanup;
}
deadline = now + env.duration * 1000LL;
err = poll_until_deadline(ring, deadline);
if (err)
goto cleanup;
fsession_latency_bpf__detach(skel);
err = ring_buffer__consume(ring);
if (err < 0) {
fprintf(stderr, "ring buffer drain failed: %s\n", strerror(-err));
goto cleanup;
}
err = 0;
printf("SUMMARY calls=%llu slow=%llu errors=%llu dropped=%llu events=%llu\n",
skel->bss->stats.calls, skel->bss->stats.slow,
skel->bss->stats.errors, skel->bss->stats.dropped, events_printed);
cleanup:
ring_buffer__free(ring);
fsession_latency_bpf__destroy(skel);
return err != 0;
}
```
加载器的流程比较标准。解析命令行后,打开 skeleton 并把 `threshold_ns` 和 `target_tgid` 写入只读 BPF 配置。用户输入的阈值是微秒单位,乘以 1000 转换为纳秒后写入。调用 `load` 完成 BPF 程序加载,然后附加到 `vfs_read`。加载失败时的诊断同时给出内核 errno 和前置条件列表,方便用户排查问题。
`ring_buffer__new` 创建 ring buffer 消费者,注册 `handle_event` 回调处理每个事件。回调打印 comm、TGID、PID、请求字节数、结果和延迟微秒数,并递增 `events_printed` 计数。
`poll_until_deadline` 函数驱动事件消费循环。它计算单调时间截止点,每次迭代最多轮询 100 ms 直到截止或收到 SIGINT/SIGTERM。轮询返回 `-EINTR` 时继续循环,其他负值表示错误。
轮询结束后,加载器先调用 `fsession_latency_bpf__detach` 解除 fsession 程序的附加,使聚合计数器停止变化。然后调用 `ring_buffer__consume` 投递解除附加前已提交但尚未消费的记录。这个顺序是观察窗口边界正确性的一部分:先停止新事件产生,再排空已有事件,最后读取稳定的计数器。退出时释放 ring buffer 并销毁 skeleton。
## 编译与运行
从源码构建(仓库内置 libbpf 1.7.0 和 bpftool v7.7.0):
```bash
cd src/52-fsession-latency
make clean
make -j2
```
追踪指定服务进程 30 秒,上报 10 ms 及以上的读操作:
```bash
SERVICE_PID=$(pgrep -n my-service)
sudo ./fsession_latency --pid "$SERVICE_PID" --threshold-us 10000 --duration 30
```
`--pid` 比较的是 `bpf_get_current_pid_tgid()` 返回的 TGID,即宿主机(初始)PID 命名空间中的 TGID。上面的 `pgrep` 适用于直接在宿主机上运行的进程,容器或嵌套 PID 命名空间内的目标需要解析并传入其宿主机可见的 TGID。
命令行参数:
```text
Usage: ./fsession_latency [--threshold-us USEC] [--duration SEC] [--pid TGID] [--verbose]
Options:
-t, --threshold-us USEC 慢读阈值(默认:1000 微秒)
-d, --duration SEC 追踪时长,1-86400(默认:10 秒)
-p, --pid TGID 追踪指定进程 ID(默认:所有进程)
-v, --verbose 打印 libbpf 诊断信息
-h, --help 显示帮助
```
默认阈值 1000 微秒、持续 10 秒、追踪所有进程。`--duration` 接受 1 到 86400 秒,`--pid` 接受 1 到 2^32-1,`--threshold-us` 接受 0 到 `UINT64_MAX / 1000` 微秒(用户空间在加载前转换为纳秒)。空值和尾随字符都会被 `parse_u64` 拒绝。
以下输出在运行内核 `7.0.0-rc2+` 的 x86_64 环境中采集。集成测试执行 64 次普通文件读取加一次延迟 50 ms 的管道读取,管道读超过 10 ms 阈值后出现在事件流中:
```console
Tracing vfs_read for 1 seconds; threshold=10000 us; pid=selected
EVENT comm=python3 tgid=1257 pid=1257 requested=1 result=1 latency_us=50160
SUMMARY calls=66 slow=1 errors=0 dropped=0 events=1
TEST-SUMMARY miss_calls=0 high_threshold_calls=66 high_threshold_slow=0 threshold_calls=66 threshold_slow=1 events=1 dropped=0
PASS: PID filtering, threshold miss, and slow-read reporting behaved as expected
```
具体调用次数、PID 值和延迟数值因运行环境而异。
运行测试:
```bash
sudo make test
```
集成测试验证四个运行时路径:目标 TGID 为 `UINT32_MAX` 产生零调用和事件、10,000,000 微秒阈值使普通读产生零慢事件、64 次普通文件读加一次延迟 50 ms 的管道读在 10,000 微秒阈值下产生慢事件、阈值为 0 时 `slow == events + dropped` 证明每个阈值匹配调用要么被投递要么被计入 ring buffer 预留失败。CLI 测试覆盖无效 duration、空 duration、空 threshold 和空 PID。
### 环境要求
| 要求 | 详情 |
|---|---|
| 内核版本 | Linux 7.0+(fsession 首次引入) |
| BTF | 必须启用 (`CONFIG_DEBUG_INFO_BTF=y`) |
| 内核配置 | `CONFIG_BPF=y`, `CONFIG_BPF_SYSCALL=y`, `CONFIG_BPF_JIT=y`, `CONFIG_BPF_EVENTS=y`, `CONFIG_DEBUG_INFO_BTF=y`, `CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y` |
| BPF JIT | 运行时必须启用 |
| 架构 | 已在 x86_64 上测试 |
| 权限 | root |
运行时行为在 x86_64 内核 `7.0.0-rc2+` 上测试。上游合并提交为 `f17b474e36647c23801ef8fdaf2255ab66dd2973`。
## 观察范围
本工具测量 `vfs_read` 内部的时间,覆盖到达该函数的读路径并按精确的宿主机可见 TGID 筛选。更完整的追踪器可以在保持相同 fsession 关联模式的基础上加入 cgroup、路径、inode、挂载点、栈或长时运行指标维度,现有的 `dropped` 计数器在负载下仍是事件丢失信号。
## 总结
本教程展示了如何使用 Linux 7.0 的 fsession 机制测量内核函数延迟。相比传统的 fentry/fexit 两程序配合哈希表的方案,fsession 用一个程序同时处理进入和返回,8 字节的调用级 session cookie 替代外部关联 map 携带时间戳,代码更简洁、无状态泄漏风险、性能开销更低。
工具的设计演示了 fsession 的核心模式:用 `bpf_session_is_return` 区分阶段,用 `bpf_session_cookie` 传递调用级状态,返回阶段直接访问函数参数和返回值。这个模式适用于任何需要关联函数进入和返回的场景。
> 如果你想深入了解 eBPF,请查看我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或访问我们的网站 <https://eunomia.dev/tutorials/>。
## 参考资料
- [Linux fsession 合并提交](https://github.com/torvalds/linux/commit/f17b474e36647c23801ef8fdaf2255ab66dd2973)
- [上游 fsession 程序自测试](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/progs/fsession_test.c)
- [上游 fsession 运行器自测试](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/prog_tests/fsession_test.c)
- [libbpf 1.7.0](https://github.com/libbpf/libbpf/releases/tag/v1.7.0)
- [bpftool v7.7.0](https://github.com/libbpf/bpftool/releases/tag/v7.7.0)

View File

@@ -0,0 +1,74 @@
// SPDX-License-Identifier: GPL-2.0
#define bpf_session_is_return bpf_session_is_return_vmlinux_snapshot
#define bpf_session_cookie bpf_session_cookie_vmlinux_snapshot
#include "vmlinux.h"
#undef bpf_session_is_return
#undef bpf_session_cookie
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include "fsession_latency.h"
char LICENSE[] SEC("license") = "GPL";
const volatile __u64 threshold_ns;
const volatile __u32 target_tgid;
struct latency_stats stats;
struct {
__uint(type, BPF_MAP_TYPE_RINGBUF);
__uint(max_entries, 256 * 1024);
} events SEC(".maps");
/*
* The repository vmlinux.h snapshot predates the ctx argument on these
* kfunc prototypes. Rename those stale declarations while including the
* snapshot, then provide the Linux 7.0 signatures below.
*/
extern bool bpf_session_is_return(void *ctx) __ksym;
extern __u64 *bpf_session_cookie(void *ctx) __ksym;
SEC("fsession/vfs_read")
int BPF_PROG(measure_vfs_read, struct file *file, char *buf, size_t count,
loff_t *pos, ssize_t ret)
{
__u64 pid_tgid = bpf_get_current_pid_tgid();
__u64 *started = bpf_session_cookie(ctx);
struct latency_event *event;
__u64 latency;
if (!bpf_session_is_return(ctx)) {
if (target_tgid && pid_tgid >> 32 != target_tgid) {
*started = 0;
return 0;
}
*started = bpf_ktime_get_ns();
return 0;
}
if (!*started)
return 0;
latency = bpf_ktime_get_ns() - *started;
__sync_fetch_and_add(&stats.calls, 1);
if (ret < 0)
__sync_fetch_and_add(&stats.errors, 1);
if (latency < threshold_ns)
return 0;
__sync_fetch_and_add(&stats.slow, 1);
event = bpf_ringbuf_reserve(&events, sizeof(*event), 0);
if (!event) {
__sync_fetch_and_add(&stats.dropped, 1);
return 0;
}
event->pid = (__u32)pid_tgid;
event->tgid = pid_tgid >> 32;
event->requested = count;
event->result = ret;
event->latency_ns = latency;
bpf_get_current_comm(event->comm, sizeof(event->comm));
bpf_ringbuf_submit(event, 0);
return 0;
}

View File

@@ -0,0 +1,255 @@
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <errno.h>
#include <getopt.h>
#include <signal.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <bpf/libbpf.h>
#include "fsession_latency.h"
#include "fsession_latency.skel.h"
static volatile sig_atomic_t exiting;
static struct env {
unsigned long long threshold_us;
unsigned int duration;
unsigned int pid;
bool verbose;
} env = {
.threshold_us = 1000,
.duration = 10,
};
static unsigned long long events_printed;
static void handle_signal(int signal)
{
(void)signal;
exiting = 1;
}
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s [--threshold-us USEC] [--duration SEC] [--pid TGID] "
"[--verbose]\n\n"
"Report vfs_read calls at or above a latency threshold.\n\n"
"Options:\n"
" -t, --threshold-us USEC slow-read threshold (default: 1000)\n"
" -d, --duration SEC trace duration, 1-86400 (default: 10)\n"
" -p, --pid TGID trace one process ID (default: all)\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_u64(const char *value, unsigned long long maximum,
unsigned long long *result)
{
char *end = NULL;
unsigned long long parsed;
errno = 0;
parsed = strtoull(value, &end, 10);
if (errno || end == value || *end || parsed > maximum)
return -EINVAL;
*result = parsed;
return 0;
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "threshold-us", required_argument, NULL, 't' },
{ "duration", required_argument, NULL, 'd' },
{ "pid", required_argument, NULL, 'p' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
unsigned long long parsed;
int option;
while ((option = getopt_long(argc, argv, "t:d:p:vh", options, NULL)) != -1) {
switch (option) {
case 't':
if (parse_u64(optarg, UINT64_MAX / 1000, &env.threshold_us)) {
fprintf(stderr, "invalid threshold in microseconds: %s\n", optarg);
return -EINVAL;
}
break;
case 'd':
if (parse_u64(optarg, 86400, &parsed) || parsed == 0) {
fprintf(stderr, "invalid duration in seconds: %s\n", optarg);
return -EINVAL;
}
env.duration = parsed;
break;
case 'p':
if (parse_u64(optarg, UINT32_MAX, &parsed) || parsed == 0) {
fprintf(stderr, "invalid process ID: %s\n", optarg);
return -EINVAL;
}
env.pid = parsed;
break;
case 'v':
env.verbose = true;
break;
case 'h':
usage(argv[0]);
exit(0);
default:
return -EINVAL;
}
}
if (optind != argc)
return -EINVAL;
return 0;
}
static long long monotonic_milliseconds(void)
{
struct timespec timestamp;
if (clock_gettime(CLOCK_MONOTONIC, &timestamp))
return -errno;
return timestamp.tv_sec * 1000LL + timestamp.tv_nsec / 1000000;
}
static int handle_event(void *context, void *data, size_t size)
{
const struct latency_event *event = data;
(void)context;
if (size < sizeof(*event))
return 0;
printf("EVENT comm=%-16s tgid=%u pid=%u requested=%llu result=%lld "
"latency_us=%llu\n",
event->comm, event->tgid, event->pid, event->requested,
event->result, event->latency_ns / 1000);
events_printed++;
return 0;
}
static int poll_until_deadline(struct ring_buffer *ring, long long deadline)
{
while (!exiting) {
long long now = monotonic_milliseconds();
int consumed;
if (now < 0)
return (int)now;
if (now >= deadline)
break;
consumed = ring_buffer__poll(ring,
deadline - now > 100 ? 100 : deadline - now);
if (consumed == -EINTR)
continue;
if (consumed < 0) {
fprintf(stderr, "ring buffer poll failed: %s\n",
strerror(-consumed));
return consumed;
}
}
return 0;
}
int main(int argc, char **argv)
{
struct fsession_latency_bpf *skel = NULL;
struct ring_buffer *ring = NULL;
long long deadline, now;
int err;
err = parse_args(argc, argv);
if (err) {
usage(argv[0]);
return 1;
}
libbpf_set_print(libbpf_print_fn);
signal(SIGINT, handle_signal);
signal(SIGTERM, handle_signal);
skel = fsession_latency_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return 1;
}
skel->rodata->threshold_ns = env.threshold_us * 1000;
skel->rodata->target_tgid = env.pid;
err = fsession_latency_bpf__load(skel);
if (err) {
fprintf(stderr,
"failed to load fsession program: %s\n"
"This tool requires Linux 7.0+, BTF, BPF JIT, and x86_64 "
"fsession support.\n",
strerror(-err));
goto cleanup;
}
err = fsession_latency_bpf__attach(skel);
if (err) {
fprintf(stderr, "failed to attach to vfs_read: %s\n", strerror(-err));
goto cleanup;
}
ring = ring_buffer__new(bpf_map__fd(skel->maps.events), handle_event, NULL, NULL);
err = libbpf_get_error(ring);
if (err) {
ring = NULL;
fprintf(stderr, "failed to create ring buffer: %s\n", strerror(-err));
goto cleanup;
}
printf("Tracing vfs_read for %u seconds; threshold=%llu us; pid=%s\n",
env.duration, env.threshold_us, env.pid ? "selected" : "all");
fflush(stdout);
now = monotonic_milliseconds();
if (now < 0) {
err = now;
goto cleanup;
}
deadline = now + env.duration * 1000LL;
err = poll_until_deadline(ring, deadline);
if (err)
goto cleanup;
fsession_latency_bpf__detach(skel);
err = ring_buffer__consume(ring);
if (err < 0) {
fprintf(stderr, "ring buffer drain failed: %s\n", strerror(-err));
goto cleanup;
}
err = 0;
printf("SUMMARY calls=%llu slow=%llu errors=%llu dropped=%llu events=%llu\n",
skel->bss->stats.calls, skel->bss->stats.slow,
skel->bss->stats.errors, skel->bss->stats.dropped, events_printed);
cleanup:
ring_buffer__free(ring);
fsession_latency_bpf__destroy(skel);
return err != 0;
}

View File

@@ -0,0 +1,23 @@
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __FSESSION_LATENCY_H
#define __FSESSION_LATENCY_H
#define FSESSION_COMM_LEN 16
struct latency_event {
unsigned int pid;
unsigned int tgid;
unsigned long long requested;
long long result;
unsigned long long latency_ns;
char comm[FSESSION_COMM_LEN];
};
struct latency_stats {
unsigned long long calls;
unsigned long long slow;
unsigned long long errors;
unsigned long long dropped;
};
#endif /* __FSESSION_LATENCY_H */

View File

@@ -0,0 +1,217 @@
#!/usr/bin/env python3
"""Deterministic runtime checks for the fsession latency tool."""
from __future__ import annotations
import os
import pathlib
import re
import select
import subprocess
import sys
import threading
import time
SUMMARY = re.compile(
r"SUMMARY calls=(\d+) slow=(\d+) errors=(\d+) dropped=(\d+) events=(\d+)"
)
EVENT = re.compile(
r"^EVENT comm=.{16} tgid=(\d+) pid=(\d+) requested=(\d+) "
r"result=(-?\d+) latency_us=(\d+)$",
re.MULTILINE,
)
def wait_until_attached(process: subprocess.Popen[str]) -> list[str]:
lines: list[str] = []
deadline = time.monotonic() + 8
assert process.stdout is not None
while time.monotonic() < deadline:
readable, _, _ = select.select([process.stdout], [], [], 0.25)
if readable:
line = process.stdout.readline()
if line:
lines.append(line)
if line.startswith("Tracing vfs_read"):
return lines
if process.poll() is not None:
break
process.kill()
remainder, _ = process.communicate()
raise AssertionError("tool did not attach:\n" + "".join(lines) + remainder)
def generate_reads() -> None:
path = pathlib.Path(__file__)
descriptor = os.open(path, os.O_RDONLY)
try:
for _ in range(64):
os.lseek(descriptor, 0, os.SEEK_SET)
os.read(descriptor, 256)
finally:
os.close(descriptor)
read_fd, write_fd = os.pipe()
def delayed_write() -> None:
time.sleep(0.05)
os.write(write_fd, b"x")
os.close(write_fd)
writer = threading.Thread(target=delayed_write)
writer.start()
try:
assert os.read(read_fd, 1) == b"x"
finally:
os.close(read_fd)
writer.join()
def run_trace(
binary: str,
*arguments: str,
workload: bool,
continuous: bool = False,
) -> tuple[dict[str, int], str]:
process = subprocess.Popen(
[binary, *arguments],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
bufsize=1,
)
lines = wait_until_attached(process)
stop_workload = threading.Event()
producer: threading.Thread | None = None
if continuous:
path = pathlib.Path(__file__)
def generate_continuous_reads() -> None:
descriptor = os.open(path, os.O_RDONLY)
try:
while not stop_workload.is_set():
os.lseek(descriptor, 0, os.SEEK_SET)
os.read(descriptor, 1)
time.sleep(0.001)
finally:
os.close(descriptor)
producer = threading.Thread(target=generate_continuous_reads)
producer.start()
elif workload:
generate_reads()
try:
remainder, _ = process.communicate(timeout=8)
finally:
stop_workload.set()
if producer is not None:
producer.join()
output = "".join(lines) + remainder
if process.returncode != 0:
raise AssertionError(f"tool failed with {process.returncode}:\n{output}")
match = SUMMARY.search(output)
if not match:
raise AssertionError("missing summary:\n" + output)
values = dict(zip(("calls", "slow", "errors", "dropped", "events"), map(int, match.groups())))
return values, output
def main() -> int:
binary = sys.argv[1] if len(sys.argv) > 1 else "./fsession_latency"
invalid = subprocess.run(
[binary, "--duration", "invalid"],
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
check=False,
)
assert invalid.returncode != 0
assert "invalid duration" in invalid.stdout
empty_threshold = subprocess.run(
[binary, "--threshold-us", ""],
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
check=False,
)
assert empty_threshold.returncode != 0
assert "invalid threshold" in empty_threshold.stdout
pid = str(os.getpid())
missed, _ = run_trace(
binary,
"--duration", "1",
"--threshold-us", "0",
"--pid", str(2**32 - 1),
workload=True,
)
assert missed == {"calls": 0, "slow": 0, "errors": 0, "dropped": 0, "events": 0}, missed
below_threshold, _ = run_trace(
binary,
"--duration", "1",
"--threshold-us", "10000000",
"--pid", pid,
workload=True,
)
assert below_threshold["calls"] >= 64, below_threshold
assert below_threshold["slow"] == 0, below_threshold
assert below_threshold["events"] == 0, below_threshold
slow, slow_output = run_trace(
binary,
"--duration", "1",
"--threshold-us", "10000",
"--pid", pid,
workload=True,
)
assert slow["calls"] >= 65, slow
assert 1 <= slow["slow"] < slow["calls"], slow
assert slow["events"] > 0, slow
event_records = [tuple(map(int, match.groups())) for match in EVENT.finditer(slow_output)]
assert any(
tgid == os.getpid()
and thread_pid == os.getpid()
and requested == 1
and result == 1
and latency_us >= 10000
for tgid, thread_pid, requested, result, latency_us in event_records
), slow_output
deadline, _ = run_trace(
binary,
"--duration", "1",
"--threshold-us", "0",
"--pid", pid,
workload=False,
continuous=True,
)
assert deadline["calls"] > 0, deadline
assert deadline["events"] > 0, deadline
assert deadline["slow"] == deadline["events"] + deadline["dropped"], deadline
tracing_line = next(line for line in slow_output.splitlines() if line.startswith("Tracing "))
first_event = next(line for line in slow_output.splitlines() if line.startswith("EVENT "))
tool_summary = next(line for line in slow_output.splitlines() if line.startswith("SUMMARY "))
print(tracing_line)
print(first_event)
print(tool_summary)
print(
"TEST-SUMMARY miss_calls=0 "
f"high_threshold_calls={below_threshold['calls']} high_threshold_slow=0 "
f"threshold_calls={slow['calls']} threshold_slow={slow['slow']} "
f"events={slow['events']} dropped={slow['dropped']}"
)
print("PASS: PID filtering, threshold miss, and slow-read reporting behaved as expected")
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,10 @@
level=Depth
type=Networking
kernel_min=6.16
kernel_min_basis=documented
architectures=x86_64
btf=required
kernel_config=CONFIG_BPF=y,CONFIG_BPF_SYSCALL=y,CONFIG_BPF_JIT=y,CONFIG_DEBUG_INFO_BTF=y,CONFIG_NET_SCHED=y,CONFIG_NET_SCH_BPF=y
hardware=network-interface
root=required
test_status=ci-build

15
src/53-egress-pacer/.gitignore vendored Normal file
View File

@@ -0,0 +1,15 @@
# Build artifacts
.output/
*.o
*.skel.h
# Generated binary
egress_pacer
# Python cache
__pycache__/
# Editor files
*.swp
*~
.vscode/

View File

@@ -0,0 +1,96 @@
# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
OUTPUT := .output
CLANG ?= clang
LIBBPF_SRC := $(abspath ../third_party/libbpf/src)
BPFTOOL_SRC := $(abspath ../third_party/bpftool/src)
LIBBPF_OBJ := $(abspath $(OUTPUT)/libbpf.a)
BPFTOOL_OUTPUT ?= $(abspath $(OUTPUT)/bpftool)
BPFTOOL ?= $(BPFTOOL_OUTPUT)/bootstrap/bpftool
ARCH ?= $(shell uname -m | sed 's/x86_64/x86/' \
| sed 's/arm.*/arm/' \
| sed 's/aarch64/arm64/' \
| sed 's/ppc64le/powerpc/' \
| sed 's/mips.*/mips/' \
| sed 's/riscv64/riscv/' \
| sed 's/loongarch64/loongarch/')
VMLINUX := ../third_party/vmlinux/$(ARCH)/vmlinux.h
INCLUDES := -I$(OUTPUT) -I../third_party/libbpf/include/uapi -I$(dir $(VMLINUX)) -I.
CFLAGS := -g -Wall -Wextra
ALL_LDFLAGS := $(LDFLAGS) $(EXTRA_LDFLAGS)
APPS = egress_pacer
CLANG_BPF_SYS_INCLUDES ?= $(shell $(CLANG) -v -E - </dev/null 2>&1 \
| sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }')
ifeq ($(V),1)
Q =
msg =
else
Q = @
msg = @printf ' %-8s %s%s\n' "$(1)" \
"$(patsubst $(abspath $(OUTPUT))/%,%,$(2))" \
"$(if $(3), $(3))";
MAKEFLAGS += --no-print-directory
endif
define allow-override
$(if $(or $(findstring environment,$(origin $(1))),\
$(findstring command line,$(origin $(1)))),,\
$(eval $(1) = $(2)))
endef
$(call allow-override,CC,$(CROSS_COMPILE)cc)
$(call allow-override,LD,$(CROSS_COMPILE)ld)
.PHONY: all clean test
all: $(APPS)
clean:
$(call msg,CLEAN)
$(Q)rm -rf $(OUTPUT) $(APPS)
test: all
@if [ "$$(id -u)" -ne 0 ]; then \
echo "test requires root; run: sudo python3 tests/test_egress_pacer.py ./egress_pacer" >&2; \
exit 1; \
fi
python3 tests/test_egress_pacer.py ./egress_pacer
$(OUTPUT) $(OUTPUT)/libbpf $(BPFTOOL_OUTPUT):
$(call msg,MKDIR,$@)
$(Q)mkdir -p $@
$(LIBBPF_OBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)/libbpf
$(call msg,LIB,$@)
$(Q)$(MAKE) -C $(LIBBPF_SRC) BUILD_STATIC_ONLY=1 \
OBJDIR=$(dir $@)/libbpf DESTDIR=$(dir $@) \
INCLUDEDIR= LIBDIR= UAPIDIR= install
$(BPFTOOL): | $(BPFTOOL_OUTPUT)
$(call msg,BPFTOOL,$@)
$(Q)$(MAKE) ARCH= CROSS_COMPILE= OUTPUT=$(BPFTOOL_OUTPUT)/ -C $(BPFTOOL_SRC) bootstrap
$(OUTPUT)/%.bpf.o: %.bpf.c $(LIBBPF_OBJ) $(wildcard %.h) $(VMLINUX) | $(OUTPUT) $(BPFTOOL)
$(call msg,BPF,$@)
$(Q)$(CLANG) -g -O2 -target bpf -D__TARGET_ARCH_$(ARCH) \
$(INCLUDES) $(CLANG_BPF_SYS_INCLUDES) \
-c $(filter %.c,$^) -o $(patsubst %.bpf.o,%.tmp.bpf.o,$@)
$(Q)$(BPFTOOL) gen object $@ $(patsubst %.bpf.o,%.tmp.bpf.o,$@)
$(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(OUTPUT) $(BPFTOOL)
$(call msg,GEN-SKEL,$@)
$(Q)$(BPFTOOL) gen skeleton $< > $@
$(patsubst %,$(OUTPUT)/%.o,$(APPS)): %.o: %.skel.h
$(OUTPUT)/%.o: %.c $(wildcard %.h) | $(OUTPUT)
$(call msg,CC,$@)
$(Q)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
$(APPS): %: $(OUTPUT)/%.o $(LIBBPF_OBJ) | $(OUTPUT)
$(call msg,BINARY,$@)
$(Q)$(CC) $(CFLAGS) $^ $(ALL_LDFLAGS) -lelf -lz -o $@
.DELETE_ON_ERROR:
.SECONDARY:

View File

@@ -0,0 +1,721 @@
# eBPF Tutorial: Building an Egress Pacer with BPF Qdisc
Imagine you need to verify how your application behaves under low-bandwidth conditions in a test environment. You create a veth pair to simulate a network link and want to limit egress to 64 Kbit/s while observing packet drops when the queue fills up. Traditional rate limiting requires complex tc configurations or user-space proxies, but Linux 6.16 introduced BPF qdisc, which lets you implement a complete queuing discipline directly in eBPF.
This tutorial shows how to use `struct_ops` to build a FIFO rate limiter. The program registers as a root qdisc, manages skb enqueue and dequeue, computes transmission time from packet length, uses the watchdog to schedule the next dequeue, and cleans up queued packets when the qdisc is removed. This is a complete qdisc lifecycle example, suitable for validating skb ownership, transmission timing, and resource cleanup on controlled interfaces like veth, TAP, or IFB.
> Complete source code: <https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/53-egress-pacer>
## Why BPF Qdisc Is Needed
Traditional traffic shaping approaches have clear limitations.
tc provides rich qdisc options such as TBF (Token Bucket Filter) and HTB (Hierarchical Token Bucket), but these are predefined algorithms in the kernel. If you need a custom scheduling policy, such as prioritizing by application type or dynamically adjusting rates based on real-time metrics, you must either modify kernel code or use complex tc classifier combinations.
Ordinary TC BPF programs can inspect and process packets, but queueing and transmission timing remain under the original qdisc's control. [Lesson 20](https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/20-tc) demonstrates this pattern: BPF programs decide whether to pass or drop packets, but cannot control when packets are sent. If you want custom queueing policies, TC BPF is not enough.
User-space rate limiting approaches like tc-netem with external tools or DPDK can implement complex traffic shaping, but they introduce additional context switches and deployment complexity. In a test environment, you may just want to quickly validate an idea rather than set up a full user-space network stack.
Linux 6.16 introduced BPF implementations of `Qdisc_ops` callbacks through `struct_ops`. Programs implement `enqueue`, `dequeue`, `init`, `reset`, and `destroy`, and can dynamically register as a qdisc. This means you can use BPF programs to fully control packet queueing and transmission timing while retaining the performance advantages of the kernel network stack. The `egress_pacer` in this lesson is a FIFO rate limiter built exactly this way.
## How BPF Qdisc Works
Before diving into the code, let's understand the overall flow of a BPF qdisc. Unlike ordinary TC BPF programs, a qdisc BPF program registers as the root qdisc and completely takes over packet queueing and transmission.
The user-space program first sets the rate and queue limit, loads the BPF program and registers the `struct_ops` implementation, then attaches it to the `TC_H_ROOT` position on the target interface. From then on, all egress packets through that interface are handled by the BPF program.
When the kernel needs to send a packet, it calls the `enqueue` callback. The BPF program creates a node to hold the skb, computes the earliest transmission time based on packet length, then adds the node to the FIFO queue. If the queue is full or node allocation fails, the program drops the packet and updates statistics.
When the kernel tries to retrieve a packet for transmission, it calls the `dequeue` callback. The BPF program checks the transmission time of the head node. If the current time has reached or passed that time, the program retrieves the skb and returns it to the kernel; if the transmission time hasn't arrived yet, the program uses `bpf_qdisc_watchdog_schedule` to set a timer and returns NULL. When the timer expires, the kernel calls `dequeue` again.
When the user-space program removes the qdisc, the kernel calls the `reset` callback. The BPF program iterates through all unsent packets in the queue, releases their resources, and zeros out counters.
Throughout this lifecycle, the BPF program has complete ownership of the skb. From receiving the skb in `enqueue`, to returning it in `dequeue` or releasing it in `reset`, packets are held in BPF-managed data structures.
![egress_pacer data flow: from configuration, attachment, enqueue, BPF FIFO, dequeue to transmit path, including policy-drop branch, watchdog loop, and reset lifecycle](https://github.com/eunomia-bpf/bpf-developer-tutorial/raw/main/src/53-egress-pacer/egress-pacer-flow.png)
The solid lines in the diagram start from configuration and attachment, then trace an skb from enqueue into the BPF FIFO and out through dequeue to the transmit path. When the queue is full or node allocation returns null, packets take the policy-drop branch. An early dequeue pushes the node back to the front, schedules the watchdog, and returns NULL; when the watchdog fires, it re-enters dequeue. The dashed lines show the lifecycle path: removing the root qdisc triggers reset, which releases skbs in the queue and zeros qlen and backlog.
## Code Implementation
This tool consists of four files: a shared header defining the statistics structure, a compatibility header providing graph-object kfunc declarations, the BPF program implementing qdisc callbacks, and the user-space loader managing the lifecycle and printing results.
### Shared Header
`egress_pacer.h` defines the statistics structure shared between BPF and user space. These six fields categorize packet outcomes throughout their lifetime, residing in the BSS section as the result channel.
```c
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __EGRESS_PACER_H
#define __EGRESS_PACER_H
struct pacer_stats {
unsigned long long enqueued;
unsigned long long dequeued;
unsigned long long policy_dropped;
unsigned long long cleanup_dropped;
unsigned long long bytes_dequeued;
unsigned long long max_qlen;
};
#endif /* __EGRESS_PACER_H */
```
Packets that successfully enter and leave the FIFO are counted in `enqueued` and `dequeued` respectively. When the queue is full or node allocation fails, `policy_dropped` increases. Packets still in the queue during reset are counted in `cleanup_dropped`. `bytes_dequeued` accumulates the bytes actually returned to the transmit path, and `max_qlen` records the peak queue depth.
### Compatibility Header
This example requires graph-object kfuncs, so a local compatibility header provides the BPF qdisc declarations. These declarations exist only in this lesson's directory and are independent from the shared headers used across the repository.
```c
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __EGRESS_PACER_BPF_EXPERIMENTAL_H
#define __EGRESS_PACER_BPF_EXPERIMENTAL_H
#include <bpf/bpf_core_read.h>
#define __contains(name, node) \
__attribute__((btf_decl_tag("contains:" #name ":" #node)))
extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym;
#define bpf_obj_new(type) \
((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL))
extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym;
#define bpf_obj_drop(kptr) bpf_obj_drop_impl(kptr, NULL)
extern int bpf_list_push_front_impl(struct bpf_list_head *head,
struct bpf_list_node *node,
void *meta, __u64 off) __ksym;
#define bpf_list_push_front(head, node) \
bpf_list_push_front_impl(head, node, NULL, 0)
extern int bpf_list_push_back_impl(struct bpf_list_head *head,
struct bpf_list_node *node,
void *meta, __u64 off) __ksym;
#define bpf_list_push_back(head, node) \
bpf_list_push_back_impl(head, node, NULL, 0)
extern struct bpf_list_node *
bpf_list_pop_front(struct bpf_list_head *head) __ksym;
#endif /* __EGRESS_PACER_BPF_EXPERIMENTAL_H */
```
This file provides the graph-object helpers needed to build the queue. `bpf_obj_new` creates a packet node, and `bpf_obj_drop` releases it when done. `bpf_list_push_back` and `bpf_list_pop_front` arrange these nodes into a FIFO. The `__contains` macro generates a BTF tag that tells the verifier which node type belongs inside the list head, enabling safe graph tracking.
### BPF Program
`egress_pacer.bpf.c` uses `SEC(".struct_ops")` to declare the `Qdisc_ops` implementation. The kernel registers these callbacks at load time, making the BPF program a complete qdisc.
```c
// SPDX-License-Identifier: GPL-2.0
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include "bpf_experimental.h"
#include "egress_pacer.h"
char LICENSE[] SEC("license") = "GPL";
#define NET_XMIT_SUCCESS 0x00
#define NET_XMIT_DROP 0x01
#define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8)))
struct bpf_sk_buff_ptr {
struct sk_buff *skb;
};
extern void bpf_qdisc_skb_drop(struct sk_buff *skb,
struct bpf_sk_buff_ptr *to_free) __ksym;
extern void bpf_qdisc_watchdog_schedule(struct Qdisc *sch, __u64 expire,
__u64 delta_ns) __ksym;
extern void bpf_qdisc_bstats_update(struct Qdisc *sch,
const struct sk_buff *skb) __ksym;
extern void bpf_kfree_skb(struct sk_buff *skb) __ksym;
const volatile __u64 rate_kbps = 1024;
const volatile __u32 queue_limit = 256;
struct pacer_stats stats;
struct packet_node {
__u64 eligible_ns;
__u32 packet_len;
struct sk_buff __kptr *skb;
struct bpf_list_node node;
};
private(A) struct bpf_spin_lock queue_lock;
private(A) struct bpf_list_head packet_queue __contains(packet_node, node);
private(A) __u64 next_departure_ns;
static struct qdisc_skb_cb *qdisc_skb_cb(const struct sk_buff *skb)
{
return (struct qdisc_skb_cb *)skb->cb;
}
static __u32 qdisc_packet_len(const struct sk_buff *skb)
{
return qdisc_skb_cb(skb)->pkt_len;
}
SEC("struct_ops/egress_pacer_enqueue")
int BPF_PROG(egress_pacer_enqueue, struct sk_buff *skb, struct Qdisc *sch,
struct bpf_sk_buff_ptr *to_free)
{
struct packet_node *packet;
__u64 eligible_ns, interval_ns, now;
__u32 packet_len;
packet_len = qdisc_packet_len(skb);
if (sch->q.qlen >= sch->limit)
goto drop;
packet = bpf_obj_new(typeof(*packet));
if (!packet)
goto drop;
now = bpf_ktime_get_ns();
interval_ns = (__u64)packet_len * 8 * 1000000ULL / rate_kbps;
if (!interval_ns)
interval_ns = 1;
packet->packet_len = packet_len;
skb = bpf_kptr_xchg(&packet->skb, skb);
if (skb)
bpf_qdisc_skb_drop(skb, to_free);
bpf_spin_lock(&queue_lock);
eligible_ns = next_departure_ns > now ? next_departure_ns : now;
next_departure_ns = eligible_ns + interval_ns;
packet->eligible_ns = eligible_ns;
bpf_list_push_back(&packet_queue, &packet->node);
sch->q.qlen++;
sch->qstats.backlog += packet_len;
__sync_fetch_and_add(&stats.enqueued, 1);
if (sch->q.qlen > stats.max_qlen)
stats.max_qlen = sch->q.qlen;
bpf_spin_unlock(&queue_lock);
return NET_XMIT_SUCCESS;
drop:
bpf_qdisc_skb_drop(skb, to_free);
__sync_fetch_and_add(&stats.policy_dropped, 1);
return NET_XMIT_DROP;
}
SEC("struct_ops/egress_pacer_dequeue")
struct sk_buff *BPF_PROG(egress_pacer_dequeue, struct Qdisc *sch)
{
struct bpf_list_node *node;
struct packet_node *packet;
struct sk_buff *skb = NULL;
__u64 expire, now;
__u32 packet_len;
bpf_spin_lock(&queue_lock);
node = bpf_list_pop_front(&packet_queue);
bpf_spin_unlock(&queue_lock);
if (!node) {
return NULL;
}
packet = container_of(node, struct packet_node, node);
now = bpf_ktime_get_ns();
if (now < packet->eligible_ns) {
expire = packet->eligible_ns;
bpf_spin_lock(&queue_lock);
bpf_list_push_front(&packet_queue, &packet->node);
bpf_spin_unlock(&queue_lock);
bpf_qdisc_watchdog_schedule(sch, expire, 0);
return NULL;
}
packet_len = packet->packet_len;
skb = bpf_kptr_xchg(&packet->skb, skb);
bpf_spin_lock(&queue_lock);
sch->q.qlen--;
sch->qstats.backlog -= packet_len;
bpf_spin_unlock(&queue_lock);
bpf_obj_drop(packet);
if (!skb)
return NULL;
bpf_qdisc_bstats_update(sch, skb);
__sync_fetch_and_add(&stats.dequeued, 1);
__sync_fetch_and_add(&stats.bytes_dequeued, packet_len);
return skb;
}
SEC("struct_ops/egress_pacer_init")
int BPF_PROG(egress_pacer_init, struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
(void)opt;
(void)extack;
sch->limit = queue_limit;
return 0;
}
SEC("struct_ops/egress_pacer_reset")
void BPF_PROG(egress_pacer_reset, struct Qdisc *sch)
{
int queued = sch->q.qlen;
int i;
bpf_for(i, 0, queued) {
struct bpf_list_node *node;
struct packet_node *packet;
struct sk_buff *skb = NULL;
bpf_spin_lock(&queue_lock);
node = bpf_list_pop_front(&packet_queue);
bpf_spin_unlock(&queue_lock);
if (!node)
break;
packet = container_of(node, struct packet_node, node);
skb = bpf_kptr_xchg(&packet->skb, skb);
if (skb) {
bpf_kfree_skb(skb);
__sync_fetch_and_add(&stats.cleanup_dropped, 1);
}
bpf_obj_drop(packet);
}
bpf_spin_lock(&queue_lock);
next_departure_ns = 0;
sch->q.qlen = 0;
sch->qstats.backlog = 0;
bpf_spin_unlock(&queue_lock);
}
SEC("struct_ops/egress_pacer_destroy")
void BPF_PROG(egress_pacer_destroy, struct Qdisc *sch)
{
(void)sch;
}
SEC(".struct_ops")
struct Qdisc_ops pacer = {
.enqueue = (void *)egress_pacer_enqueue,
.dequeue = (void *)egress_pacer_dequeue,
.init = (void *)egress_pacer_init,
.reset = (void *)egress_pacer_reset,
.destroy = (void *)egress_pacer_destroy,
.id = "bpf_pacer",
};
```
The program structure centers on three core qdisc phases. The two `const volatile` variables `rate_kbps` and `queue_limit` reside in the `.rodata` section; user space writes the rate and queue limit after `open()` but before `load()`, and the verifier treats them as compile-time constants. The `packet_node` structure defines nodes in the queue, containing transmission time, packet length, skb pointer, and list node. The `__kptr` annotation tells the verifier this field owns a kernel pointer.
`egress_pacer_enqueue` is the enqueue callback. It first retrieves the packet length from `qdisc_skb_cb(skb)->pkt_len`, then checks if the queue has space. If the queue is full or node creation fails, it calls `bpf_qdisc_skb_drop` to drop the packet and increments `policy_dropped`. The actual ownership transfer happens via `bpf_kptr_xchg`: after success, the skb is placed in the node's `__kptr` field, and the node becomes responsible for it. The program then computes transmission time while holding `queue_lock`, adds the node to the FIFO tail, and updates qlen and backlog. The transmission interval is calculated as `packet_len * 8 * 1000000 / rate_kbps` in nanoseconds. After the queue has been idle for a while, `next_departure_ns` will be earlier than the current time, allowing the first arriving packet to leave immediately.
`egress_pacer_dequeue` is the dequeue callback. It retrieves the node from the front and compares `eligible_ns` with the current time. If the packet is ready to send, it uses `bpf_kptr_xchg` to retrieve the skb from the node, decrements qlen and backlog, releases the node, then calls `bpf_qdisc_bstats_update` to update qdisc statistics and returns the skb. If transmission time hasn't arrived, it pushes the node back to the front, calls `bpf_qdisc_watchdog_schedule` to set a timer, and returns NULL. The kernel will call dequeue again at the specified time.
`egress_pacer_reset` handles cleanup. When the qdisc is removed, there may still be unsent packets in the queue. The program uses `bpf_for` to retrieve nodes one by one, calls `bpf_kfree_skb` to release the skb, then destroys the node object. Cleared packets are counted in `cleanup_dropped`, separate from enqueue-stage `policy_dropped`. After draining the list, the program zeros `next_departure_ns`, qlen, and backlog together.
The `SEC(".struct_ops")` block at the end registers all callbacks as a `Qdisc_ops` structure. The `id` field specifies the qdisc name as `bpf_pacer`, which user space uses to create qdisc instances.
### User-Space Loader
`egress_pacer.c` parses command-line arguments, configures BPF constants, creates the qdisc, waits for the specified duration, then cleans up.
```c
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <errno.h>
#include <getopt.h>
#include <linux/pkt_sched.h>
#include <net/if.h>
#include <signal.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <unistd.h>
#include <bpf/libbpf.h>
#include "egress_pacer.h"
#include "egress_pacer.skel.h"
static volatile sig_atomic_t exiting;
static struct env {
const char *interface;
unsigned long long rate_kbps;
unsigned int queue_limit;
unsigned int duration;
bool verbose;
} env = {
.rate_kbps = 1024,
.queue_limit = 256,
.duration = 10,
};
static void handle_signal(int signal)
{
(void)signal;
exiting = 1;
}
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s --interface IFACE [--rate-kbps KBPS] "
"[--queue-limit PACKETS] [--duration SEC] [--verbose]\n\n"
"Temporarily pace one interface with a bounded BPF qdisc.\n\n"
"Options:\n"
" -i, --interface IFACE interface to control (required)\n"
" -r, --rate-kbps KBPS egress rate, 8-100000000 "
"(default: 1024)\n"
" -q, --queue-limit PACKETS queue capacity, 1-65535 "
"(default: 256)\n"
" -d, --duration SEC control window, 1-86400 "
"(default: 10)\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_u64(const char *value, unsigned long long maximum,
unsigned long long *result)
{
char *end = NULL;
unsigned long long parsed;
errno = 0;
parsed = strtoull(value, &end, 10);
if (errno || end == value || *end || parsed > maximum)
return -EINVAL;
*result = parsed;
return 0;
}
static int parse_rate(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 100000000, &parsed) || parsed < 8) {
fprintf(stderr, "invalid rate in Kbit/s: %s\n", value);
return -EINVAL;
}
env.rate_kbps = parsed;
return 0;
}
static int parse_queue_limit(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 65535, &parsed) || parsed == 0) {
fprintf(stderr, "invalid queue limit: %s\n", value);
return -EINVAL;
}
env.queue_limit = parsed;
return 0;
}
static int parse_duration(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 86400, &parsed) || parsed == 0) {
fprintf(stderr, "invalid duration in seconds: %s\n", value);
return -EINVAL;
}
env.duration = parsed;
return 0;
}
static int parse_option(int option, const char *program)
{
switch (option) {
case 'i':
env.interface = optarg;
return 0;
case 'r':
return parse_rate(optarg);
case 'q':
return parse_queue_limit(optarg);
case 'd':
return parse_duration(optarg);
case 'v':
env.verbose = true;
return 0;
case 'h':
usage(program);
exit(0);
default:
return -EINVAL;
}
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "interface", required_argument, NULL, 'i' },
{ "rate-kbps", required_argument, NULL, 'r' },
{ "queue-limit", required_argument, NULL, 'q' },
{ "duration", required_argument, NULL, 'd' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
int error, option;
while ((option = getopt_long(argc, argv, "i:r:q:d:vh", options, NULL)) != -1) {
error = parse_option(option, argv[0]);
if (error)
return error;
}
if (!env.interface) {
fprintf(stderr, "--interface is required\n");
return -EINVAL;
}
if (optind != argc)
return -EINVAL;
return 0;
}
static long long monotonic_milliseconds(void)
{
struct timespec timestamp;
if (clock_gettime(CLOCK_MONOTONIC, &timestamp))
return -errno;
return timestamp.tv_sec * 1000LL + timestamp.tv_nsec / 1000000;
}
static int wait_for_duration(void)
{
struct timespec interval = { .tv_nsec = 100000000 };
long long deadline, now;
now = monotonic_milliseconds();
if (now < 0)
return (int)now;
deadline = now + env.duration * 1000LL;
while (!exiting) {
now = monotonic_milliseconds();
if (now < 0)
return (int)now;
if (now >= deadline)
break;
if (nanosleep(&interval, NULL) && errno != EINTR)
return -errno;
}
return 0;
}
int main(int argc, char **argv)
{
struct egress_pacer_bpf *skel = NULL;
struct pacer_stats final_stats = {};
struct bpf_tc_hook hook = {
.sz = sizeof(hook),
.attach_point = BPF_TC_QDISC,
.parent = TC_H_ROOT,
.handle = TC_H_MAKE(1 << 16, 0),
.qdisc = "bpf_pacer",
};
bool qdisc_created = false;
unsigned int ifindex;
int cleanup_err;
int err;
err = parse_args(argc, argv);
if (err) {
usage(argv[0]);
return 1;
}
ifindex = if_nametoindex(env.interface);
if (!ifindex) {
fprintf(stderr, "interface does not exist: %s\n", env.interface);
return 1;
}
hook.ifindex = ifindex;
libbpf_set_print(libbpf_print_fn);
signal(SIGINT, handle_signal);
signal(SIGTERM, handle_signal);
skel = egress_pacer_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return 1;
}
skel->rodata->rate_kbps = env.rate_kbps;
skel->rodata->queue_limit = env.queue_limit;
err = egress_pacer_bpf__load(skel);
if (err) {
fprintf(stderr,
"failed to load BPF qdisc: %s\n"
"This tool requires Linux 6.16+, CONFIG_NET_SCH_BPF, "
"BTF, and BPF JIT.\n",
strerror(-err));
goto cleanup;
}
err = egress_pacer_bpf__attach(skel);
if (err) {
fprintf(stderr, "failed to register bpf_pacer qdisc: %s\n",
strerror(-err));
goto cleanup;
}
err = bpf_tc_hook_create(&hook);
if (err) {
if (err == -EEXIST)
fprintf(stderr,
"refusing to replace the existing root qdisc on %s\n",
env.interface);
else
fprintf(stderr, "failed to attach bpf_pacer to %s: %s\n",
env.interface, strerror(-err));
goto cleanup;
}
qdisc_created = true;
printf("READY interface=%s rate_kbps=%llu queue_limit=%u duration=%u\n",
env.interface, env.rate_kbps, env.queue_limit, env.duration);
fflush(stdout);
err = wait_for_duration();
cleanup:
if (qdisc_created) {
cleanup_err = bpf_tc_hook_destroy(&hook);
if (cleanup_err) {
fprintf(stderr, "failed to remove bpf_pacer from %s: %s\n",
env.interface, strerror(-cleanup_err));
if (!err)
err = cleanup_err;
}
}
if (skel && skel->bss)
final_stats = skel->bss->stats;
if (qdisc_created) {
printf("SUMMARY enqueued=%llu dequeued=%llu policy_dropped=%llu "
"cleanup_dropped=%llu bytes_dequeued=%llu max_qlen=%llu\n",
final_stats.enqueued, final_stats.dequeued,
final_stats.policy_dropped, final_stats.cleanup_dropped,
final_stats.bytes_dequeued, final_stats.max_qlen);
}
egress_pacer_bpf__destroy(skel);
return err != 0;
}
```
The loader follows a standard flow. After parsing command-line arguments, it uses `if_nametoindex` to verify the interface exists. After opening the skeleton, it writes the rate and queue limit to the `.rodata` section, then calls `load` to complete BPF program loading. After successful loading, it calls `attach` to register the `struct_ops` implementation, making `bpf_pacer` an available qdisc type.
The subsequent `bpf_tc_hook_create` creates a root qdisc named `bpf_pacer` at the `TC_H_ROOT` position on the target interface. This call has exclusive semantics: if the target interface already has a root qdisc, the kernel returns `-EEXIST` and the program stops immediately, preserving the existing configuration. This exclusive approach limits the tool's scope, making it suitable for use on dedicated test interfaces.
After successful installation, the program prints the `READY` line, then checks the runtime and `exiting` flag every 100 ms. SIGINT, SIGTERM, or timeout all exit the wait loop. The cleanup order ensures `SUMMARY` includes packets cleared during reset: first call `bpf_tc_hook_destroy` to trigger reset, then read `skel->bss->stats`, and finally destroy the skeleton to unregister the `struct_ops` link.
## Compilation and Execution
Build from source:
```bash
cd src/53-egress-pacer
make clean
make -j2
```
For the first run, you can use the integration test directly. It creates a disposable `epac_tx`/`epac_rx` veth pair and deletes it after testing. Loading BPF and modifying qdisc require root:
```bash
sudo python3 tests/test_egress_pacer.py ./egress_pacer
```
Here is a completed run. The environment was x86_64, with kernel `7.0.0-rc2+` built from commit `a03114efd0720dff230388f7e160e427e54ea31b`:
```console
READY interface=epac_tx rate_kbps=64 queue_limit=8 duration=2
SUMMARY enqueued=9 dequeued=9 policy_dropped=31 cleanup_dropped=0 bytes_dequeued=9216 max_qlen=8
TEST-SUMMARY attempts=40 received=9 send_errors=31 span_ms=1024 observed_kbps=64.0
PASS: conflict refusal, bounded drops, pacing, accounting, normal/signal cleanup, and SIGKILL recovery succeeded
```
This output demonstrates functional correctness of the main paths: queue limits, rate limiting schedule, and counter consistency. The test sends 40 1024-byte raw EtherType `0x88B5` frames continuously into a 64 Kbit/s queue that can hold 8 packets. In this run, the receiver saw 9 packets with 1024 ms between first and last; another 31 sends went to `policy_dropped`. The exact numbers will vary depending on the system and timing.
The test also covers conflict detection, signal cleanup, and abnormal recovery. It pre-installs a conflicting `pfifo` to confirm the loader preserves existing configuration; creates a backlog then sends SIGTERM, checking that `cleanup_dropped` is greater than zero and verifying `enqueued = dequeued + cleanup_dropped`. The final SIGKILL is handled by the test program: it uses `tc` to delete the residual qdisc, confirms the interface recovers normally, and cleans up the veth pair in a Python `finally` block.
For manual runs, choose an interface you fully control and first check its current root qdisc:
```bash
tc qdisc show dev veth-service
```
After confirming the interface is safe to use, run the pacer for 30 seconds:
```bash
sudo ./egress_pacer --interface veth-service --rate-kbps 64000 --queue-limit 256 --duration 30
```
Command-line arguments:
```text
Usage: ./egress_pacer --interface IFACE [--rate-kbps KBPS] [--queue-limit PACKETS] [--duration SEC] [--verbose]
Options:
-i, --interface IFACE target interface (required)
-r, --rate-kbps KBPS egress rate, 8-100000000 (default: 1024)
-q, --queue-limit PACKETS queue capacity, 1-65535 packets (default: 256)
-d, --duration SEC control duration, 1-86400 seconds (default: 10)
-v, --verbose print libbpf diagnostics
-h, --help show help
```
Seeing `READY` means the qdisc is installed. After the run completes and removal succeeds, the program prints `SUMMARY`.
### Environment Requirements
This feature is available starting with Linux 6.16. The kernel must have BTF and BPF JIT enabled, with at least `CONFIG_BPF=y`, `CONFIG_BPF_SYSCALL=y`, `CONFIG_BPF_JIT=y`, `CONFIG_DEBUG_INFO_BTF=y`, `CONFIG_NET_SCHED=y`, and `CONFIG_NET_SCH_BPF=y`.
BPF qdisc TC hook requires libbpf 1.6.0 or newer. The repository includes libbpf and bpftool in `src/third_party`. Root privileges are required at runtime. The integration test additionally requires Python 3, raw packet sockets, and `ip` and `tc` from `iproute2`. A controlled veth pair is sufficient; the tested architecture is x86_64.
| Requirement | Minimum Version / Config |
| --- | --- |
| Linux kernel | 6.16+ |
| `CONFIG_BPF` | y |
| `CONFIG_BPF_SYSCALL` | y |
| `CONFIG_BPF_JIT` | y |
| `CONFIG_DEBUG_INFO_BTF` | y |
| `CONFIG_NET_SCHED` | y |
| `CONFIG_NET_SCH_BPF` | y |
| libbpf | 1.6.0+ |
| Privileges | root |
## Summary
This tutorial demonstrated how to implement a complete BPF qdisc using `struct_ops`. `enqueue` takes ownership of the skb and schedules transmission time based on packet length, `dequeue` uses the watchdog to wake up at the right moment and return the skb, and `reset` releases packets still in the queue during removal. These three ownership and timing stages form a reusable qdisc pattern.
This scheduler implements a single aggregate FIFO, with queue limits counted by packet count and transmission intervals derived from qdisc packet length. Rate and queue limit are fixed at load time, making it suitable for validating functional correctness on controlled interfaces. A more complete scheduler could add fairness, dynamic policies, and persistent recovery on top of this foundation.
> To learn more about eBPF, check out our tutorial repository at <https://github.com/eunomia-bpf/bpf-developer-tutorial> or visit our website at <https://eunomia.dev/tutorials/>.
## References
- [Linux BPF qdisc merge commit](https://github.com/torvalds/linux/commit/c8240344956e3f0b4e8f1d40ec3435e47040cacb)
- [Qdisc watchdog follow-up commit](https://github.com/torvalds/linux/commit/7a2dafda950b78611dc441c83d105dfdc7082681)
- [libbpf qdisc TC hook support](https://github.com/libbpf/libbpf/commit/f580871b429c550edf910a1b0d700510245351df)
- [libbpf 1.6.0 release](https://github.com/libbpf/libbpf/releases/tag/v1.6.0)
- [Upstream BPF FIFO qdisc selftest](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/progs/bpf_qdisc_fifo.c)
- [Upstream BPF qdisc test runner](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c)
- [libbpf 1.7.0 release](https://github.com/libbpf/libbpf/releases/tag/v1.7.0)

View File

@@ -0,0 +1,709 @@
# eBPF 教程:用 BPF Qdisc 实现出口限速
假设你需要在测试环境中验证应用程序在低带宽条件下的行为。你创建了一对 veth 接口模拟网络链路,现在想把出口速率限制到 64 Kbit/s,同时观察队列深度达到上限时的丢包行为。传统的限速方案需要复杂的 tc 配置或用户态代理,而 Linux 6.16 引入的 BPF qdisc 允许你用 eBPF 程序直接实现完整的排队规则。
本教程展示如何使用 `struct_ops` 实现一个 FIFO 限速器。程序注册为 root qdisc,自己管理 skb 入队和出队,按报文长度计算发送时间,使用 watchdog 调度下一次出队,并在 qdisc 移除时清理队列中的报文。这是一个完整的 qdisc 生命周期示例,适合在 veth、TAP 或 IFB 这类受控接口上验证 skb 所有权、发送时序和资源清理。
> 完整源码:<https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/53-egress-pacer>
## 为什么需要 BPF Qdisc
传统的流量整形方案有明显的局限性。
tc 提供了丰富的 qdisc 选项,如 TBF(Token Bucket Filter)和 HTB(Hierarchical Token Bucket),但这些都是内核中预定义的算法。如果你需要一种特殊的调度策略,比如按应用程序类型区分优先级,或者根据实时指标动态调整速率,就必须修改内核代码或使用复杂的 tc 分类器组合。
普通的 TC BPF 程序可以检查和处理报文,但排队和发送时间仍由原有的 qdisc 管理。[第 20 课](https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/20-tc) 展示的就是这种模式:BPF 程序决定放行或丢弃,但无法控制报文何时发送。如果你想实现自定义的排队策略,TC BPF 的能力是不够的。
用户态限速方案如 tc-netem 配合外部工具或 DPDK 可以实现复杂的流量整形,但引入了额外的上下文切换和复杂的部署依赖。在测试环境中,你可能只想快速验证一个想法,而不是搭建完整的用户态网络栈。
Linux 6.16 通过 `struct_ops` 引入了 BPF 实现的 `Qdisc_ops` 回调。程序实现 `enqueue`、`dequeue`、`init`、`reset` 和 `destroy`,可以动态注册为一个 qdisc。这意味着你可以用 BPF 程序完全控制报文的排队和发送时序,同时保持内核网络栈的性能优势。本课的 `egress_pacer` 就是用这种方式构建的 FIFO 限速器。
## BPF Qdisc 的工作原理
在深入代码之前,先理解 BPF qdisc 的整体流程。与普通 TC BPF 程序不同,qdisc BPF 程序注册为 root qdisc,完全接管报文的排队和发送。
用户态程序首先设置速率和队列上限,加载 BPF 程序并注册 `struct_ops` 实现,然后把它挂载到指定接口的 `TC_H_ROOT` 位置。此后所有经过该接口的出口报文都由 BPF 程序处理。
当内核需要发送报文时,它调用 `enqueue` 回调。BPF 程序创建一个节点来保存 skb,根据报文长度计算最早可以发送的时间,然后把节点加入 FIFO 队列。如果队列已满或节点分配失败,程序丢弃报文并更新统计。
当内核尝试取出报文发送时,它调用 `dequeue` 回调。BPF 程序检查队首节点的发送时间。如果当前时间已经到达或超过该时间,程序取出 skb 返回给内核;如果还没到发送时间,程序用 `bpf_qdisc_watchdog_schedule` 设置一个定时器,然后返回 NULL。定时器到期后,内核会再次调用 `dequeue`。
当用户态程序移除 qdisc 时,内核调用 `reset` 回调。BPF 程序遍历队列中所有未发送的报文,释放它们占用的资源,并清零计数器。
整个生命周期中,BPF 程序完全控制 skb 的所有权。从 `enqueue` 接收 skb 开始,到 `dequeue` 返回 skb 或 `reset` 释放 skb 为止,报文都由 BPF 管理的数据结构保存。
![egress_pacer 数据流:从配置、挂载、入队、BPF FIFO、出队到发送路径,包含 policy-drop 分支、watchdog 循环和 reset 生命周期](https://github.com/eunomia-bpf/bpf-developer-tutorial/raw/main/src/53-egress-pacer/egress-pacer-flow.png)
图中实线从配置和挂载开始,随后跟踪一个 skb 从 enqueue 进入 BPF FIFO,再由 dequeue 交还给发送路径。队列已满或节点分配返回空时走 policy-drop 分支。早到的 dequeue 把节点推回队首,安排 watchdog 并返回 NULL,等 watchdog 唤醒后重新进入 dequeue。虚线表示生命周期路径:移除 root qdisc 后 reset 释放队列中的 skb 并清零 qlen 和 backlog。
## 代码实现
本工具由四个文件组成:共享头文件定义统计结构、兼容头文件提供 graph-object kfunc 声明、BPF 程序实现 qdisc 回调、用户空间加载器管理生命周期并打印结果。
### 共享头文件
`egress_pacer.h` 定义了 BPF 和用户空间共享的统计结构。这六个字段把报文的一生分成几种结果,位于 BSS 段作为结果通道。
```c
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __EGRESS_PACER_H
#define __EGRESS_PACER_H
struct pacer_stats {
unsigned long long enqueued;
unsigned long long dequeued;
unsigned long long policy_dropped;
unsigned long long cleanup_dropped;
unsigned long long bytes_dequeued;
unsigned long long max_qlen;
};
#endif /* __EGRESS_PACER_H */
```
成功进入和离开 FIFO 的报文分别计入 `enqueued` 和 `dequeued`。队列已满或节点分配失败时,`policy_dropped` 增加。reset 时仍在队列中的报文计入 `cleanup_dropped`。`bytes_dequeued` 累计实际交还给发送路径的字节数,`max_qlen` 记录队列的峰值深度。
### 兼容头文件
这个示例需要 graph-object kfunc,因此用一个本地兼容头文件补齐 BPF qdisc 声明。这些声明仅存在于本课目录,与仓库统一使用的头文件相互独立。
```c
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __EGRESS_PACER_BPF_EXPERIMENTAL_H
#define __EGRESS_PACER_BPF_EXPERIMENTAL_H
#include <bpf/bpf_core_read.h>
#define __contains(name, node) \
__attribute__((btf_decl_tag("contains:" #name ":" #node)))
extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym;
#define bpf_obj_new(type) \
((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL))
extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym;
#define bpf_obj_drop(kptr) bpf_obj_drop_impl(kptr, NULL)
extern int bpf_list_push_front_impl(struct bpf_list_head *head,
struct bpf_list_node *node,
void *meta, __u64 off) __ksym;
#define bpf_list_push_front(head, node) \
bpf_list_push_front_impl(head, node, NULL, 0)
extern int bpf_list_push_back_impl(struct bpf_list_head *head,
struct bpf_list_node *node,
void *meta, __u64 off) __ksym;
#define bpf_list_push_back(head, node) \
bpf_list_push_back_impl(head, node, NULL, 0)
extern struct bpf_list_node *
bpf_list_pop_front(struct bpf_list_head *head) __ksym;
#endif /* __EGRESS_PACER_BPF_EXPERIMENTAL_H */
```
这个文件提供构建队列所需的 graph-object 辅助函数。`bpf_obj_new` 创建报文节点,`bpf_obj_drop` 在节点用完后释放它。`bpf_list_push_back` 和 `bpf_list_pop_front` 让这些节点组成 FIFO。`__contains` 生成 BTF tag,告诉 verifier 这个 list head 装的是哪一种节点,使 verifier 可以安全地跟踪链表对象。
### BPF 程序
`egress_pacer.bpf.c` 使用 `SEC(".struct_ops")` 声明 `Qdisc_ops` 实现。内核在加载时注册这些回调,使 BPF 程序成为一个完整的 qdisc。
```c
// SPDX-License-Identifier: GPL-2.0
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include "bpf_experimental.h"
#include "egress_pacer.h"
char LICENSE[] SEC("license") = "GPL";
#define NET_XMIT_SUCCESS 0x00
#define NET_XMIT_DROP 0x01
#define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8)))
struct bpf_sk_buff_ptr {
struct sk_buff *skb;
};
extern void bpf_qdisc_skb_drop(struct sk_buff *skb,
struct bpf_sk_buff_ptr *to_free) __ksym;
extern void bpf_qdisc_watchdog_schedule(struct Qdisc *sch, __u64 expire,
__u64 delta_ns) __ksym;
extern void bpf_qdisc_bstats_update(struct Qdisc *sch,
const struct sk_buff *skb) __ksym;
extern void bpf_kfree_skb(struct sk_buff *skb) __ksym;
const volatile __u64 rate_kbps = 1024;
const volatile __u32 queue_limit = 256;
struct pacer_stats stats;
struct packet_node {
__u64 eligible_ns;
__u32 packet_len;
struct sk_buff __kptr *skb;
struct bpf_list_node node;
};
private(A) struct bpf_spin_lock queue_lock;
private(A) struct bpf_list_head packet_queue __contains(packet_node, node);
private(A) __u64 next_departure_ns;
static struct qdisc_skb_cb *qdisc_skb_cb(const struct sk_buff *skb)
{
return (struct qdisc_skb_cb *)skb->cb;
}
static __u32 qdisc_packet_len(const struct sk_buff *skb)
{
return qdisc_skb_cb(skb)->pkt_len;
}
SEC("struct_ops/egress_pacer_enqueue")
int BPF_PROG(egress_pacer_enqueue, struct sk_buff *skb, struct Qdisc *sch,
struct bpf_sk_buff_ptr *to_free)
{
struct packet_node *packet;
__u64 eligible_ns, interval_ns, now;
__u32 packet_len;
packet_len = qdisc_packet_len(skb);
if (sch->q.qlen >= sch->limit)
goto drop;
packet = bpf_obj_new(typeof(*packet));
if (!packet)
goto drop;
now = bpf_ktime_get_ns();
interval_ns = (__u64)packet_len * 8 * 1000000ULL / rate_kbps;
if (!interval_ns)
interval_ns = 1;
packet->packet_len = packet_len;
skb = bpf_kptr_xchg(&packet->skb, skb);
if (skb)
bpf_qdisc_skb_drop(skb, to_free);
bpf_spin_lock(&queue_lock);
eligible_ns = next_departure_ns > now ? next_departure_ns : now;
next_departure_ns = eligible_ns + interval_ns;
packet->eligible_ns = eligible_ns;
bpf_list_push_back(&packet_queue, &packet->node);
sch->q.qlen++;
sch->qstats.backlog += packet_len;
__sync_fetch_and_add(&stats.enqueued, 1);
if (sch->q.qlen > stats.max_qlen)
stats.max_qlen = sch->q.qlen;
bpf_spin_unlock(&queue_lock);
return NET_XMIT_SUCCESS;
drop:
bpf_qdisc_skb_drop(skb, to_free);
__sync_fetch_and_add(&stats.policy_dropped, 1);
return NET_XMIT_DROP;
}
SEC("struct_ops/egress_pacer_dequeue")
struct sk_buff *BPF_PROG(egress_pacer_dequeue, struct Qdisc *sch)
{
struct bpf_list_node *node;
struct packet_node *packet;
struct sk_buff *skb = NULL;
__u64 expire, now;
__u32 packet_len;
bpf_spin_lock(&queue_lock);
node = bpf_list_pop_front(&packet_queue);
bpf_spin_unlock(&queue_lock);
if (!node) {
return NULL;
}
packet = container_of(node, struct packet_node, node);
now = bpf_ktime_get_ns();
if (now < packet->eligible_ns) {
expire = packet->eligible_ns;
bpf_spin_lock(&queue_lock);
bpf_list_push_front(&packet_queue, &packet->node);
bpf_spin_unlock(&queue_lock);
bpf_qdisc_watchdog_schedule(sch, expire, 0);
return NULL;
}
packet_len = packet->packet_len;
skb = bpf_kptr_xchg(&packet->skb, skb);
bpf_spin_lock(&queue_lock);
sch->q.qlen--;
sch->qstats.backlog -= packet_len;
bpf_spin_unlock(&queue_lock);
bpf_obj_drop(packet);
if (!skb)
return NULL;
bpf_qdisc_bstats_update(sch, skb);
__sync_fetch_and_add(&stats.dequeued, 1);
__sync_fetch_and_add(&stats.bytes_dequeued, packet_len);
return skb;
}
SEC("struct_ops/egress_pacer_init")
int BPF_PROG(egress_pacer_init, struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
(void)opt;
(void)extack;
sch->limit = queue_limit;
return 0;
}
SEC("struct_ops/egress_pacer_reset")
void BPF_PROG(egress_pacer_reset, struct Qdisc *sch)
{
int queued = sch->q.qlen;
int i;
bpf_for(i, 0, queued) {
struct bpf_list_node *node;
struct packet_node *packet;
struct sk_buff *skb = NULL;
bpf_spin_lock(&queue_lock);
node = bpf_list_pop_front(&packet_queue);
bpf_spin_unlock(&queue_lock);
if (!node)
break;
packet = container_of(node, struct packet_node, node);
skb = bpf_kptr_xchg(&packet->skb, skb);
if (skb) {
bpf_kfree_skb(skb);
__sync_fetch_and_add(&stats.cleanup_dropped, 1);
}
bpf_obj_drop(packet);
}
bpf_spin_lock(&queue_lock);
next_departure_ns = 0;
sch->q.qlen = 0;
sch->qstats.backlog = 0;
bpf_spin_unlock(&queue_lock);
}
SEC("struct_ops/egress_pacer_destroy")
void BPF_PROG(egress_pacer_destroy, struct Qdisc *sch)
{
(void)sch;
}
SEC(".struct_ops")
struct Qdisc_ops pacer = {
.enqueue = (void *)egress_pacer_enqueue,
.dequeue = (void *)egress_pacer_dequeue,
.init = (void *)egress_pacer_init,
.reset = (void *)egress_pacer_reset,
.destroy = (void *)egress_pacer_destroy,
.id = "bpf_pacer",
};
```
程序结构围绕 qdisc 的三个核心阶段展开。两个 `const volatile` 变量 `rate_kbps` 和 `queue_limit` 位于 `.rodata` 段,用户态在 `open()` 之后、`load()` 之前写入速率和队列上限,验证器将它们视为编译期常量。`packet_node` 结构体定义了队列中的节点,包含发送时间、报文长度、skb 指针和链表节点。`__kptr` 标记告诉 verifier 这个字段持有内核指针的所有权。
`egress_pacer_enqueue` 是入队回调。它首先从 `qdisc_skb_cb(skb)->pkt_len` 取得 qdisc 看到的报文长度,然后检查队列是否还有空间。如果队列已满或节点创建失败,程序调用 `bpf_qdisc_skb_drop` 丢弃报文并递增 `policy_dropped`。真正的所有权转移发生在 `bpf_kptr_xchg`:调用成功后,skb 被放进节点的 `__kptr` 字段,节点从此负责保管它。程序随后在持有 `queue_lock` 的情况下计算发送时间、把节点加入 FIFO 尾部、更新 qlen 和 backlog。发送间隔按 `packet_len * 8 * 1000000 / rate_kbps` 计算,单位是纳秒。队列空闲一段时间后,`next_departure_ns` 会早于当前时间,新来的第一个报文可以立刻离开。
`egress_pacer_dequeue` 是出队回调。它从队首取出节点,比较 `eligible_ns` 与当前时间。如果报文已经可以发送,程序用 `bpf_kptr_xchg` 把 skb 从节点取回,扣减 qlen 和 backlog,释放节点,然后调用 `bpf_qdisc_bstats_update` 更新 qdisc 统计并返回 skb。如果还没到发送时间,程序把节点放回队首,调用 `bpf_qdisc_watchdog_schedule` 设置定时器,然后返回 NULL。内核会在指定时间再次调用 dequeue。
`egress_pacer_reset` 负责清理。当 qdisc 被移除时,队列中可能还有未发送的报文。程序用 `bpf_for` 逐个取出节点,调用 `bpf_kfree_skb` 释放 skb,然后销毁节点对象。清掉的报文计入 `cleanup_dropped`,与 enqueue 阶段的 `policy_dropped` 分开。链表排空后,程序把 `next_departure_ns`、qlen 和 backlog 一起归零。
程序末尾的 `SEC(".struct_ops")` 块把所有回调注册为 `Qdisc_ops` 结构体。`id` 字段指定 qdisc 名称为 `bpf_pacer`,用户态通过这个名称创建 qdisc 实例。
### 用户空间加载器
`egress_pacer.c` 解析命令行参数、配置 BPF 常量、创建 qdisc 并等待指定时间后清理。
```c
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <errno.h>
#include <getopt.h>
#include <linux/pkt_sched.h>
#include <net/if.h>
#include <signal.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <unistd.h>
#include <bpf/libbpf.h>
#include "egress_pacer.h"
#include "egress_pacer.skel.h"
static volatile sig_atomic_t exiting;
static struct env {
const char *interface;
unsigned long long rate_kbps;
unsigned int queue_limit;
unsigned int duration;
bool verbose;
} env = {
.rate_kbps = 1024,
.queue_limit = 256,
.duration = 10,
};
static void handle_signal(int signal)
{
(void)signal;
exiting = 1;
}
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s --interface IFACE [--rate-kbps KBPS] "
"[--queue-limit PACKETS] [--duration SEC] [--verbose]\n\n"
"Temporarily pace one interface with a bounded BPF qdisc.\n\n"
"Options:\n"
" -i, --interface IFACE interface to control (required)\n"
" -r, --rate-kbps KBPS egress rate, 8-100000000 "
"(default: 1024)\n"
" -q, --queue-limit PACKETS queue capacity, 1-65535 "
"(default: 256)\n"
" -d, --duration SEC control window, 1-86400 "
"(default: 10)\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_u64(const char *value, unsigned long long maximum,
unsigned long long *result)
{
char *end = NULL;
unsigned long long parsed;
errno = 0;
parsed = strtoull(value, &end, 10);
if (errno || end == value || *end || parsed > maximum)
return -EINVAL;
*result = parsed;
return 0;
}
static int parse_rate(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 100000000, &parsed) || parsed < 8) {
fprintf(stderr, "invalid rate in Kbit/s: %s\n", value);
return -EINVAL;
}
env.rate_kbps = parsed;
return 0;
}
static int parse_queue_limit(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 65535, &parsed) || parsed == 0) {
fprintf(stderr, "invalid queue limit: %s\n", value);
return -EINVAL;
}
env.queue_limit = parsed;
return 0;
}
static int parse_duration(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 86400, &parsed) || parsed == 0) {
fprintf(stderr, "invalid duration in seconds: %s\n", value);
return -EINVAL;
}
env.duration = parsed;
return 0;
}
static int parse_option(int option, const char *program)
{
switch (option) {
case 'i':
env.interface = optarg;
return 0;
case 'r':
return parse_rate(optarg);
case 'q':
return parse_queue_limit(optarg);
case 'd':
return parse_duration(optarg);
case 'v':
env.verbose = true;
return 0;
case 'h':
usage(program);
exit(0);
default:
return -EINVAL;
}
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "interface", required_argument, NULL, 'i' },
{ "rate-kbps", required_argument, NULL, 'r' },
{ "queue-limit", required_argument, NULL, 'q' },
{ "duration", required_argument, NULL, 'd' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
int error, option;
while ((option = getopt_long(argc, argv, "i:r:q:d:vh", options, NULL)) != -1) {
error = parse_option(option, argv[0]);
if (error)
return error;
}
if (!env.interface) {
fprintf(stderr, "--interface is required\n");
return -EINVAL;
}
if (optind != argc)
return -EINVAL;
return 0;
}
static long long monotonic_milliseconds(void)
{
struct timespec timestamp;
if (clock_gettime(CLOCK_MONOTONIC, &timestamp))
return -errno;
return timestamp.tv_sec * 1000LL + timestamp.tv_nsec / 1000000;
}
static int wait_for_duration(void)
{
struct timespec interval = { .tv_nsec = 100000000 };
long long deadline, now;
now = monotonic_milliseconds();
if (now < 0)
return (int)now;
deadline = now + env.duration * 1000LL;
while (!exiting) {
now = monotonic_milliseconds();
if (now < 0)
return (int)now;
if (now >= deadline)
break;
if (nanosleep(&interval, NULL) && errno != EINTR)
return -errno;
}
return 0;
}
int main(int argc, char **argv)
{
struct egress_pacer_bpf *skel = NULL;
struct pacer_stats final_stats = {};
struct bpf_tc_hook hook = {
.sz = sizeof(hook),
.attach_point = BPF_TC_QDISC,
.parent = TC_H_ROOT,
.handle = TC_H_MAKE(1 << 16, 0),
.qdisc = "bpf_pacer",
};
bool qdisc_created = false;
unsigned int ifindex;
int cleanup_err;
int err;
err = parse_args(argc, argv);
if (err) {
usage(argv[0]);
return 1;
}
ifindex = if_nametoindex(env.interface);
if (!ifindex) {
fprintf(stderr, "interface does not exist: %s\n", env.interface);
return 1;
}
hook.ifindex = ifindex;
libbpf_set_print(libbpf_print_fn);
signal(SIGINT, handle_signal);
signal(SIGTERM, handle_signal);
skel = egress_pacer_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return 1;
}
skel->rodata->rate_kbps = env.rate_kbps;
skel->rodata->queue_limit = env.queue_limit;
err = egress_pacer_bpf__load(skel);
if (err) {
fprintf(stderr,
"failed to load BPF qdisc: %s\n"
"This tool requires Linux 6.16+, CONFIG_NET_SCH_BPF, "
"BTF, and BPF JIT.\n",
strerror(-err));
goto cleanup;
}
err = egress_pacer_bpf__attach(skel);
if (err) {
fprintf(stderr, "failed to register bpf_pacer qdisc: %s\n",
strerror(-err));
goto cleanup;
}
err = bpf_tc_hook_create(&hook);
if (err) {
if (err == -EEXIST)
fprintf(stderr,
"refusing to replace the existing root qdisc on %s\n",
env.interface);
else
fprintf(stderr, "failed to attach bpf_pacer to %s: %s\n",
env.interface, strerror(-err));
goto cleanup;
}
qdisc_created = true;
printf("READY interface=%s rate_kbps=%llu queue_limit=%u duration=%u\n",
env.interface, env.rate_kbps, env.queue_limit, env.duration);
fflush(stdout);
err = wait_for_duration();
cleanup:
if (qdisc_created) {
cleanup_err = bpf_tc_hook_destroy(&hook);
if (cleanup_err) {
fprintf(stderr, "failed to remove bpf_pacer from %s: %s\n",
env.interface, strerror(-cleanup_err));
if (!err)
err = cleanup_err;
}
}
if (skel && skel->bss)
final_stats = skel->bss->stats;
if (qdisc_created) {
printf("SUMMARY enqueued=%llu dequeued=%llu policy_dropped=%llu "
"cleanup_dropped=%llu bytes_dequeued=%llu max_qlen=%llu\n",
final_stats.enqueued, final_stats.dequeued,
final_stats.policy_dropped, final_stats.cleanup_dropped,
final_stats.bytes_dequeued, final_stats.max_qlen);
}
egress_pacer_bpf__destroy(skel);
return err != 0;
}
```
加载器的流程比较标准。解析命令行后,用 `if_nametoindex` 验证接口存在。打开 skeleton 后把速率和队列上限写入 `.rodata` 段,然后调用 `load` 完成 BPF 程序加载。加载成功后调用 `attach` 注册 `struct_ops` 实现,使 `bpf_pacer` 成为可用的 qdisc 类型。
接下来的 `bpf_tc_hook_create` 在目标接口的 `TC_H_ROOT` 位置创建名为 `bpf_pacer` 的 root qdisc。这个调用带着独占语义:目标接口已经有 root qdisc 时,内核返回 `-EEXIST`,程序立即停止,保留现有配置。这种独占做法限制了工具的作用范围,适合把它用在专门创建的测试接口上。
安装成功后,程序打印 `READY` 行,然后每隔 100 ms 检查运行时间和 `exiting` 标志。SIGINT、SIGTERM 或时间到期都会退出等待循环。清理顺序保证 `SUMMARY` 包含 reset 清掉的报文:先调用 `bpf_tc_hook_destroy` 触发 reset,再读取 `skel->bss->stats`,最后销毁 skeleton 并注销 `struct_ops` link。
## 编译与运行
从源码构建:
```bash
cd src/53-egress-pacer
make clean
make -j2
```
第一次运行可以直接使用集成测试程序,它会创建一次性的 `epac_tx`/`epac_rx` veth 对,完成测试后删除。加载 BPF 和修改 qdisc 需要 root:
```bash
sudo python3 tests/test_egress_pacer.py ./egress_pacer
```
下面是一次已经完成的运行。环境为 x86_64,内核是从 commit `a03114efd0720dff230388f7e160e427e54ea31b` 构建的 `7.0.0-rc2+`:
```console
READY interface=epac_tx rate_kbps=64 queue_limit=8 duration=2
SUMMARY enqueued=9 dequeued=9 policy_dropped=31 cleanup_dropped=0 bytes_dequeued=9216 max_qlen=8
TEST-SUMMARY attempts=40 received=9 send_errors=31 span_ms=1024 observed_kbps=64.0
PASS: conflict refusal, bounded drops, pacing, accounting, normal/signal cleanup, and SIGKILL recovery succeeded
```
这段输出展示主要路径的功能正确性:队列上限、限速调度和计数一致。测试向一个 64 Kbit/s、最多容纳 8 个报文的队列连续发送 40 个 1024 字节 EtherType `0x88B5` 原始帧。本次运行中,接收端看到 9 个报文,首尾相隔 1024 ms;另外 31 次发送走到 `policy_dropped`。换一个系统或时序,这些具体数字会有所不同。
测试同时覆盖冲突检测、信号清理和异常恢复。它会预先安装一个冲突的 `pfifo`,确认加载器保留现有配置;制造 backlog 后发送 SIGTERM,检查 `cleanup_dropped` 大于零,并验证 `enqueued = dequeued + cleanup_dropped`。最后的 SIGKILL 由测试程序接管:用 `tc` 删除残留的 qdisc,确认接口恢复正常,并在 Python 的 `finally` 块中清掉 veth 对。
手动运行时,选择一个完全由你控制的接口,先查看它当前使用的 root qdisc:
```bash
tc qdisc show dev veth-service
```
确认接口可以安全使用后,让 pacer 持续 30 秒:
```bash
sudo ./egress_pacer --interface veth-service --rate-kbps 64000 --queue-limit 256 --duration 30
```
命令行参数:
```text
Usage: ./egress_pacer --interface IFACE [--rate-kbps KBPS] [--queue-limit PACKETS] [--duration SEC] [--verbose]
Options:
-i, --interface IFACE 目标接口(必选)
-r, --rate-kbps KBPS 出口速率,8-100000000(默认:1024)
-q, --queue-limit PACKETS 队列容量,1-65535 个报文(默认:256)
-d, --duration SEC 控制时长,1-86400 秒(默认:10)
-v, --verbose 打印 libbpf 诊断信息
-h, --help 显示帮助
```
看到 `READY` 就表示 qdisc 已经安装。运行结束并成功移除后,程序打印 `SUMMARY`。
### 环境要求
这个功能从 Linux 6.16 开始可用。内核需要启用 BTF 和 BPF JIT,并至少包含 `CONFIG_BPF=y`、`CONFIG_BPF_SYSCALL=y`、`CONFIG_BPF_JIT=y`、`CONFIG_DEBUG_INFO_BTF=y`、`CONFIG_NET_SCHED=y` 与 `CONFIG_NET_SCH_BPF=y`。
BPF qdisc TC hook 需要 libbpf 1.6.0 或更新版本。仓库已经在 `src/third_party` 中集成 libbpf 和 bpftool。运行时需要 root 权限。集成测试另外依赖 Python 3、原始报文套接字,以及 `iproute2` 中的 `ip` 和 `tc`。一个受控的 veth 对就足够了;本课验证的架构是 x86_64。
## 总结
本教程展示了如何使用 `struct_ops` 实现一个完整的 BPF qdisc。`enqueue` 接管 skb 并按报文长度安排发送时间,`dequeue` 用 watchdog 在适当时刻被唤醒并返回 skb,`reset` 释放移除时仍在队列里的报文。这三段所有权和时序处理构成可复用的 qdisc 模式。
这个调度器实现一条聚合 FIFO,队列上限按报文个数计算,发送间隔使用 qdisc 报文长度推导。速率和队列上限在加载时固定,适合在受控接口上验证功能正确性。更完整的调度器可以在此基础上加入公平性、动态策略和持久化恢复。
> 如果你想深入了解 eBPF,请查看我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或访问我们的网站 <https://eunomia.dev/tutorials/>。
## 参考资料
- [Linux BPF qdisc 合入 commit](https://github.com/torvalds/linux/commit/c8240344956e3f0b4e8f1d40ec3435e47040cacb)
- [Qdisc watchdog 后续 commit](https://github.com/torvalds/linux/commit/7a2dafda950b78611dc441c83d105dfdc7082681)
- [libbpf qdisc TC hook 支持](https://github.com/libbpf/libbpf/commit/f580871b429c550edf910a1b0d700510245351df)
- [libbpf 1.6.0 发布说明](https://github.com/libbpf/libbpf/releases/tag/v1.6.0)
- [上游 BPF FIFO qdisc selftest](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/progs/bpf_qdisc_fifo.c)
- [上游 BPF qdisc test runner](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c)
- [libbpf 1.7.0 发布说明](https://github.com/libbpf/libbpf/releases/tag/v1.7.0)

View File

@@ -0,0 +1,32 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __EGRESS_PACER_BPF_EXPERIMENTAL_H
#define __EGRESS_PACER_BPF_EXPERIMENTAL_H
#include <bpf/bpf_core_read.h>
#define __contains(name, node) \
__attribute__((btf_decl_tag("contains:" #name ":" #node)))
extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym;
#define bpf_obj_new(type) \
((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL))
extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym;
#define bpf_obj_drop(kptr) bpf_obj_drop_impl(kptr, NULL)
extern int bpf_list_push_front_impl(struct bpf_list_head *head,
struct bpf_list_node *node,
void *meta, __u64 off) __ksym;
#define bpf_list_push_front(head, node) \
bpf_list_push_front_impl(head, node, NULL, 0)
extern int bpf_list_push_back_impl(struct bpf_list_head *head,
struct bpf_list_node *node,
void *meta, __u64 off) __ksym;
#define bpf_list_push_back(head, node) \
bpf_list_push_back_impl(head, node, NULL, 0)
extern struct bpf_list_node *
bpf_list_pop_front(struct bpf_list_head *head) __ksym;
#endif /* __EGRESS_PACER_BPF_EXPERIMENTAL_H */

View File

@@ -0,0 +1,129 @@
digraph egress_pacer_flow {
graph [
rankdir=TB,
bgcolor="white",
pad="0.25",
nodesep="0.32",
ranksep="0.44",
splines=spline,
size="9,8",
ratio=compress,
newrank=true,
fontname="DejaVu Sans"
];
node [
shape=box,
style="rounded,filled",
fillcolor="#F7FAFC",
color="#4A5568",
fontcolor="#1A202C",
fontname="DejaVu Sans",
fontsize=12,
margin="0.16,0.10"
];
edge [
color="#4A5568",
fontcolor="#2D3748",
fontname="DejaVu Sans",
fontsize=10,
arrowsize=0.75
];
configure [
label="User space\nrate_kbps + queue_limit",
fillcolor="#EBF8FF",
color="#3182CE"
];
attach [
label="bpf_pacer\nTC_H_ROOT",
fillcolor="#EBF8FF",
color="#3182CE"
];
enqueue [
label="enqueue\nskb → packet_node\ncompute eligible_ns",
fillcolor="#E6FFFA",
color="#319795"
];
admitted [
label="queue has capacity\nand node is allocated?",
shape=diamond,
style="filled",
fillcolor="#FFFAF0",
color="#DD6B20",
margin="0.10,0.05"
];
policy_drop [
label="policy drop\nbpf_qdisc_skb_drop",
fillcolor="#FFF5F5",
color="#E53E3E"
];
fifo [
label="BPF FIFO\nqlen + backlog",
shape=cylinder,
fillcolor="#F0FFF4",
color="#38A169"
];
dequeue [
label="dequeue\npop queue head",
fillcolor="#E6FFFA",
color="#319795"
];
eligible [
label="now >= eligible_ns?",
shape=diamond,
style="filled",
fillcolor="#FFFAF0",
color="#DD6B20",
margin="0.10,0.05"
];
transmit [
label="return skb\nupdate bstats + counters",
fillcolor="#F0FFF4",
color="#38A169"
];
watchdog [
label="push_front\nwatchdog_schedule\nreturn NULL",
fillcolor="#FAF5FF",
color="#805AD5"
];
remove [
label="remove root qdisc",
fillcolor="#EBF8FF",
color="#3182CE"
];
reset [
label="reset\nkfree queued skb\nzero qlen + backlog",
fillcolor="#FFF5F5",
color="#C53030"
];
{ rank=same; attach; remove; }
{ rank=same; admitted; policy_drop; }
{ rank=same; fifo; reset; }
{ rank=same; eligible; transmit; }
configure -> attach [label="load + attach"];
attach -> enqueue [label="packet arrives"];
enqueue -> admitted;
admitted -> fifo [label="yes"];
admitted -> policy_drop [label="no"];
fifo -> dequeue;
dequeue -> eligible;
eligible -> transmit [label="yes"];
eligible -> watchdog [label="no"];
watchdog -> dequeue [label="watchdog wakeup", constraint=false];
attach -> remove [
label="control window ends",
style=dashed,
color="#718096",
constraint=false
];
remove -> reset [style=dashed, color="#718096"];
fifo -> reset [
label="queued skb",
style=dashed,
color="#718096",
constraint=false
];
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 159 KiB

View File

@@ -0,0 +1,197 @@
// SPDX-License-Identifier: GPL-2.0
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include "bpf_experimental.h"
#include "egress_pacer.h"
char LICENSE[] SEC("license") = "GPL";
#define NET_XMIT_SUCCESS 0x00
#define NET_XMIT_DROP 0x01
#define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8)))
struct bpf_sk_buff_ptr {
struct sk_buff *skb;
};
extern void bpf_qdisc_skb_drop(struct sk_buff *skb,
struct bpf_sk_buff_ptr *to_free) __ksym;
extern void bpf_qdisc_watchdog_schedule(struct Qdisc *sch, __u64 expire,
__u64 delta_ns) __ksym;
extern void bpf_qdisc_bstats_update(struct Qdisc *sch,
const struct sk_buff *skb) __ksym;
extern void bpf_kfree_skb(struct sk_buff *skb) __ksym;
const volatile __u64 rate_kbps = 1024;
const volatile __u32 queue_limit = 256;
struct pacer_stats stats;
struct packet_node {
__u64 eligible_ns;
__u32 packet_len;
struct sk_buff __kptr *skb;
struct bpf_list_node node;
};
private(A) struct bpf_spin_lock queue_lock;
private(A) struct bpf_list_head packet_queue __contains(packet_node, node);
private(A) __u64 next_departure_ns;
static struct qdisc_skb_cb *qdisc_skb_cb(const struct sk_buff *skb)
{
return (struct qdisc_skb_cb *)skb->cb;
}
static __u32 qdisc_packet_len(const struct sk_buff *skb)
{
return qdisc_skb_cb(skb)->pkt_len;
}
SEC("struct_ops/egress_pacer_enqueue")
int BPF_PROG(egress_pacer_enqueue, struct sk_buff *skb, struct Qdisc *sch,
struct bpf_sk_buff_ptr *to_free)
{
struct packet_node *packet;
__u64 eligible_ns, interval_ns, now;
__u32 packet_len;
packet_len = qdisc_packet_len(skb);
if (sch->q.qlen >= sch->limit)
goto drop;
packet = bpf_obj_new(typeof(*packet));
if (!packet)
goto drop;
now = bpf_ktime_get_ns();
interval_ns = (__u64)packet_len * 8 * 1000000ULL / rate_kbps;
if (!interval_ns)
interval_ns = 1;
packet->packet_len = packet_len;
skb = bpf_kptr_xchg(&packet->skb, skb);
if (skb)
bpf_qdisc_skb_drop(skb, to_free);
bpf_spin_lock(&queue_lock);
eligible_ns = next_departure_ns > now ? next_departure_ns : now;
next_departure_ns = eligible_ns + interval_ns;
packet->eligible_ns = eligible_ns;
bpf_list_push_back(&packet_queue, &packet->node);
sch->q.qlen++;
sch->qstats.backlog += packet_len;
__sync_fetch_and_add(&stats.enqueued, 1);
if (sch->q.qlen > stats.max_qlen)
stats.max_qlen = sch->q.qlen;
bpf_spin_unlock(&queue_lock);
return NET_XMIT_SUCCESS;
drop:
bpf_qdisc_skb_drop(skb, to_free);
__sync_fetch_and_add(&stats.policy_dropped, 1);
return NET_XMIT_DROP;
}
SEC("struct_ops/egress_pacer_dequeue")
struct sk_buff *BPF_PROG(egress_pacer_dequeue, struct Qdisc *sch)
{
struct bpf_list_node *node;
struct packet_node *packet;
struct sk_buff *skb = NULL;
__u64 expire, now;
__u32 packet_len;
bpf_spin_lock(&queue_lock);
node = bpf_list_pop_front(&packet_queue);
bpf_spin_unlock(&queue_lock);
if (!node) {
return NULL;
}
packet = container_of(node, struct packet_node, node);
now = bpf_ktime_get_ns();
if (now < packet->eligible_ns) {
expire = packet->eligible_ns;
bpf_spin_lock(&queue_lock);
bpf_list_push_front(&packet_queue, &packet->node);
bpf_spin_unlock(&queue_lock);
bpf_qdisc_watchdog_schedule(sch, expire, 0);
return NULL;
}
packet_len = packet->packet_len;
skb = bpf_kptr_xchg(&packet->skb, skb);
bpf_spin_lock(&queue_lock);
sch->q.qlen--;
sch->qstats.backlog -= packet_len;
bpf_spin_unlock(&queue_lock);
bpf_obj_drop(packet);
if (!skb)
return NULL;
bpf_qdisc_bstats_update(sch, skb);
__sync_fetch_and_add(&stats.dequeued, 1);
__sync_fetch_and_add(&stats.bytes_dequeued, packet_len);
return skb;
}
SEC("struct_ops/egress_pacer_init")
int BPF_PROG(egress_pacer_init, struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
(void)opt;
(void)extack;
sch->limit = queue_limit;
return 0;
}
SEC("struct_ops/egress_pacer_reset")
void BPF_PROG(egress_pacer_reset, struct Qdisc *sch)
{
int queued = sch->q.qlen;
int i;
bpf_for(i, 0, queued) {
struct bpf_list_node *node;
struct packet_node *packet;
struct sk_buff *skb = NULL;
bpf_spin_lock(&queue_lock);
node = bpf_list_pop_front(&packet_queue);
bpf_spin_unlock(&queue_lock);
if (!node)
break;
packet = container_of(node, struct packet_node, node);
skb = bpf_kptr_xchg(&packet->skb, skb);
if (skb) {
bpf_kfree_skb(skb);
__sync_fetch_and_add(&stats.cleanup_dropped, 1);
}
bpf_obj_drop(packet);
}
bpf_spin_lock(&queue_lock);
next_departure_ns = 0;
sch->q.qlen = 0;
sch->qstats.backlog = 0;
bpf_spin_unlock(&queue_lock);
}
SEC("struct_ops/egress_pacer_destroy")
void BPF_PROG(egress_pacer_destroy, struct Qdisc *sch)
{
(void)sch;
}
SEC(".struct_ops")
struct Qdisc_ops pacer = {
.enqueue = (void *)egress_pacer_enqueue,
.dequeue = (void *)egress_pacer_dequeue,
.init = (void *)egress_pacer_init,
.reset = (void *)egress_pacer_reset,
.destroy = (void *)egress_pacer_destroy,
.id = "bpf_pacer",
};

View File

@@ -0,0 +1,296 @@
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <errno.h>
#include <getopt.h>
#include <linux/pkt_sched.h>
#include <net/if.h>
#include <signal.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <unistd.h>
#include <bpf/libbpf.h>
#include "egress_pacer.h"
#include "egress_pacer.skel.h"
static volatile sig_atomic_t exiting;
static struct env {
const char *interface;
unsigned long long rate_kbps;
unsigned int queue_limit;
unsigned int duration;
bool verbose;
} env = {
.rate_kbps = 1024,
.queue_limit = 256,
.duration = 10,
};
static void handle_signal(int signal)
{
(void)signal;
exiting = 1;
}
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s --interface IFACE [--rate-kbps KBPS] "
"[--queue-limit PACKETS] [--duration SEC] [--verbose]\n\n"
"Temporarily pace one interface with a bounded BPF qdisc.\n\n"
"Options:\n"
" -i, --interface IFACE interface to control (required)\n"
" -r, --rate-kbps KBPS egress rate, 8-100000000 "
"(default: 1024)\n"
" -q, --queue-limit PACKETS queue capacity, 1-65535 "
"(default: 256)\n"
" -d, --duration SEC control window, 1-86400 "
"(default: 10)\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_u64(const char *value, unsigned long long maximum,
unsigned long long *result)
{
char *end = NULL;
unsigned long long parsed;
errno = 0;
parsed = strtoull(value, &end, 10);
if (errno || end == value || *end || parsed > maximum)
return -EINVAL;
*result = parsed;
return 0;
}
static int parse_rate(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 100000000, &parsed) || parsed < 8) {
fprintf(stderr, "invalid rate in Kbit/s: %s\n", value);
return -EINVAL;
}
env.rate_kbps = parsed;
return 0;
}
static int parse_queue_limit(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 65535, &parsed) || parsed == 0) {
fprintf(stderr, "invalid queue limit: %s\n", value);
return -EINVAL;
}
env.queue_limit = parsed;
return 0;
}
static int parse_duration(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 86400, &parsed) || parsed == 0) {
fprintf(stderr, "invalid duration in seconds: %s\n", value);
return -EINVAL;
}
env.duration = parsed;
return 0;
}
static int parse_option(int option, const char *program)
{
switch (option) {
case 'i':
env.interface = optarg;
return 0;
case 'r':
return parse_rate(optarg);
case 'q':
return parse_queue_limit(optarg);
case 'd':
return parse_duration(optarg);
case 'v':
env.verbose = true;
return 0;
case 'h':
usage(program);
exit(0);
default:
return -EINVAL;
}
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "interface", required_argument, NULL, 'i' },
{ "rate-kbps", required_argument, NULL, 'r' },
{ "queue-limit", required_argument, NULL, 'q' },
{ "duration", required_argument, NULL, 'd' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
int error, option;
while ((option = getopt_long(argc, argv, "i:r:q:d:vh", options, NULL)) != -1) {
error = parse_option(option, argv[0]);
if (error)
return error;
}
if (!env.interface) {
fprintf(stderr, "--interface is required\n");
return -EINVAL;
}
if (optind != argc)
return -EINVAL;
return 0;
}
static long long monotonic_milliseconds(void)
{
struct timespec timestamp;
if (clock_gettime(CLOCK_MONOTONIC, &timestamp))
return -errno;
return timestamp.tv_sec * 1000LL + timestamp.tv_nsec / 1000000;
}
static int wait_for_duration(void)
{
struct timespec interval = { .tv_nsec = 100000000 };
long long deadline, now;
now = monotonic_milliseconds();
if (now < 0)
return (int)now;
deadline = now + env.duration * 1000LL;
while (!exiting) {
now = monotonic_milliseconds();
if (now < 0)
return (int)now;
if (now >= deadline)
break;
if (nanosleep(&interval, NULL) && errno != EINTR)
return -errno;
}
return 0;
}
int main(int argc, char **argv)
{
struct egress_pacer_bpf *skel = NULL;
struct pacer_stats final_stats = {};
struct bpf_tc_hook hook = {
.sz = sizeof(hook),
.attach_point = BPF_TC_QDISC,
.parent = TC_H_ROOT,
.handle = TC_H_MAKE(1 << 16, 0),
.qdisc = "bpf_pacer",
};
bool qdisc_created = false;
unsigned int ifindex;
int cleanup_err;
int err;
err = parse_args(argc, argv);
if (err) {
usage(argv[0]);
return 1;
}
ifindex = if_nametoindex(env.interface);
if (!ifindex) {
fprintf(stderr, "interface does not exist: %s\n", env.interface);
return 1;
}
hook.ifindex = ifindex;
libbpf_set_print(libbpf_print_fn);
signal(SIGINT, handle_signal);
signal(SIGTERM, handle_signal);
skel = egress_pacer_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return 1;
}
skel->rodata->rate_kbps = env.rate_kbps;
skel->rodata->queue_limit = env.queue_limit;
err = egress_pacer_bpf__load(skel);
if (err) {
fprintf(stderr,
"failed to load BPF qdisc: %s\n"
"This tool requires Linux 6.16+, CONFIG_NET_SCH_BPF, "
"BTF, and BPF JIT.\n",
strerror(-err));
goto cleanup;
}
err = egress_pacer_bpf__attach(skel);
if (err) {
fprintf(stderr, "failed to register bpf_pacer qdisc: %s\n",
strerror(-err));
goto cleanup;
}
err = bpf_tc_hook_create(&hook);
if (err) {
if (err == -EEXIST)
fprintf(stderr,
"refusing to replace the existing root qdisc on %s\n",
env.interface);
else
fprintf(stderr, "failed to attach bpf_pacer to %s: %s\n",
env.interface, strerror(-err));
goto cleanup;
}
qdisc_created = true;
printf("READY interface=%s rate_kbps=%llu queue_limit=%u duration=%u\n",
env.interface, env.rate_kbps, env.queue_limit, env.duration);
fflush(stdout);
err = wait_for_duration();
cleanup:
if (qdisc_created) {
cleanup_err = bpf_tc_hook_destroy(&hook);
if (cleanup_err) {
fprintf(stderr, "failed to remove bpf_pacer from %s: %s\n",
env.interface, strerror(-cleanup_err));
if (!err)
err = cleanup_err;
}
}
if (skel && skel->bss)
final_stats = skel->bss->stats;
if (qdisc_created) {
printf("SUMMARY enqueued=%llu dequeued=%llu policy_dropped=%llu "
"cleanup_dropped=%llu bytes_dequeued=%llu max_qlen=%llu\n",
final_stats.enqueued, final_stats.dequeued,
final_stats.policy_dropped, final_stats.cleanup_dropped,
final_stats.bytes_dequeued, final_stats.max_qlen);
}
egress_pacer_bpf__destroy(skel);
return err != 0;
}

View File

@@ -0,0 +1,14 @@
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __EGRESS_PACER_H
#define __EGRESS_PACER_H
struct pacer_stats {
unsigned long long enqueued;
unsigned long long dequeued;
unsigned long long policy_dropped;
unsigned long long cleanup_dropped;
unsigned long long bytes_dequeued;
unsigned long long max_qlen;
};
#endif /* __EGRESS_PACER_H */

View File

@@ -0,0 +1,300 @@
#!/usr/bin/env python3
"""Deterministic KVM integration checks for the BPF egress pacer."""
from __future__ import annotations
import errno
import os
import re
import select
import signal
import socket
import struct
import subprocess
import sys
import threading
import time
TX_INTERFACE = "epac_tx"
RX_INTERFACE = "epac_rx"
ETHERTYPE = 0x88B5
MAGIC = b"EPAC"
FRAME_SIZE = 1024
ATTEMPTS = 40
RATE_KBPS = 64
QUEUE_LIMIT = 8
SUMMARY = re.compile(
r"SUMMARY enqueued=(\d+) dequeued=(\d+) policy_dropped=(\d+) "
r"cleanup_dropped=(\d+) bytes_dequeued=(\d+) max_qlen=(\d+)"
)
def run(*command: str, check: bool = True) -> subprocess.CompletedProcess[str]:
return subprocess.run(
command,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
check=check,
)
def delete_test_link() -> None:
run("ip", "link", "del", TX_INTERFACE, check=False)
def create_test_link() -> None:
delete_test_link()
run("ip", "link", "add", TX_INTERFACE, "type", "veth", "peer", "name", RX_INTERFACE)
run("ip", "link", "set", "dev", TX_INTERFACE, "addrgenmode", "none")
run("ip", "link", "set", "dev", RX_INTERFACE, "addrgenmode", "none")
run("ip", "link", "set", "dev", TX_INTERFACE, "up")
run("ip", "link", "set", "dev", RX_INTERFACE, "up")
def wait_until_ready(process: subprocess.Popen[str]) -> list[str]:
lines: list[str] = []
deadline = time.monotonic() + 8
assert process.stdout is not None
while time.monotonic() < deadline:
readable, _, _ = select.select([process.stdout], [], [], 0.25)
if readable:
line = process.stdout.readline()
if line:
lines.append(line)
if line.startswith("READY "):
return lines
if process.poll() is not None:
break
process.kill()
remainder, _ = process.communicate()
raise AssertionError("pacer did not become ready:\n" + "".join(lines) + remainder)
def receive_frames(
stop: threading.Event,
ready: threading.Event,
times: list[float],
errors: list[Exception],
) -> None:
receiver: socket.socket | None = None
try:
receiver = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.htons(ETHERTYPE))
receiver.bind((RX_INTERFACE, 0))
receiver.settimeout(0.1)
ready.set()
while not stop.is_set():
try:
frame = receiver.recv(2048)
except TimeoutError:
continue
if len(frame) >= 18 and frame[12:14] == struct.pack("!H", ETHERTYPE):
if frame[14:18] == MAGIC:
times.append(time.monotonic())
except Exception as error:
errors.append(error)
ready.set()
finally:
if receiver is not None:
receiver.close()
def send_burst() -> int:
sender = socket.socket(socket.AF_PACKET, socket.SOCK_RAW)
sender.bind((TX_INTERFACE, 0))
send_errors = 0
try:
for sequence in range(ATTEMPTS):
header = b"\xff" * 6 + b"\x02\x00\x00\x00\x00\x01" + struct.pack("!H", ETHERTYPE)
payload = MAGIC + struct.pack("!I", sequence)
frame = header + payload + b"x" * (FRAME_SIZE - len(header) - len(payload))
try:
sender.send(frame)
except OSError as error:
if error.errno != errno.ENOBUFS:
raise
send_errors += 1
finally:
sender.close()
return send_errors
def parse_summary(output: str) -> dict[str, int]:
match = SUMMARY.search(output)
if not match:
raise AssertionError("missing summary:\n" + output)
names = (
"enqueued",
"dequeued",
"policy_dropped",
"cleanup_dropped",
"bytes_dequeued",
"max_qlen",
)
return dict(zip(names, map(int, match.groups())))
def main() -> int:
binary = os.path.abspath(sys.argv[1] if len(sys.argv) > 1 else "./egress_pacer")
stop = threading.Event()
receiver: threading.Thread | None = None
process: subprocess.Popen[str] | None = None
invalid = run(binary, "--interface", "definitely_missing", "--duration", "1", check=False)
assert invalid.returncode != 0
assert "interface does not exist" in invalid.stdout
create_test_link()
try:
run("tc", "qdisc", "add", "dev", TX_INTERFACE, "root", "handle", "1:",
"pfifo", "limit", "16")
conflict = run(
binary,
"--interface", TX_INTERFACE,
"--rate-kbps", str(RATE_KBPS),
"--queue-limit", str(QUEUE_LIMIT),
"--duration", "1",
check=False,
)
assert conflict.returncode != 0, conflict.stdout
assert "refusing to replace" in conflict.stdout, conflict.stdout
run("tc", "qdisc", "del", "dev", TX_INTERFACE, "root")
receiver_ready = threading.Event()
receive_times: list[float] = []
receiver_errors: list[Exception] = []
receiver = threading.Thread(
target=receive_frames,
args=(stop, receiver_ready, receive_times, receiver_errors),
)
receiver.start()
assert receiver_ready.wait(timeout=2)
assert not receiver_errors, receiver_errors
process = subprocess.Popen(
[
binary,
"--interface", TX_INTERFACE,
"--rate-kbps", str(RATE_KBPS),
"--queue-limit", str(QUEUE_LIMIT),
"--duration", "2",
],
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
bufsize=1,
)
lines = wait_until_ready(process)
send_errors = send_burst()
remainder, _ = process.communicate(timeout=8)
output = "".join(lines) + remainder
stop.set()
receiver.join(timeout=2)
assert not receiver.is_alive()
assert not receiver_errors, receiver_errors
assert process.returncode == 0, output
stats = parse_summary(output)
assert stats["enqueued"] + stats["policy_dropped"] == ATTEMPTS, stats
assert stats["policy_dropped"] > 0, stats
assert stats["dequeued"] == stats["enqueued"], stats
assert stats["cleanup_dropped"] == 0, stats
assert stats["max_qlen"] <= QUEUE_LIMIT, stats
assert stats["max_qlen"] >= 2, stats
assert len(receive_times) == stats["dequeued"], (len(receive_times), stats)
assert len(receive_times) >= 5, receive_times
span = receive_times[-1] - receive_times[0]
assert span >= 0.4, span
observed_kbps = (len(receive_times) - 1) * FRAME_SIZE * 8 / span / 1000
assert observed_kbps <= RATE_KBPS * 1.5, observed_kbps
qdisc_after = run("tc", "qdisc", "show", "dev", TX_INTERFACE).stdout
assert "bpf_pacer" not in qdisc_after, qdisc_after
process = subprocess.Popen(
[
binary,
"--interface", TX_INTERFACE,
"--rate-kbps", str(RATE_KBPS),
"--queue-limit", str(QUEUE_LIMIT),
"--duration", "30",
],
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
bufsize=1,
)
signal_lines = wait_until_ready(process)
send_burst()
process.terminate()
signal_remainder, _ = process.communicate(timeout=4)
signal_output = "".join(signal_lines) + signal_remainder
assert process.returncode == 0, signal_output
signal_stats = parse_summary(signal_output)
assert signal_stats["enqueued"] + signal_stats["policy_dropped"] == ATTEMPTS, signal_stats
assert signal_stats["cleanup_dropped"] > 0, signal_stats
assert signal_stats["enqueued"] == (
signal_stats["dequeued"] + signal_stats["cleanup_dropped"]
), signal_stats
qdisc_after_signal = run("tc", "qdisc", "show", "dev", TX_INTERFACE).stdout
assert "bpf_pacer" not in qdisc_after_signal, qdisc_after_signal
process = subprocess.Popen(
[
binary,
"--interface", TX_INTERFACE,
"--rate-kbps", str(RATE_KBPS),
"--queue-limit", str(QUEUE_LIMIT),
"--duration", "30",
],
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
bufsize=1,
)
wait_until_ready(process)
process.kill()
process.communicate(timeout=4)
assert process.returncode == -signal.SIGKILL, process.returncode
qdisc_after_kill = run("tc", "qdisc", "show", "dev", TX_INTERFACE).stdout
assert "bpf_pacer" in qdisc_after_kill, qdisc_after_kill
run("tc", "qdisc", "del", "dev", TX_INTERFACE, "root")
qdisc_after_recovery = run("tc", "qdisc", "show", "dev", TX_INTERFACE).stdout
assert "bpf_pacer" not in qdisc_after_recovery, qdisc_after_recovery
ready_line = next(line for line in output.splitlines() if line.startswith("READY "))
summary_line = next(line for line in output.splitlines() if line.startswith("SUMMARY "))
print(ready_line)
print(summary_line)
print(
f"TEST-SUMMARY attempts={ATTEMPTS} received={len(receive_times)} "
f"send_errors={send_errors} span_ms={span * 1000:.0f} "
f"observed_kbps={observed_kbps:.1f}"
)
print(
"PASS: conflict refusal, bounded drops, pacing, accounting, "
"normal/signal cleanup, and SIGKILL recovery succeeded"
)
finally:
if process is not None and process.poll() is None:
process.terminate()
try:
process.communicate(timeout=2)
except subprocess.TimeoutExpired:
process.kill()
process.communicate()
stop.set()
if receiver is not None:
receiver.join(timeout=2)
delete_test_link()
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,10 @@
level=Depth
type=Security
kernel_min=6.19
kernel_min_basis=required-feature
architectures=x86_64
btf=required
kernel_config=CONFIG_BPF=y,CONFIG_BPF_SYSCALL=y,CONFIG_BPF_JIT=y,CONFIG_BPF_LSM=y,CONFIG_SECURITY=y,CONFIG_DEBUG_INFO_BTF=y
hardware=none
root=required
test_status=ci-build

16
src/54-exec-image-inspector/.gitignore vendored Normal file
View File

@@ -0,0 +1,16 @@
# Build artifacts
.output/
*.o
*.skel.h
# Generated binaries
exec_image_inspector
tests/exec_fixture
# Python cache
__pycache__/
# Editor files
*.swp
*~
.vscode/

View File

@@ -0,0 +1,101 @@
# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
OUTPUT := .output
CLANG ?= clang
LIBBPF_SRC := $(abspath ../third_party/libbpf/src)
BPFTOOL_SRC := $(abspath ../third_party/bpftool/src)
LIBBPF_OBJ := $(abspath $(OUTPUT)/libbpf.a)
BPFTOOL_OUTPUT ?= $(abspath $(OUTPUT)/bpftool)
BPFTOOL ?= $(BPFTOOL_OUTPUT)/bootstrap/bpftool
ARCH ?= $(shell uname -m | sed 's/x86_64/x86/' \
| sed 's/arm.*/arm/' \
| sed 's/aarch64/arm64/' \
| sed 's/ppc64le/powerpc/' \
| sed 's/mips.*/mips/' \
| sed 's/riscv64/riscv/' \
| sed 's/loongarch64/loongarch/')
VMLINUX := ../third_party/vmlinux/$(ARCH)/vmlinux.h
INCLUDES := -I$(OUTPUT) -I../third_party/libbpf/include/uapi -I$(dir $(VMLINUX)) -I.
CFLAGS := -g -Wall -Wextra -D_GNU_SOURCE
ALL_LDFLAGS := $(LDFLAGS) $(EXTRA_LDFLAGS)
APPS = exec_image_inspector
FIXTURES = tests/exec_fixture
CLANG_BPF_SYS_INCLUDES ?= $(shell $(CLANG) -v -E - </dev/null 2>&1 \
| sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }')
ifeq ($(V),1)
Q =
msg =
else
Q = @
msg = @printf ' %-8s %s%s\n' "$(1)" \
"$(patsubst $(abspath $(OUTPUT))/%,%,$(2))" \
"$(if $(3), $(3))";
MAKEFLAGS += --no-print-directory
endif
define allow-override
$(if $(or $(findstring environment,$(origin $(1))),\
$(findstring command line,$(origin $(1)))),,\
$(eval $(1) = $(2)))
endef
$(call allow-override,CC,$(CROSS_COMPILE)cc)
$(call allow-override,LD,$(CROSS_COMPILE)ld)
.PHONY: all clean test
all: $(APPS) $(FIXTURES)
clean:
$(call msg,CLEAN)
$(Q)rm -rf $(OUTPUT) $(APPS) $(FIXTURES)
test: all
@if [ "$$(id -u)" -ne 0 ]; then \
echo "test requires root; run inside the documented KVM guest" >&2; \
exit 1; \
fi
python3 tests/test_exec_image_inspector.py ./exec_image_inspector ./tests/exec_fixture
$(OUTPUT) $(OUTPUT)/libbpf $(BPFTOOL_OUTPUT):
$(call msg,MKDIR,$@)
$(Q)mkdir -p $@
$(LIBBPF_OBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)/libbpf
$(call msg,LIB,$@)
$(Q)$(MAKE) -C $(LIBBPF_SRC) BUILD_STATIC_ONLY=1 \
OBJDIR=$(dir $@)/libbpf DESTDIR=$(dir $@) \
INCLUDEDIR= LIBDIR= UAPIDIR= install
$(BPFTOOL): | $(BPFTOOL_OUTPUT)
$(call msg,BPFTOOL,$@)
$(Q)$(MAKE) ARCH= CROSS_COMPILE= OUTPUT=$(BPFTOOL_OUTPUT)/ -C $(BPFTOOL_SRC) bootstrap
$(OUTPUT)/%.bpf.o: %.bpf.c $(LIBBPF_OBJ) $(wildcard %.h) $(VMLINUX) | $(OUTPUT) $(BPFTOOL)
$(call msg,BPF,$@)
$(Q)$(CLANG) -g -O2 -target bpf -D__TARGET_ARCH_$(ARCH) \
$(INCLUDES) $(CLANG_BPF_SYS_INCLUDES) \
-c $(filter %.c,$^) -o $(patsubst %.bpf.o,%.tmp.bpf.o,$@)
$(Q)$(BPFTOOL) gen object $@ $(patsubst %.bpf.o,%.tmp.bpf.o,$@)
$(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(OUTPUT) $(BPFTOOL)
$(call msg,GEN-SKEL,$@)
$(Q)$(BPFTOOL) gen skeleton $< > $@
$(patsubst %,$(OUTPUT)/%.o,$(APPS)): %.o: %.skel.h
$(OUTPUT)/%.o: %.c $(wildcard %.h) | $(OUTPUT)
$(call msg,CC,$@)
$(Q)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
$(APPS): %: $(OUTPUT)/%.o $(LIBBPF_OBJ) | $(OUTPUT)
$(call msg,BINARY,$@)
$(Q)$(CC) $(CFLAGS) $^ $(ALL_LDFLAGS) -lelf -lz -o $@
$(FIXTURES): %: %.c
$(call msg,CC,$@)
$(Q)$(CC) -g -Wall -Wextra $< -o $@
.DELETE_ON_ERROR:
.SECONDARY:

View File

@@ -0,0 +1,999 @@
# eBPF Tutorial: Inspecting the Executable Image After exec
Imagine a container running a series of nested wrapper scripts. What you see from `ps` or `/proc` is `/usr/bin/wrapper.sh --config /etc/app.conf`, but the actual business logic might be a Python interpreter or some dynamically linked binary three layers deep. When troubleshooting, you need to know which executable image the kernel finally installed, what architecture it targets, and what byte order it uses - not just the command-line arguments.
This tutorial shows how to achieve this using BPF task work and file-backed dynptr. The tool attaches an LSM hook to observe the `bprm_committed_creds` moment, schedules a task work callback for the target process after credentials are committed, and that callback runs in a sleepable context to reacquire the installed executable file, resolve its path, read the ELF header, and send the results to user space through a ring buffer.
> Complete source: <https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/54-exec-image-inspector>
## Why BPF task work and file dynptr are needed
Traditional approaches to inspecting executable files all have significant limitations.
The most straightforward idea is to read the `/proc/<pid>/exe` symlink, but this requires the process to still be alive. Short-lived processes may have already exited before you can read it. Furthermore, `/proc` access happens in user space, creating a race window with the kernel's exec event - the same process might have already executed `execve` multiple times.
Using a tracepoint or kprobe to observe `sched_process_exec` can capture exec events, but these hooks run in a non-sleepable context. You can obtain the file path, but if you need to read file contents to verify the ELF header or check embedded metadata, you run into a problem: the file-backed dynptr's `bpf_dynptr_read` triggers a page fault when target bytes are on a cold page, and page faults require sleep to complete. BPF programs in a non-sleepable context cannot wait for I/O.
BPF task work solves this problem. Linux 6.18 introduced the `bpf_task_work_schedule_signal` kfunc, which allows a BPF program to schedule a callback for a target task. The kernel executes this callback in a safe sleepable context. Linux 6.19 introduced file-backed dynptr, providing verifier-tracked file data access. Combining these two features, you can identify the target process in an LSM hook, record a timestamp, and schedule task work. The callback then runs in a sleepable context where it can reacquire the task's installed executable file, read content from any offset through the dynptr (including cold pages), and finally send the complete inspection results to user space.
This is the design philosophy behind `exec_image_inspector`: the LSM hook selects the child process and schedules task work, while the sleepable callback performs the actual file reading and parsing.
## How BPF task work and file dynptr work
BPF task work is a deferred execution mechanism. When you call `bpf_task_work_schedule_signal(task, work, map, callback)` in a BPF program, the kernel associates the callback with the specified task. The callback does not execute immediately - it runs at a safe point before that task returns to user space, where the context allows sleeping.
`struct bpf_task_work` is an opaque 64-byte structure. The BPF program only needs to allocate storage for it; the kernel manages its internal state. This tool uses a single-element ARRAY map to store `struct exec_work`, which contains the `bpf_task_work` storage, a scheduling timestamp, and direct probe results.
File-backed dynptr provides a verifier-tracked file data access interface. `bpf_dynptr_from_file(file, flags, dynptr)` creates a dynptr from a file, and `bpf_dynptr_read(dst, len, dynptr, offset, flags)` reads content at the specified offset. The dynptr holds internal state, so every path that creates a dynptr must call `bpf_dynptr_file_discard` to release it, including failure branches.
In a non-sleepable context, the dynptr can only read content already in the page cache; accessing a cold page returns `-EFAULT`. In a sleepable context, the dynptr can trigger page fault operations and wait for I/O to complete. This is precisely the value of the task work callback.
## Overall tool flow
After user space starts, it forks the target command, but the child process blocks waiting for a release signal on a pipe. This way the parent knows the child's PID (which is also the TGID) before the child executes `execvp`, and can write this value into the BPF program's read-only data section as a filter condition. Blocking the child eliminates the attach race with short-lived commands and keeps the example self-contained.
The parent opens the BPF skeleton, sets `target_tgid` and the optional `probe_offset`, loads and attaches the LSM program, creates a ring buffer reader, then writes a release byte to the pipe to let the child execute `execvp`.
After the child executes exec, the `lsm/bprm_committed_creds` hook triggers when credentials are committed. The BPF program compares the current TGID with `target_tgid` and returns immediately if they do not match. After matching the target, the program records a scheduling timestamp, optionally attempts a direct read in the current context if `--probe-offset` is set (to verify the cold page scenario), then calls `bpf_task_work_schedule_signal` to schedule the callback.
The callback `inspect_executable` runs in that task's sleepable context. It calls `bpf_get_task_exe_file` to obtain the task's currently installed executable file (returning a referenced `struct file` that must be released with `bpf_put_file`), resolves the path with `bpf_path_d_path`, and reads the 64-byte ELF header plus the optional marker location using the file dynptr. After a successful read, it parses the ELF magic, class, data, type, and machine fields. Finally, it packages all the information into a ring buffer event and sends it to user space.
User space alternates between polling the ring buffer and checking child process status with `waitpid(WNOHANG)`. The loop ends when the child is reaped and at least one event is received, or when the timeout is reached. The same child might install multiple images in succession (for example, `/bin/sh -c 'exec /bin/true'`), so after the child is reaped, `drain_events` performs a zero-timeout poll to collect any remaining events in the ring buffer.
![Exec image inspector data flow](https://github.com/eunomia-bpf/bpf-developer-tutorial/raw/main/src/54-exec-image-inspector/exec-image-flow.png)
## Code implementation
This tool consists of four files: a shared header defining event and statistics structures, a compatibility header declaring new kernel kfuncs, a BPF program implementing the hook and callback, and a user-space loader managing the lifecycle.
### Shared header
`exec_image_inspector.h` defines the ring buffer event structure and statistics structure shared between BPF and user space.
```c
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __EXEC_IMAGE_INSPECTOR_H
#define __EXEC_IMAGE_INSPECTOR_H
#define EXEC_COMM_LEN 16
#define EXEC_PATH_LEN 256
#define EXEC_PROBE_LEN 8
struct exec_event {
unsigned int pid;
unsigned int tgid;
unsigned char is_elf;
unsigned char elf_class;
unsigned char elf_data;
unsigned char reserved;
unsigned short elf_type;
unsigned short elf_machine;
int header_error;
int path_error;
int direct_probe_error;
int deferred_probe_error;
unsigned long long latency_ns;
unsigned long long probe_offset;
char comm[EXEC_COMM_LEN];
char path[EXEC_PATH_LEN];
unsigned char probe_bytes[EXEC_PROBE_LEN];
};
struct inspector_stats {
unsigned long long matched;
unsigned long long scheduled;
unsigned long long schedule_errors;
unsigned long long callbacks;
unsigned long long header_errors;
unsigned long long path_errors;
unsigned long long direct_probes;
unsigned long long direct_probe_errors;
unsigned long long deferred_probes;
unsigned long long deferred_probe_errors;
unsigned long long dropped;
};
#endif /* __EXEC_IMAGE_INSPECTOR_H */
```
`exec_event` contains all the information needed to inspect one exec: PID, TGID, process name, path, ELF parsing results, probe results, callback latency, and various error codes. `inspector_stats` accumulates match, schedule, callback, error, and drop counts; user space reads the BSS section at the end and reports the statistics.
### Compatibility declarations
Linux 6.18 introduced BPF task work, and 6.19 introduced file-backed dynptr. The repository's currently generated UAPI and BTF headers predate these features, so this tool places the missing declarations in a local header `bpf_experimental.h`.
```c
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __EXEC_IMAGE_INSPECTOR_BPF_EXPERIMENTAL_H
#define __EXEC_IMAGE_INSPECTOR_BPF_EXPERIMENTAL_H
/*
* These Linux 6.18/6.19 declarations are not present in the repository's
* older generated UAPI and vmlinux headers. Keep them local until those
* vendored headers are regenerated.
*/
struct bpf_task_work {
__u64 opaque;
} __attribute__((aligned(8)));
typedef int (*bpf_task_work_callback_t)(struct bpf_map *map, void *key,
void *value);
extern int bpf_task_work_schedule_signal(struct task_struct *task,
struct bpf_task_work *work,
void *map__map,
bpf_task_work_callback_t callback) __ksym;
extern struct file *bpf_get_task_exe_file(struct task_struct *task) __ksym;
extern void bpf_put_file(struct file *file) __ksym;
extern int bpf_path_d_path(const struct path *path, char *buf,
__u64 buf__sz) __ksym;
extern int bpf_dynptr_from_file(struct file *file, __u32 flags,
struct bpf_dynptr *ptr__uninit) __ksym;
extern int bpf_dynptr_file_discard(struct bpf_dynptr *dynptr) __ksym;
#endif /* __EXEC_IMAGE_INSPECTOR_BPF_EXPERIMENTAL_H */
```
This file declares kfuncs including `bpf_task_work_schedule_signal`, `bpf_get_task_exe_file`, `bpf_put_file`, `bpf_path_d_path`, `bpf_dynptr_from_file`, and `bpf_dynptr_file_discard`. This file can be removed once the repository's integrated vmlinux headers include these declarations.
### BPF program
`exec_image_inspector.bpf.c` implements the LSM hook and task work callback.
```c
// SPDX-License-Identifier: GPL-2.0
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include "bpf_experimental.h"
#include "exec_image_inspector.h"
char LICENSE[] SEC("license") = "GPL";
#define ENOENT 2
#define EI_CLASS 4
#define EI_DATA 5
#define ELFCLASS32 1
#define ELFCLASS64 2
#define ELFDATA2LSB 1
#define ELFDATA2MSB 2
const volatile __u32 target_tgid;
const volatile __u32 probe_offset;
struct inspector_stats stats;
struct {
__uint(type, BPF_MAP_TYPE_RINGBUF);
__uint(max_entries, 256 * 1024);
} events SEC(".maps");
struct exec_work {
__u64 scheduled_ns;
int direct_probe_error;
struct bpf_task_work work;
};
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 1);
__type(key, __u32);
__type(value, struct exec_work);
} pending SEC(".maps");
static __u16 read_elf_u16(const unsigned char *header, int offset, __u8 data)
{
if (data == ELFDATA2MSB)
return ((__u16)header[offset] << 8) | header[offset + 1];
return header[offset] | ((__u16)header[offset + 1] << 8);
}
static int probe_file_without_sleep(struct file *file)
{
unsigned char sample[EXEC_PROBE_LEN];
struct bpf_dynptr dynptr;
int err;
if (!probe_offset)
return 0;
__sync_fetch_and_add(&stats.direct_probes, 1);
if (!file) {
err = -ENOENT;
goto record;
}
err = bpf_dynptr_from_file(file, 0, &dynptr);
if (err) {
bpf_dynptr_file_discard(&dynptr);
goto record;
}
err = bpf_dynptr_read(sample, sizeof(sample), &dynptr, probe_offset, 0);
bpf_dynptr_file_discard(&dynptr);
record:
if (err)
__sync_fetch_and_add(&stats.direct_probe_errors, 1);
return err;
}
static int inspect_executable(struct bpf_map *map, void *key, void *value)
{
unsigned char header[64] = {};
struct exec_work *work = value;
struct exec_event event = {};
struct task_struct *task;
struct bpf_dynptr dynptr;
struct file *file;
__u64 pid_tgid;
int err;
(void)map;
(void)key;
__sync_fetch_and_add(&stats.callbacks, 1);
pid_tgid = bpf_get_current_pid_tgid();
event.pid = (__u32)pid_tgid;
event.tgid = pid_tgid >> 32;
event.latency_ns = bpf_ktime_get_ns() - work->scheduled_ns;
event.direct_probe_error = work->direct_probe_error;
event.probe_offset = probe_offset;
bpf_get_current_comm(event.comm, sizeof(event.comm));
task = bpf_get_current_task_btf();
file = bpf_get_task_exe_file(task);
if (!file) {
event.header_error = -ENOENT;
__sync_fetch_and_add(&stats.header_errors, 1);
if (probe_offset) {
event.deferred_probe_error = -ENOENT;
__sync_fetch_and_add(&stats.deferred_probes, 1);
__sync_fetch_and_add(&stats.deferred_probe_errors, 1);
}
goto emit;
}
err = bpf_path_d_path(&file->f_path, event.path, sizeof(event.path));
if (err < 0) {
event.path_error = err;
__sync_fetch_and_add(&stats.path_errors, 1);
}
err = bpf_dynptr_from_file(file, 0, &dynptr);
if (err) {
bpf_dynptr_file_discard(&dynptr);
event.header_error = err;
__sync_fetch_and_add(&stats.header_errors, 1);
if (probe_offset) {
event.deferred_probe_error = err;
__sync_fetch_and_add(&stats.deferred_probes, 1);
__sync_fetch_and_add(&stats.deferred_probe_errors, 1);
}
goto put_file;
}
err = bpf_dynptr_read(header, sizeof(header), &dynptr, 0, 0);
if (err) {
event.header_error = err;
__sync_fetch_and_add(&stats.header_errors, 1);
}
if (probe_offset) {
__sync_fetch_and_add(&stats.deferred_probes, 1);
err = bpf_dynptr_read(event.probe_bytes, sizeof(event.probe_bytes),
&dynptr, probe_offset, 0);
event.deferred_probe_error = err;
if (err)
__sync_fetch_and_add(&stats.deferred_probe_errors, 1);
}
bpf_dynptr_file_discard(&dynptr);
if (!event.header_error && header[0] == 0x7f && header[1] == 'E' &&
header[2] == 'L' && header[3] == 'F') {
event.is_elf = 1;
event.elf_class = header[EI_CLASS];
event.elf_data = header[EI_DATA];
event.elf_type = read_elf_u16(header, 16, event.elf_data);
event.elf_machine = read_elf_u16(header, 18, event.elf_data);
}
put_file:
bpf_put_file(file);
emit:
if (bpf_ringbuf_output(&events, &event, sizeof(event), 0))
__sync_fetch_and_add(&stats.dropped, 1);
return 0;
}
SEC("lsm/bprm_committed_creds")
void BPF_PROG(schedule_exec_inspection, struct linux_binprm *bprm)
{
struct task_struct *task;
struct exec_work *work;
__u64 pid_tgid;
__u32 key = 0, tgid;
int err;
pid_tgid = bpf_get_current_pid_tgid();
tgid = pid_tgid >> 32;
if (target_tgid && tgid != target_tgid)
return;
__sync_fetch_and_add(&stats.matched, 1);
work = bpf_map_lookup_elem(&pending, &key);
if (!work) {
__sync_fetch_and_add(&stats.schedule_errors, 1);
return;
}
work->scheduled_ns = bpf_ktime_get_ns();
work->direct_probe_error = probe_file_without_sleep(bprm->file);
task = bpf_get_current_task_btf();
err = bpf_task_work_schedule_signal(task, &work->work, &pending,
inspect_executable);
if (err) {
__sync_fetch_and_add(&stats.schedule_errors, 1);
return;
}
__sync_fetch_and_add(&stats.scheduled, 1);
}
```
The program entry point is the `schedule_exec_inspection` function declared with `SEC("lsm/bprm_committed_creds")`. The `bprm_committed_creds` hook runs after the new executable's credentials have been committed, at which point `bprm->file` points to the image involved in this exec. This hook itself is non-sleepable, but we can identify the target process here and schedule task work.
The two `const volatile` variables `target_tgid` and `probe_offset` are in the `.rodata` section. User space writes values after `open()` but before `load()`, and the verifier treats them as compile-time constants. The program first compares the current TGID with `target_tgid`, returning immediately if they do not match to avoid observing unrelated process execs.
After matching the target, the program looks up key 0 from the single-element `pending` ARRAY map to obtain `struct exec_work`. This structure stores the scheduling timestamp, direct probe result, and `struct bpf_task_work` storage. This tool observes only one child process per invocation, so one slot is sufficient. When `--probe-offset` is set, `probe_file_without_sleep` attempts to read 8 bytes from the specified location in `bprm->file` in the current non-sleepable context. The test program places the marker on a cold page to verify that direct reads return `-EFAULT` in this scenario.
After the hook saves the timestamp and direct read result, it calls `bpf_task_work_schedule_signal`, passing the current task, work storage, map pointer, and the `inspect_executable` callback function pointer. The kernel holds the references needed for callback execution, and the callback will run in a sleepable context before that task returns to user space.
`inspect_executable` is the task work callback function, with signature `int callback(struct bpf_map *map, void *key, void *value)`. It first calculates callback latency (current time minus scheduling timestamp) for diagnostics, then fills PID, TGID, process name, and direct probe result into the event structure. It then calls `bpf_get_task_exe_file` to obtain the task's installed executable file. This kfunc returns a referenced `struct file`; every successful acquisition must be released with `bpf_put_file`.
The callback first resolves the file path with `bpf_path_d_path`, then creates a dynptr with `bpf_dynptr_from_file`. Since we are now in a sleepable context, `bpf_dynptr_read` can trigger page fault operations and wait for I/O to complete. The callback only reads a 64-byte ELF header and an optional 8-byte marker. After a successful header read, the program checks the 4-byte ELF magic `\x7fELF`, then parses `EI_CLASS` (32-bit or 64-bit), `EI_DATA` (little-endian or big-endian), `e_type` (executable or shared object), and `e_machine` (architecture). The `read_elf_u16` helper function correctly parses 16-bit fields according to byte order.
Every path that creates a dynptr must call `bpf_dynptr_file_discard` to release its internal state, including branches where the helper fails. Finally, `bpf_ringbuf_output` sends the event to user space; the `dropped` counter is incremented if sending fails.
### User-space loader
`exec_image_inspector.c` handles command-line parsing, coordinates the blocked child, loads BPF, formats events, and cleans up resources.
```c
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <errno.h>
#include <getopt.h>
#include <limits.h>
#include <signal.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <time.h>
#include <unistd.h>
#include <bpf/libbpf.h>
#include "exec_image_inspector.h"
#include "exec_image_inspector.skel.h"
struct environment {
unsigned long long probe_offset;
unsigned int timeout_ms;
bool verbose;
char **command;
};
struct child_process {
pid_t pid;
int release_fd;
bool released;
bool reaped;
int status;
};
struct event_context {
unsigned int seen;
};
static struct environment env = {
.timeout_ms = 5000,
};
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s [--probe-offset BYTES] [--timeout-ms MS] [--verbose] "
"-- COMMAND [ARG...]\n\n"
"Inspect the executable image installed by one command.\n\n"
"Options:\n"
" -p, --probe-offset BYTES also compare direct/deferred file reads\n"
" -t, --timeout-ms MS bound the command, 100-60000 "
"(default: 5000)\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_u64(const char *value, unsigned long long maximum,
unsigned long long *result)
{
char *end = NULL;
unsigned long long parsed;
errno = 0;
parsed = strtoull(value, &end, 10);
if (errno || end == value || *end || parsed > maximum)
return -EINVAL;
*result = parsed;
return 0;
}
static int parse_probe_offset(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, UINT_MAX - EXEC_PROBE_LEN, &parsed)) {
fprintf(stderr, "invalid probe offset: %s\n", value);
return -EINVAL;
}
env.probe_offset = parsed;
return 0;
}
static int parse_timeout(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 60000, &parsed) || parsed < 100) {
fprintf(stderr, "invalid timeout in milliseconds: %s\n", value);
return -EINVAL;
}
env.timeout_ms = parsed;
return 0;
}
static int parse_option(int option, const char *program)
{
switch (option) {
case 'p':
return parse_probe_offset(optarg);
case 't':
return parse_timeout(optarg);
case 'v':
env.verbose = true;
return 0;
case 'h':
usage(program);
exit(0);
default:
return -EINVAL;
}
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "probe-offset", required_argument, NULL, 'p' },
{ "timeout-ms", required_argument, NULL, 't' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
int error, option;
while ((option = getopt_long(argc, argv, "+p:t:vh", options, NULL)) != -1) {
error = parse_option(option, argv[0]);
if (error)
return error;
}
if (optind == argc) {
fprintf(stderr, "COMMAND is required\n");
return -EINVAL;
}
env.command = &argv[optind];
return 0;
}
static long long monotonic_milliseconds(void)
{
struct timespec timestamp;
if (clock_gettime(CLOCK_MONOTONIC, &timestamp))
return -errno;
return timestamp.tv_sec * 1000LL + timestamp.tv_nsec / 1000000;
}
static int start_blocked_child(struct child_process *child)
{
int pipe_fds[2];
pid_t pid;
if (pipe(pipe_fds))
return -errno;
pid = fork();
if (pid < 0) {
int error = -errno;
close(pipe_fds[0]);
close(pipe_fds[1]);
return error;
}
if (pid == 0) {
char release;
ssize_t count;
close(pipe_fds[1]);
do {
count = read(pipe_fds[0], &release, sizeof(release));
} while (count < 0 && errno == EINTR);
close(pipe_fds[0]);
if (count != sizeof(release))
_exit(126);
/* Intentional argv execution; no shell parses the supplied arguments. */
execvp(env.command[0], env.command); /* Flawfinder: ignore */
fprintf(stderr, "failed to execute %s: %s\n", env.command[0],
strerror(errno));
_exit(127);
}
close(pipe_fds[0]);
child->pid = pid;
child->release_fd = pipe_fds[1];
return 0;
}
static int release_child(struct child_process *child)
{
char release = 1;
ssize_t count;
do {
count = write(child->release_fd, &release, sizeof(release));
} while (count < 0 && errno == EINTR);
close(child->release_fd);
child->release_fd = -1;
if (count != sizeof(release))
return count < 0 ? -errno : -EIO;
child->released = true;
return 0;
}
static int child_exit_code(int status)
{
if (WIFEXITED(status))
return WEXITSTATUS(status);
if (WIFSIGNALED(status))
return 128 + WTERMSIG(status);
return 125;
}
static int reap_child(struct child_process *child, int options)
{
pid_t result;
if (child->reaped)
return 1;
do {
result = waitpid(child->pid, &child->status, options);
} while (result < 0 && errno == EINTR);
if (result < 0)
return -errno;
if (result == 0)
return 0;
child->reaped = true;
return 1;
}
static int drain_events(struct ring_buffer *ring_buffer)
{
int error;
for (;;) {
error = ring_buffer__poll(ring_buffer, 0);
if (error == -EINTR)
continue;
if (error < 0) {
fprintf(stderr, "ring-buffer drain failed: %s\n",
strerror(-error));
return error;
}
if (!error)
return 0;
}
}
static const char *elf_class_name(unsigned char value)
{
switch (value) {
case 1:
return "ELF32";
case 2:
return "ELF64";
default:
return "UNKNOWN";
}
}
static const char *elf_data_name(unsigned char value)
{
switch (value) {
case 1:
return "LSB";
case 2:
return "MSB";
default:
return "UNKNOWN";
}
}
static const char *elf_type_name(unsigned short value)
{
switch (value) {
case 2:
return "ET_EXEC";
case 3:
return "ET_DYN";
default:
return "OTHER";
}
}
static const char *elf_machine_name(unsigned short value)
{
switch (value) {
case 3:
return "EM_386";
case 62:
return "EM_X86_64";
case 183:
return "EM_AARCH64";
default:
return "OTHER";
}
}
static int handle_event(void *context, void *data, size_t size)
{
const struct exec_event *event = data;
struct event_context *events = context;
unsigned int index;
if (size < sizeof(*event)) {
fprintf(stderr, "short ring-buffer event: %zu bytes\n", size);
return 0;
}
events->seen++;
printf("EXEC pid=%u tgid=%u comm=%.*s path=%.*s is_elf=%u "
"class=%s endian=%s type=%s(%u) machine=%s(%u) "
"header_error=%d path_error=%d latency_us=%llu\n",
event->pid, event->tgid, EXEC_COMM_LEN, event->comm,
EXEC_PATH_LEN, event->path, event->is_elf,
elf_class_name(event->elf_class), elf_data_name(event->elf_data),
elf_type_name(event->elf_type), event->elf_type,
elf_machine_name(event->elf_machine), event->elf_machine,
event->header_error, event->path_error,
event->latency_ns / 1000);
if (event->probe_offset) {
printf("PROBE offset=%llu direct_error=%d deferred_error=%d bytes=",
event->probe_offset, event->direct_probe_error,
event->deferred_probe_error);
for (index = 0; index < EXEC_PROBE_LEN; index++)
printf("%02x", event->probe_bytes[index]);
putchar('\n');
}
fflush(stdout);
return 0;
}
static void stop_child(struct child_process *child)
{
if (child->reaped || child->pid <= 0)
return;
if (!child->released && child->release_fd >= 0) {
close(child->release_fd);
child->release_fd = -1;
} else {
kill(child->pid, SIGKILL);
}
(void)reap_child(child, 0);
}
static int setup_inspector(const struct child_process *child,
struct event_context *events,
struct exec_image_inspector_bpf **skeleton,
struct ring_buffer **ring_buffer)
{
struct exec_image_inspector_bpf *skel;
struct ring_buffer *ring;
int error;
skel = exec_image_inspector_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return -ENOMEM;
}
*skeleton = skel;
skel->rodata->target_tgid = child->pid;
skel->rodata->probe_offset = env.probe_offset;
error = exec_image_inspector_bpf__load(skel);
if (error) {
fprintf(stderr, "failed to load BPF object: %s\n", strerror(-error));
return error;
}
error = exec_image_inspector_bpf__attach(skel);
if (error) {
fprintf(stderr, "failed to attach bprm_committed_creds LSM hook: %s\n",
strerror(-error));
return error;
}
ring = ring_buffer__new(bpf_map__fd(skel->maps.events), handle_event,
events, NULL);
if (!ring) {
fprintf(stderr, "failed to create ring buffer: %s\n", strerror(errno));
return errno ? -errno : -ENOMEM;
}
*ring_buffer = ring;
return 0;
}
static int reap_timed_out_child(struct child_process *child)
{
int error;
if (child->reaped)
return 0;
fprintf(stderr, "command exceeded timeout; sending SIGKILL\n");
kill(child->pid, SIGKILL);
error = reap_child(child, 0);
if (error < 0) {
fprintf(stderr, "waitpid after timeout failed: %s\n",
strerror(-error));
return error;
}
return 0;
}
static int wait_for_command(struct ring_buffer *ring_buffer,
struct child_process *child,
const struct event_context *events)
{
long long deadline, now;
int error;
printf("READY target_tgid=%d probe_offset=%llu timeout_ms=%u command=%s\n",
child->pid, env.probe_offset, env.timeout_ms, env.command[0]);
fflush(stdout);
error = release_child(child);
if (error) {
fprintf(stderr, "failed to release command process: %s\n",
strerror(-error));
return error;
}
now = monotonic_milliseconds();
if (now < 0) {
fprintf(stderr, "failed to read monotonic clock: %s\n",
strerror((int)-now));
return (int)now;
}
deadline = now + env.timeout_ms;
for (;;) {
error = ring_buffer__poll(ring_buffer, 50);
if (error == -EINTR)
continue;
if (error < 0) {
fprintf(stderr, "ring-buffer poll failed: %s\n", strerror(-error));
return error;
}
error = reap_child(child, WNOHANG);
if (error < 0) {
fprintf(stderr, "waitpid failed: %s\n", strerror(-error));
return error;
}
if (child->reaped && events->seen)
break;
now = monotonic_milliseconds();
if (now < 0)
return (int)now;
if (now >= deadline)
break;
if (child->reaped && !events->seen)
continue;
}
error = reap_timed_out_child(child);
if (error)
return error;
error = drain_events(ring_buffer);
if (error)
return error;
return child_exit_code(child->status);
}
static int report_result(const struct exec_image_inspector_bpf *skel,
const struct event_context *events, int command_exit)
{
struct inspector_stats final_stats = skel->bss->stats;
printf("SUMMARY matched=%llu scheduled=%llu schedule_errors=%llu "
"callbacks=%llu header_errors=%llu path_errors=%llu "
"direct_probes=%llu direct_probe_errors=%llu "
"deferred_probes=%llu deferred_probe_errors=%llu dropped=%llu "
"events=%u command_exit=%d\n",
final_stats.matched, final_stats.scheduled,
final_stats.schedule_errors, final_stats.callbacks,
final_stats.header_errors, final_stats.path_errors,
final_stats.direct_probes, final_stats.direct_probe_errors,
final_stats.deferred_probes, final_stats.deferred_probe_errors,
final_stats.dropped, events->seen, command_exit);
if (!events->seen) {
fprintf(stderr, "no executable image event was observed\n");
return 1;
}
if (command_exit) {
fprintf(stderr, "command exited with status %d\n", command_exit);
return command_exit;
}
return 0;
}
int main(int argc, char **argv)
{
struct exec_image_inspector_bpf *skel = NULL;
struct child_process child = { .release_fd = -1 };
struct event_context events = {};
struct ring_buffer *ring_buffer = NULL;
int command_exit, error, result = 1;
error = parse_args(argc, argv);
if (error) {
usage(argv[0]);
return 2;
}
libbpf_set_print(libbpf_print_fn);
error = start_blocked_child(&child);
if (error) {
fprintf(stderr, "failed to create command process: %s\n",
strerror(-error));
return 1;
}
error = setup_inspector(&child, &events, &skel, &ring_buffer);
if (error)
goto cleanup;
command_exit = wait_for_command(ring_buffer, &child, &events);
if (command_exit < 0)
goto cleanup;
result = report_result(skel, &events, command_exit);
cleanup:
stop_child(&child);
ring_buffer__free(ring_buffer);
exec_image_inspector_bpf__destroy(skel);
return result;
}
```
The loader's flow centers on the blocked child handshake. `start_blocked_child` forks before BPF setup, but the child blocks on a pipe read. The parent already knows the child PID at this point and can write it to `rodata->target_tgid` after opening the skeleton. After attachment, `release_child` writes one byte to the pipe to let the child continue with `execvp`. This handshake eliminates the attach race with short-lived commands.
The `setup_inspector` function opens the skeleton, sets read-only data, loads and attaches the BPF program, and creates the ring buffer reader. Error messages on load failure indicate possible causes to help users troubleshoot kernel version or configuration issues.
`wait_for_command` is the main loop. It first prints a `READY` line indicating the tool is ready, then releases the child, sets a timeout deadline, and enters the poll loop. The loop alternates between calling `ring_buffer__poll` to receive events and `waitpid(WNOHANG)` to check child status. The loop exits when the child is reaped and at least one event is received, or when the timeout is reached. After timeout, `reap_timed_out_child` sends SIGKILL to the child and waits for it to exit.
The same child may install multiple images in succession. For example, running `/bin/sh -c 'exec /bin/true'` causes the shell to exec itself first, then exec `/bin/true`. After the child is reaped, `drain_events` uses zero-timeout poll to collect any remaining events in the ring buffer, ensuring the last image event also reaches user space.
`handle_event` parses ring buffer events and prints formatted output. It converts ELF numeric values to readable names like `ELF64`, `LSB`, `ET_DYN`, `EM_X86_64`, while preserving original values. When `--probe-offset` is set, it also prints a `PROBE` line showing probe results.
## Compilation and execution
Build from source:
```bash
git submodule update --init --recursive
make -C src/54-exec-image-inspector clean
make -C src/54-exec-image-inspector -j2
```
The integration test requires Linux 6.19 or newer. Before running, confirm that `bpf` appears in the active LSM list:
```bash
cat /sys/kernel/security/lsm
```
To add `bpf`, change `lsm=<existing-list>` to `lsm=<existing-list>,bpf` in the kernel boot parameters.
Run tests:
```bash
cd src/54-exec-image-inspector
sudo make test
```
Repository CI only compiles this lesson. Runtime behavior was functionally tested on x86_64 with kernel `7.0.0-rc2+`. The following session output shows the test harness and tool output:
```text
TEST-MISSING matched=0 events=0 command_exit=127
TEST-TIMEOUT matched=1 callbacks=1 events=1 command_exit=137
TEST-REEXEC matched=2 callbacks=2 events=2 command_exit=0 final_path=/usr/bin/true
READY target_tgid=1265 probe_offset=4214784 timeout_ms=3000 command=/tmp/exec-image-inspector-sxm3lumw/exec_fixture_image
EXEC pid=1265 tgid=1265 comm=exec_fixture_im path=/tmp/exec-image-inspector-sxm3lumw/exec_fixture_image is_elf=1 class=ELF64 endian=LSB type=ET_DYN(3) machine=EM_X86_64(62) header_error=0 path_error=0 latency_us=37
PROBE offset=4214784 direct_error=-14 deferred_error=0 bytes=454950524f424521
exec fixture completed
SUMMARY matched=1 scheduled=1 schedule_errors=0 callbacks=1 header_errors=0 path_errors=0 direct_probes=1 direct_probe_errors=1 deferred_probes=1 deferred_probe_errors=0 dropped=0 events=1 command_exit=0
PASS: missing-command, timeout cleanup, re-exec drain, ELF decode, and deferred file read succeeded
```
The first three test records cover important boundaries: a missing command exits with status 127, at which point the committed-exec hook has not yet triggered, so `matched=0` and `events=0`; the timeout case produces an exec event then exceeds the test deadline, is terminated by SIGKILL, and reaped with status 137; the successive exec case drains two image events and confirms `/usr/bin/true` appears last.
The `EXEC` line shows the installed image and ELF header parsing results. The test program flushes and evicts the marker page before exec, so the direct read in the hook returns `-EFAULT` (`-14`), while the task-work callback successfully returns `454950524f424521` (the hexadecimal bytes for `EIPROBE!`). This verifies that non-sleepable context cannot read cold pages, while sleepable context can.
To inspect other commands, omit the test-program-specific probe offset:
```bash
sudo ./exec_image_inspector --timeout-ms 3000 -- /bin/true
```
Command-line format:
```text
exec_image_inspector [--probe-offset BYTES] [--timeout-ms MS] [--verbose] -- COMMAND [ARG...]
```
`--timeout-ms` accepts 100 to 60000 milliseconds, default is 5000 milliseconds; the loader terminates and reaps the command after the deadline. `--probe-offset` compares 8-byte direct and deferred reads at the specified location, used to verify cold page scenarios. `--verbose` outputs libbpf diagnostics for troubleshooting load or attach issues. `--` explicitly ends inspector option parsing; all subsequent arguments belong to the observed command.
### Environment requirements
| Requirement | Details |
|---|---|
| Kernel version | Linux 6.19+ (BPF task work introduced in 6.18, file-backed dynptr introduced in 6.19) |
| Kernel configuration | `CONFIG_BPF=y`, `CONFIG_BPF_SYSCALL=y`, `CONFIG_BPF_JIT=y`, `CONFIG_BPF_LSM=y`, `CONFIG_SECURITY=y`, `CONFIG_DEBUG_INFO_BTF=y` |
| Active LSM | `/sys/kernel/security/lsm` contains `bpf` |
| Architecture | Tested on x86_64 |
| Privilege | root |
## Scope
This tool observes one direct child that it creates, and a single map slot is sufficient for this CLI scenario. To extend it to concurrent services, you could allocate per-task state, set admission limits, and reclaim pending callbacks. Timeout cleanup sends SIGKILL to the child; callers can add process-group management and external signal handling to cover more complex scenarios.
## Summary
This tutorial demonstrates how to use BPF task work and file-backed dynptr to inspect the executable image actually installed after exec. Compared to reading `/proc/<pid>/exe` or obtaining paths in tracepoints, this approach can read file contents in a sleepable context, even when target bytes are on cold pages.
The core idea of this tool is to separate the moment of identifying the target process (non-sleepable LSM hook) from the moment of reading file contents (sleepable task work callback). The blocked-child handshake eliminates attach races, bounded command execution and final drain ensure all events reach user space, and explicit resource release guarantees complete cleanup. These kernel features combine into a reproducible single-command tool while leaving room for extension to concurrent scenarios.
> If you want to learn more about eBPF, check out our tutorial repository at <https://github.com/eunomia-bpf/bpf-developer-tutorial> or visit our website at <https://eunomia.dev/tutorials/>.
## References
- [BPF task-work plumbing](https://github.com/torvalds/linux/commit/5c8fd7e2b5b0a527cf88740da122166695382a78)
- [bpf_task_work_schedule_signal kfunc](https://github.com/torvalds/linux/commit/38aa7003e369802f81a078f6673d10d97013f04f)
- [File-backed dynptr plumbing](https://github.com/torvalds/linux/commit/8d8771dc03e48300e80b43744dd3c320ccaf746a)
- [File dynptr kfuncs and helpers](https://github.com/torvalds/linux/commit/e3e36edb1b8f0e6975c68acd2e1202ec0397fd75)
- [Sleepable file-dynptr dispatch](https://github.com/torvalds/linux/commit/2c52e8943a437af6093d8b0f0920f1764f0e5f64)
- [Kernel kfunc documentation](https://github.com/torvalds/linux/blob/v7.1/Documentation/bpf/kfuncs.rst)
- [File-reader BPF selftest](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/progs/file_reader.c)
- [BPF task-work selftest](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/progs/task_work.c)

View File

@@ -0,0 +1,999 @@
# eBPF 教程:检查 exec 后真正安装的可执行镜像
假设某个容器中运行着一系列嵌套的包装脚本,你从 `ps` 或 `/proc` 看到的命令行是 `/usr/bin/wrapper.sh --config /etc/app.conf`,但实际执行业务逻辑的可能是三层深处的 Python 解释器或某个动态链接的二进制。排查故障时你需要知道内核最终安装的是哪个可执行镜像、它是什么架构、用什么字节序,而不只是命令行参数。
本教程展示如何使用 BPF task work 和 file-backed dynptr 实现这一目标。工具挂载 LSM hook 观察 `bprm_committed_creds` 时刻,在凭据提交后为目标进程安排 task work 回调,回调在可睡眠上下文中重新获取已安装的可执行文件、解析路径、读取 ELF 头部,并把结果通过 ring buffer 发送给用户态。
> 完整源码:<https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/54-exec-image-inspector>
## 为什么需要 BPF task work 和 file dynptr
传统的可执行文件检查方案都有明显的局限性。
最直接的想法是读取 `/proc/<pid>/exe` 符号链接,但这需要在进程存活期间完成,短命进程可能在你读取之前就已经退出。而且 `/proc` 访问发生在用户态,与内核的 exec 事件之间存在竞态窗口,同一进程可能已经执行了多次 `execve`。
使用 tracepoint 或 kprobe 观察 `sched_process_exec` 可以捕获 exec 事件,但这些 hook 运行在不可睡眠上下文中。你可以获取文件路径,但如果需要读取文件内容来验证 ELF 头部或检查嵌入的元数据,问题就来了:file-backed dynptr 的 `bpf_dynptr_read` 在目标字节位于冷页时会触发缺页操作,而缺页需要睡眠才能完成。不可睡眠上下文中的 BPF 程序无法等待 I/O。
BPF task work 才能真正解决这个问题。Linux 6.18 引入了 `bpf_task_work_schedule_signal` kfunc,允许 BPF 程序为目标 task 安排回调,内核会在安全的可睡眠上下文中执行这个回调。Linux 6.19 引入了 file-backed dynptr,提供验证器跟踪的文件数据访问方式。把这两个功能组合起来,你可以在 LSM hook 中识别目标进程、记录时间戳,然后安排 task work;回调在可睡眠上下文中执行,可以重新获取 task 已安装的可执行文件、通过 dynptr 读取任意位置的内容(包括冷页),最后把完整的检查结果发送给用户态。
这就是 `exec_image_inspector` 的设计思路:LSM hook 选定子进程并调度 task work,可睡眠回调完成实际的文件读取和解析工作。
## BPF task work 与 file dynptr 的工作原理
BPF task work 是一种延迟执行机制。当你在 BPF 程序中调用 `bpf_task_work_schedule_signal(task, work, map, callback)` 时,内核会把回调关联到指定的 task。回调不会立即执行,而是在该 task 返回用户态前的某个安全点执行,此时上下文允许睡眠。
`struct bpf_task_work` 是一个不透明的 64 字节结构,BPF 程序只需要为它分配存储空间,内核负责管理其内部状态。本工具使用单元素 ARRAY map 存储 `struct exec_work`,其中包含 `bpf_task_work` 存储、调度时间戳和直接探测结果。
File-backed dynptr 则提供了验证器跟踪的文件数据访问接口。`bpf_dynptr_from_file(file, flags, dynptr)` 从文件创建 dynptr,`bpf_dynptr_read(dst, len, dynptr, offset, flags)` 读取指定偏移处的内容。dynptr 持有内部状态,因此所有创建 dynptr 的路径都必须调用 `bpf_dynptr_file_discard` 来释放,包括失败分支。
在不可睡眠上下文中,dynptr 只能读取已经在页缓存中的内容,访问冷页会返回 `-EFAULT`。在可睡眠上下文中,dynptr 可以触发缺页操作并等待 I/O 完成。这正是 task work 回调的价值所在。
## 工具的整体流程
用户态启动后先 fork 目标命令,但子进程会在 pipe 上阻塞等待释放信号。这样父进程就能在子进程执行 `execvp` 之前知道它的 PID(也是 TGID),可以把这个值写入 BPF 程序的只读数据段作为过滤条件。阻塞子进程消除了短命命令带来的 attach 竞态,让整个示例保持自包含。
父进程打开 BPF skeleton、设置 `target_tgid` 和可选的 `probe_offset`、加载并挂载 LSM 程序、创建 ring buffer reader,随后向 pipe 写入释放字节让子进程执行 `execvp`。
子进程执行 exec 后,凭据提交时 `lsm/bprm_committed_creds` hook 触发。BPF 程序比较当前 TGID 与 `target_tgid`,不匹配则直接返回。命中目标后程序记录调度时间戳,如果设置了 `--probe-offset` 则先尝试在当前上下文直接读取(用于验证冷页场景),然后调用 `bpf_task_work_schedule_signal` 安排回调。
回调 `inspect_executable` 在该 task 的可睡眠上下文中执行。它调用 `bpf_get_task_exe_file` 获取 task 当前安装的可执行文件(返回带引用的 `struct file`,必须用 `bpf_put_file` 释放),用 `bpf_path_d_path` 解析路径,用 file dynptr 读取 64 字节 ELF 头部和可选的标记位置。读取成功后解析 ELF 魔数、class、data、type 和 machine 字段。最后把所有信息打包成 ring buffer 事件发送给用户态。
用户态边轮询 ring buffer 边用 `waitpid(WNOHANG)` 检查子进程状态。子进程回收且至少收到一条事件时结束循环,或者等到超时。同一子进程可能连续安装多个镜像(例如 `/bin/sh -c 'exec /bin/true'`),子进程回收后还要执行 `drain_events` 取尽 ring buffer 中的剩余事件。
![exec 镜像检查器数据流](https://github.com/eunomia-bpf/bpf-developer-tutorial/raw/main/src/54-exec-image-inspector/exec-image-flow.png)
## 代码实现
本工具由四个文件组成:共享头文件定义事件和统计结构、兼容性头文件声明新内核 kfunc、BPF 程序实现 hook 和回调、用户空间加载器管理生命周期。
### 共享头文件
`exec_image_inspector.h` 定义了 BPF 和用户空间共享的 ring buffer 事件结构和统计结构。
```c
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __EXEC_IMAGE_INSPECTOR_H
#define __EXEC_IMAGE_INSPECTOR_H
#define EXEC_COMM_LEN 16
#define EXEC_PATH_LEN 256
#define EXEC_PROBE_LEN 8
struct exec_event {
unsigned int pid;
unsigned int tgid;
unsigned char is_elf;
unsigned char elf_class;
unsigned char elf_data;
unsigned char reserved;
unsigned short elf_type;
unsigned short elf_machine;
int header_error;
int path_error;
int direct_probe_error;
int deferred_probe_error;
unsigned long long latency_ns;
unsigned long long probe_offset;
char comm[EXEC_COMM_LEN];
char path[EXEC_PATH_LEN];
unsigned char probe_bytes[EXEC_PROBE_LEN];
};
struct inspector_stats {
unsigned long long matched;
unsigned long long scheduled;
unsigned long long schedule_errors;
unsigned long long callbacks;
unsigned long long header_errors;
unsigned long long path_errors;
unsigned long long direct_probes;
unsigned long long direct_probe_errors;
unsigned long long deferred_probes;
unsigned long long deferred_probe_errors;
unsigned long long dropped;
};
#endif /* __EXEC_IMAGE_INSPECTOR_H */
```
`exec_event` 包含了检查一次 exec 所需的全部信息:PID、TGID、进程名、路径、ELF 解析结果、探测结果、回调延迟和各类错误代码。`inspector_stats` 累计匹配、调度、回调、错误和丢弃计数,用户态在结束时读取 BSS 段并报告。
### 兼容性声明
Linux 6.18 引入 BPF task work,6.19 引入 file-backed dynptr。仓库当前生成的 UAPI 与 BTF 头文件早于这些功能,因此本工具把缺失声明放在局部头文件 `bpf_experimental.h` 中。
```c
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __EXEC_IMAGE_INSPECTOR_BPF_EXPERIMENTAL_H
#define __EXEC_IMAGE_INSPECTOR_BPF_EXPERIMENTAL_H
/*
* These Linux 6.18/6.19 declarations are not present in the repository's
* older generated UAPI and vmlinux headers. Keep them local until those
* vendored headers are regenerated.
*/
struct bpf_task_work {
__u64 opaque;
} __attribute__((aligned(8)));
typedef int (*bpf_task_work_callback_t)(struct bpf_map *map, void *key,
void *value);
extern int bpf_task_work_schedule_signal(struct task_struct *task,
struct bpf_task_work *work,
void *map__map,
bpf_task_work_callback_t callback) __ksym;
extern struct file *bpf_get_task_exe_file(struct task_struct *task) __ksym;
extern void bpf_put_file(struct file *file) __ksym;
extern int bpf_path_d_path(const struct path *path, char *buf,
__u64 buf__sz) __ksym;
extern int bpf_dynptr_from_file(struct file *file, __u32 flags,
struct bpf_dynptr *ptr__uninit) __ksym;
extern int bpf_dynptr_file_discard(struct bpf_dynptr *dynptr) __ksym;
#endif /* __EXEC_IMAGE_INSPECTOR_BPF_EXPERIMENTAL_H */
```
这个文件声明了 `bpf_task_work_schedule_signal`、`bpf_get_task_exe_file`、`bpf_put_file`、`bpf_path_d_path`、`bpf_dynptr_from_file` 和 `bpf_dynptr_file_discard` 等 kfunc。等仓库集成的 vmlinux 头文件包含这些声明后即可移除此文件。
### BPF 程序
`exec_image_inspector.bpf.c` 实现了 LSM hook 和 task work 回调。
```c
// SPDX-License-Identifier: GPL-2.0
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include "bpf_experimental.h"
#include "exec_image_inspector.h"
char LICENSE[] SEC("license") = "GPL";
#define ENOENT 2
#define EI_CLASS 4
#define EI_DATA 5
#define ELFCLASS32 1
#define ELFCLASS64 2
#define ELFDATA2LSB 1
#define ELFDATA2MSB 2
const volatile __u32 target_tgid;
const volatile __u32 probe_offset;
struct inspector_stats stats;
struct {
__uint(type, BPF_MAP_TYPE_RINGBUF);
__uint(max_entries, 256 * 1024);
} events SEC(".maps");
struct exec_work {
__u64 scheduled_ns;
int direct_probe_error;
struct bpf_task_work work;
};
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 1);
__type(key, __u32);
__type(value, struct exec_work);
} pending SEC(".maps");
static __u16 read_elf_u16(const unsigned char *header, int offset, __u8 data)
{
if (data == ELFDATA2MSB)
return ((__u16)header[offset] << 8) | header[offset + 1];
return header[offset] | ((__u16)header[offset + 1] << 8);
}
static int probe_file_without_sleep(struct file *file)
{
unsigned char sample[EXEC_PROBE_LEN];
struct bpf_dynptr dynptr;
int err;
if (!probe_offset)
return 0;
__sync_fetch_and_add(&stats.direct_probes, 1);
if (!file) {
err = -ENOENT;
goto record;
}
err = bpf_dynptr_from_file(file, 0, &dynptr);
if (err) {
bpf_dynptr_file_discard(&dynptr);
goto record;
}
err = bpf_dynptr_read(sample, sizeof(sample), &dynptr, probe_offset, 0);
bpf_dynptr_file_discard(&dynptr);
record:
if (err)
__sync_fetch_and_add(&stats.direct_probe_errors, 1);
return err;
}
static int inspect_executable(struct bpf_map *map, void *key, void *value)
{
unsigned char header[64] = {};
struct exec_work *work = value;
struct exec_event event = {};
struct task_struct *task;
struct bpf_dynptr dynptr;
struct file *file;
__u64 pid_tgid;
int err;
(void)map;
(void)key;
__sync_fetch_and_add(&stats.callbacks, 1);
pid_tgid = bpf_get_current_pid_tgid();
event.pid = (__u32)pid_tgid;
event.tgid = pid_tgid >> 32;
event.latency_ns = bpf_ktime_get_ns() - work->scheduled_ns;
event.direct_probe_error = work->direct_probe_error;
event.probe_offset = probe_offset;
bpf_get_current_comm(event.comm, sizeof(event.comm));
task = bpf_get_current_task_btf();
file = bpf_get_task_exe_file(task);
if (!file) {
event.header_error = -ENOENT;
__sync_fetch_and_add(&stats.header_errors, 1);
if (probe_offset) {
event.deferred_probe_error = -ENOENT;
__sync_fetch_and_add(&stats.deferred_probes, 1);
__sync_fetch_and_add(&stats.deferred_probe_errors, 1);
}
goto emit;
}
err = bpf_path_d_path(&file->f_path, event.path, sizeof(event.path));
if (err < 0) {
event.path_error = err;
__sync_fetch_and_add(&stats.path_errors, 1);
}
err = bpf_dynptr_from_file(file, 0, &dynptr);
if (err) {
bpf_dynptr_file_discard(&dynptr);
event.header_error = err;
__sync_fetch_and_add(&stats.header_errors, 1);
if (probe_offset) {
event.deferred_probe_error = err;
__sync_fetch_and_add(&stats.deferred_probes, 1);
__sync_fetch_and_add(&stats.deferred_probe_errors, 1);
}
goto put_file;
}
err = bpf_dynptr_read(header, sizeof(header), &dynptr, 0, 0);
if (err) {
event.header_error = err;
__sync_fetch_and_add(&stats.header_errors, 1);
}
if (probe_offset) {
__sync_fetch_and_add(&stats.deferred_probes, 1);
err = bpf_dynptr_read(event.probe_bytes, sizeof(event.probe_bytes),
&dynptr, probe_offset, 0);
event.deferred_probe_error = err;
if (err)
__sync_fetch_and_add(&stats.deferred_probe_errors, 1);
}
bpf_dynptr_file_discard(&dynptr);
if (!event.header_error && header[0] == 0x7f && header[1] == 'E' &&
header[2] == 'L' && header[3] == 'F') {
event.is_elf = 1;
event.elf_class = header[EI_CLASS];
event.elf_data = header[EI_DATA];
event.elf_type = read_elf_u16(header, 16, event.elf_data);
event.elf_machine = read_elf_u16(header, 18, event.elf_data);
}
put_file:
bpf_put_file(file);
emit:
if (bpf_ringbuf_output(&events, &event, sizeof(event), 0))
__sync_fetch_and_add(&stats.dropped, 1);
return 0;
}
SEC("lsm/bprm_committed_creds")
void BPF_PROG(schedule_exec_inspection, struct linux_binprm *bprm)
{
struct task_struct *task;
struct exec_work *work;
__u64 pid_tgid;
__u32 key = 0, tgid;
int err;
pid_tgid = bpf_get_current_pid_tgid();
tgid = pid_tgid >> 32;
if (target_tgid && tgid != target_tgid)
return;
__sync_fetch_and_add(&stats.matched, 1);
work = bpf_map_lookup_elem(&pending, &key);
if (!work) {
__sync_fetch_and_add(&stats.schedule_errors, 1);
return;
}
work->scheduled_ns = bpf_ktime_get_ns();
work->direct_probe_error = probe_file_without_sleep(bprm->file);
task = bpf_get_current_task_btf();
err = bpf_task_work_schedule_signal(task, &work->work, &pending,
inspect_executable);
if (err) {
__sync_fetch_and_add(&stats.schedule_errors, 1);
return;
}
__sync_fetch_and_add(&stats.scheduled, 1);
}
```
程序的入口是 `SEC("lsm/bprm_committed_creds")` 声明的 `schedule_exec_inspection` 函数。`bprm_committed_creds` hook 在新可执行文件的凭据提交后运行,此时 `bprm->file` 指向本次 exec 涉及的镜像。这个 hook 本身不可睡眠,但我们可以在这里识别目标进程并安排 task work。
两个 `const volatile` 变量 `target_tgid` 和 `probe_offset` 位于 `.rodata` 段,用户态在 `open()` 之后、`load()` 之前写入值,验证器将它们视为编译期常量。程序首先比较当前 TGID 与 `target_tgid`,不匹配则直接返回,避免观察无关进程的 exec。
命中目标后程序从单元素 `pending` ARRAY map 查找 key 0 得到 `struct exec_work`。这个结构保存调度时间戳、直接探测结果和 `struct bpf_task_work` 存储。本工具一次调用只观察一个子进程,一个槽位已经足够。设置 `--probe-offset` 时,`probe_file_without_sleep` 在当前不可睡眠上下文中尝试从 `bprm->file` 读取指定位置的 8 字节。测试程序会把标记放到冷页位置,验证此时直接读取返回 `-EFAULT`。
hook 保存时间戳和直接读取结果后调用 `bpf_task_work_schedule_signal`,传入当前 task、work 存储、map 指针与 `inspect_executable` 回调函数指针。内核持有回调执行所需的引用,回调会在该 task 返回用户态前的可睡眠上下文中执行。
`inspect_executable` 是 task work 回调函数,签名为 `int callback(struct bpf_map *map, void *key, void *value)`。它首先计算回调延迟(当前时间减去调度时间戳)用于诊断,填充 PID、TGID、进程名和直接探测结果到事件结构。然后调用 `bpf_get_task_exe_file` 获取 task 已安装的可执行文件。这个 kfunc 返回带引用的 `struct file`,每次成功获取都必须用 `bpf_put_file` 释放。
回调先用 `bpf_path_d_path` 解析文件路径,再用 `bpf_dynptr_from_file` 创建 dynptr。因为现在处于可睡眠上下文,`bpf_dynptr_read` 可以触发缺页操作并等待 I/O 完成。回调只读取 64 字节 ELF 头部和可选的 8 字节标记。头部读取成功后程序检查 4 字节 ELF 魔数 `\x7fELF`,再解析 `EI_CLASS`(32 位或 64 位)、`EI_DATA`(小端或大端)、`e_type`(可执行文件或共享对象)和 `e_machine`(架构)。`read_elf_u16` 辅助函数根据字节序正确解析 16 位字段。
所有创建 dynptr 的路径都必须调用 `bpf_dynptr_file_discard` 释放其内部状态,包括 helper 失败的分支。最后 `bpf_ringbuf_output` 把事件发送给用户态,发送失败时递增 `dropped` 计数器。
### 用户空间加载器
`exec_image_inspector.c` 负责解析命令行、协调阻塞子进程、加载 BPF、格式化事件并清理资源。
```c
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <errno.h>
#include <getopt.h>
#include <limits.h>
#include <signal.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <time.h>
#include <unistd.h>
#include <bpf/libbpf.h>
#include "exec_image_inspector.h"
#include "exec_image_inspector.skel.h"
struct environment {
unsigned long long probe_offset;
unsigned int timeout_ms;
bool verbose;
char **command;
};
struct child_process {
pid_t pid;
int release_fd;
bool released;
bool reaped;
int status;
};
struct event_context {
unsigned int seen;
};
static struct environment env = {
.timeout_ms = 5000,
};
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s [--probe-offset BYTES] [--timeout-ms MS] [--verbose] "
"-- COMMAND [ARG...]\n\n"
"Inspect the executable image installed by one command.\n\n"
"Options:\n"
" -p, --probe-offset BYTES also compare direct/deferred file reads\n"
" -t, --timeout-ms MS bound the command, 100-60000 "
"(default: 5000)\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_u64(const char *value, unsigned long long maximum,
unsigned long long *result)
{
char *end = NULL;
unsigned long long parsed;
errno = 0;
parsed = strtoull(value, &end, 10);
if (errno || end == value || *end || parsed > maximum)
return -EINVAL;
*result = parsed;
return 0;
}
static int parse_probe_offset(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, UINT_MAX - EXEC_PROBE_LEN, &parsed)) {
fprintf(stderr, "invalid probe offset: %s\n", value);
return -EINVAL;
}
env.probe_offset = parsed;
return 0;
}
static int parse_timeout(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 60000, &parsed) || parsed < 100) {
fprintf(stderr, "invalid timeout in milliseconds: %s\n", value);
return -EINVAL;
}
env.timeout_ms = parsed;
return 0;
}
static int parse_option(int option, const char *program)
{
switch (option) {
case 'p':
return parse_probe_offset(optarg);
case 't':
return parse_timeout(optarg);
case 'v':
env.verbose = true;
return 0;
case 'h':
usage(program);
exit(0);
default:
return -EINVAL;
}
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "probe-offset", required_argument, NULL, 'p' },
{ "timeout-ms", required_argument, NULL, 't' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
int error, option;
while ((option = getopt_long(argc, argv, "+p:t:vh", options, NULL)) != -1) {
error = parse_option(option, argv[0]);
if (error)
return error;
}
if (optind == argc) {
fprintf(stderr, "COMMAND is required\n");
return -EINVAL;
}
env.command = &argv[optind];
return 0;
}
static long long monotonic_milliseconds(void)
{
struct timespec timestamp;
if (clock_gettime(CLOCK_MONOTONIC, &timestamp))
return -errno;
return timestamp.tv_sec * 1000LL + timestamp.tv_nsec / 1000000;
}
static int start_blocked_child(struct child_process *child)
{
int pipe_fds[2];
pid_t pid;
if (pipe(pipe_fds))
return -errno;
pid = fork();
if (pid < 0) {
int error = -errno;
close(pipe_fds[0]);
close(pipe_fds[1]);
return error;
}
if (pid == 0) {
char release;
ssize_t count;
close(pipe_fds[1]);
do {
count = read(pipe_fds[0], &release, sizeof(release));
} while (count < 0 && errno == EINTR);
close(pipe_fds[0]);
if (count != sizeof(release))
_exit(126);
/* Intentional argv execution; no shell parses the supplied arguments. */
execvp(env.command[0], env.command); /* Flawfinder: ignore */
fprintf(stderr, "failed to execute %s: %s\n", env.command[0],
strerror(errno));
_exit(127);
}
close(pipe_fds[0]);
child->pid = pid;
child->release_fd = pipe_fds[1];
return 0;
}
static int release_child(struct child_process *child)
{
char release = 1;
ssize_t count;
do {
count = write(child->release_fd, &release, sizeof(release));
} while (count < 0 && errno == EINTR);
close(child->release_fd);
child->release_fd = -1;
if (count != sizeof(release))
return count < 0 ? -errno : -EIO;
child->released = true;
return 0;
}
static int child_exit_code(int status)
{
if (WIFEXITED(status))
return WEXITSTATUS(status);
if (WIFSIGNALED(status))
return 128 + WTERMSIG(status);
return 125;
}
static int reap_child(struct child_process *child, int options)
{
pid_t result;
if (child->reaped)
return 1;
do {
result = waitpid(child->pid, &child->status, options);
} while (result < 0 && errno == EINTR);
if (result < 0)
return -errno;
if (result == 0)
return 0;
child->reaped = true;
return 1;
}
static int drain_events(struct ring_buffer *ring_buffer)
{
int error;
for (;;) {
error = ring_buffer__poll(ring_buffer, 0);
if (error == -EINTR)
continue;
if (error < 0) {
fprintf(stderr, "ring-buffer drain failed: %s\n",
strerror(-error));
return error;
}
if (!error)
return 0;
}
}
static const char *elf_class_name(unsigned char value)
{
switch (value) {
case 1:
return "ELF32";
case 2:
return "ELF64";
default:
return "UNKNOWN";
}
}
static const char *elf_data_name(unsigned char value)
{
switch (value) {
case 1:
return "LSB";
case 2:
return "MSB";
default:
return "UNKNOWN";
}
}
static const char *elf_type_name(unsigned short value)
{
switch (value) {
case 2:
return "ET_EXEC";
case 3:
return "ET_DYN";
default:
return "OTHER";
}
}
static const char *elf_machine_name(unsigned short value)
{
switch (value) {
case 3:
return "EM_386";
case 62:
return "EM_X86_64";
case 183:
return "EM_AARCH64";
default:
return "OTHER";
}
}
static int handle_event(void *context, void *data, size_t size)
{
const struct exec_event *event = data;
struct event_context *events = context;
unsigned int index;
if (size < sizeof(*event)) {
fprintf(stderr, "short ring-buffer event: %zu bytes\n", size);
return 0;
}
events->seen++;
printf("EXEC pid=%u tgid=%u comm=%.*s path=%.*s is_elf=%u "
"class=%s endian=%s type=%s(%u) machine=%s(%u) "
"header_error=%d path_error=%d latency_us=%llu\n",
event->pid, event->tgid, EXEC_COMM_LEN, event->comm,
EXEC_PATH_LEN, event->path, event->is_elf,
elf_class_name(event->elf_class), elf_data_name(event->elf_data),
elf_type_name(event->elf_type), event->elf_type,
elf_machine_name(event->elf_machine), event->elf_machine,
event->header_error, event->path_error,
event->latency_ns / 1000);
if (event->probe_offset) {
printf("PROBE offset=%llu direct_error=%d deferred_error=%d bytes=",
event->probe_offset, event->direct_probe_error,
event->deferred_probe_error);
for (index = 0; index < EXEC_PROBE_LEN; index++)
printf("%02x", event->probe_bytes[index]);
putchar('\n');
}
fflush(stdout);
return 0;
}
static void stop_child(struct child_process *child)
{
if (child->reaped || child->pid <= 0)
return;
if (!child->released && child->release_fd >= 0) {
close(child->release_fd);
child->release_fd = -1;
} else {
kill(child->pid, SIGKILL);
}
(void)reap_child(child, 0);
}
static int setup_inspector(const struct child_process *child,
struct event_context *events,
struct exec_image_inspector_bpf **skeleton,
struct ring_buffer **ring_buffer)
{
struct exec_image_inspector_bpf *skel;
struct ring_buffer *ring;
int error;
skel = exec_image_inspector_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return -ENOMEM;
}
*skeleton = skel;
skel->rodata->target_tgid = child->pid;
skel->rodata->probe_offset = env.probe_offset;
error = exec_image_inspector_bpf__load(skel);
if (error) {
fprintf(stderr, "failed to load BPF object: %s\n", strerror(-error));
return error;
}
error = exec_image_inspector_bpf__attach(skel);
if (error) {
fprintf(stderr, "failed to attach bprm_committed_creds LSM hook: %s\n",
strerror(-error));
return error;
}
ring = ring_buffer__new(bpf_map__fd(skel->maps.events), handle_event,
events, NULL);
if (!ring) {
fprintf(stderr, "failed to create ring buffer: %s\n", strerror(errno));
return errno ? -errno : -ENOMEM;
}
*ring_buffer = ring;
return 0;
}
static int reap_timed_out_child(struct child_process *child)
{
int error;
if (child->reaped)
return 0;
fprintf(stderr, "command exceeded timeout; sending SIGKILL\n");
kill(child->pid, SIGKILL);
error = reap_child(child, 0);
if (error < 0) {
fprintf(stderr, "waitpid after timeout failed: %s\n",
strerror(-error));
return error;
}
return 0;
}
static int wait_for_command(struct ring_buffer *ring_buffer,
struct child_process *child,
const struct event_context *events)
{
long long deadline, now;
int error;
printf("READY target_tgid=%d probe_offset=%llu timeout_ms=%u command=%s\n",
child->pid, env.probe_offset, env.timeout_ms, env.command[0]);
fflush(stdout);
error = release_child(child);
if (error) {
fprintf(stderr, "failed to release command process: %s\n",
strerror(-error));
return error;
}
now = monotonic_milliseconds();
if (now < 0) {
fprintf(stderr, "failed to read monotonic clock: %s\n",
strerror((int)-now));
return (int)now;
}
deadline = now + env.timeout_ms;
for (;;) {
error = ring_buffer__poll(ring_buffer, 50);
if (error == -EINTR)
continue;
if (error < 0) {
fprintf(stderr, "ring-buffer poll failed: %s\n", strerror(-error));
return error;
}
error = reap_child(child, WNOHANG);
if (error < 0) {
fprintf(stderr, "waitpid failed: %s\n", strerror(-error));
return error;
}
if (child->reaped && events->seen)
break;
now = monotonic_milliseconds();
if (now < 0)
return (int)now;
if (now >= deadline)
break;
if (child->reaped && !events->seen)
continue;
}
error = reap_timed_out_child(child);
if (error)
return error;
error = drain_events(ring_buffer);
if (error)
return error;
return child_exit_code(child->status);
}
static int report_result(const struct exec_image_inspector_bpf *skel,
const struct event_context *events, int command_exit)
{
struct inspector_stats final_stats = skel->bss->stats;
printf("SUMMARY matched=%llu scheduled=%llu schedule_errors=%llu "
"callbacks=%llu header_errors=%llu path_errors=%llu "
"direct_probes=%llu direct_probe_errors=%llu "
"deferred_probes=%llu deferred_probe_errors=%llu dropped=%llu "
"events=%u command_exit=%d\n",
final_stats.matched, final_stats.scheduled,
final_stats.schedule_errors, final_stats.callbacks,
final_stats.header_errors, final_stats.path_errors,
final_stats.direct_probes, final_stats.direct_probe_errors,
final_stats.deferred_probes, final_stats.deferred_probe_errors,
final_stats.dropped, events->seen, command_exit);
if (!events->seen) {
fprintf(stderr, "no executable image event was observed\n");
return 1;
}
if (command_exit) {
fprintf(stderr, "command exited with status %d\n", command_exit);
return command_exit;
}
return 0;
}
int main(int argc, char **argv)
{
struct exec_image_inspector_bpf *skel = NULL;
struct child_process child = { .release_fd = -1 };
struct event_context events = {};
struct ring_buffer *ring_buffer = NULL;
int command_exit, error, result = 1;
error = parse_args(argc, argv);
if (error) {
usage(argv[0]);
return 2;
}
libbpf_set_print(libbpf_print_fn);
error = start_blocked_child(&child);
if (error) {
fprintf(stderr, "failed to create command process: %s\n",
strerror(-error));
return 1;
}
error = setup_inspector(&child, &events, &skel, &ring_buffer);
if (error)
goto cleanup;
command_exit = wait_for_command(ring_buffer, &child, &events);
if (command_exit < 0)
goto cleanup;
result = report_result(skel, &events, command_exit);
cleanup:
stop_child(&child);
ring_buffer__free(ring_buffer);
exec_image_inspector_bpf__destroy(skel);
return result;
}
```
加载器的流程围绕阻塞子进程握手展开。`start_blocked_child` 在 BPF 设置完成前 fork,但子进程会阻塞在 pipe 读取上。父进程此时已经知道子进程 PID,可以在打开 skeleton 后把它写入 `rodata->target_tgid`。挂载完成后 `release_child` 向 pipe 写入一个字节让子进程继续执行 `execvp`。这个握手消除了短命命令带来的 attach 竞态。
`setup_inspector` 函数打开 skeleton、设置只读数据、加载并挂载 BPF 程序、创建 ring buffer reader。加载失败时的错误信息会指出可能的原因,帮助用户排查内核版本或配置问题。
`wait_for_command` 是主循环。它先打印 `READY` 行表示工具已就绪,然后释放子进程、设置超时 deadline、进入 poll 循环。循环中交替调用 `ring_buffer__poll` 接收事件和 `waitpid(WNOHANG)` 检查子进程状态。子进程回收且至少收到一条事件时结束循环,或者等到超时。超时后 `reap_timed_out_child` 向子进程发送 SIGKILL 并等待其退出。
同一子进程可能连续安装多个镜像。例如运行 `/bin/sh -c 'exec /bin/true'` 时,shell 先 exec 自身,然后 exec `/bin/true`。子进程回收后 `drain_events` 用零超时 poll 取尽 ring buffer 中的剩余事件,确保最后一条镜像事件也能到达用户态。
`handle_event` 解析 ring buffer 事件并打印格式化输出。它把 ELF 数值转换成可读名称如 `ELF64`、`LSB`、`ET_DYN`、`EM_X86_64`,同时保留原始数值。设置 `--probe-offset` 时还会打印 `PROBE` 行显示探测结果。
## 编译与运行
从源码构建:
```bash
git submodule update --init --recursive
make -C src/54-exec-image-inspector clean
make -C src/54-exec-image-inspector -j2
```
集成测试需要 Linux 6.19 或更新版本,执行前请先确认活动 LSM 列表中包含 `bpf`:
```bash
cat /sys/kernel/security/lsm
```
要添加 `bpf`,请在内核启动参数中把 `lsm=<existing-list>` 改成 `lsm=<existing-list>,bpf`。
运行测试:
```bash
cd src/54-exec-image-inspector
sudo make test
```
仓库 CI 只编译本课。运行时行为在 x86_64 上通过功能测试,内核版本 `7.0.0-rc2+`。下面的会话记录给出测试框架和工具输出:
```text
TEST-MISSING matched=0 events=0 command_exit=127
TEST-TIMEOUT matched=1 callbacks=1 events=1 command_exit=137
TEST-REEXEC matched=2 callbacks=2 events=2 command_exit=0 final_path=/usr/bin/true
READY target_tgid=1265 probe_offset=4214784 timeout_ms=3000 command=/tmp/exec-image-inspector-sxm3lumw/exec_fixture_image
EXEC pid=1265 tgid=1265 comm=exec_fixture_im path=/tmp/exec-image-inspector-sxm3lumw/exec_fixture_image is_elf=1 class=ELF64 endian=LSB type=ET_DYN(3) machine=EM_X86_64(62) header_error=0 path_error=0 latency_us=37
PROBE offset=4214784 direct_error=-14 deferred_error=0 bytes=454950524f424521
exec fixture completed
SUMMARY matched=1 scheduled=1 schedule_errors=0 callbacks=1 header_errors=0 path_errors=0 direct_probes=1 direct_probe_errors=1 deferred_probes=1 deferred_probe_errors=0 dropped=0 events=1 command_exit=0
PASS: missing-command, timeout cleanup, re-exec drain, ELF decode, and deferred file read succeeded
```
前三条测试记录覆盖重要边界:缺失命令以状态 127 退出,此时 committed-exec hook 尚未触发,因此 `matched=0`、`events=0`;超时用例产生 exec 事件后超过测试时限,由 SIGKILL 终止并以状态 137 回收;连续 exec 用例取尽两条镜像事件,确认 `/usr/bin/true` 最后出现。
`EXEC` 行给出已安装镜像和 ELF 头部解析结果。测试程序在 exec 前刷新并驱逐标记所在页面,hook 中直接读取返回 `-EFAULT`(`-14`),而 task-work 回调成功返回 `454950524f424521`(`EIPROBE!` 的十六进制字节)。这验证了不可睡眠上下文无法读取冷页,而可睡眠上下文可以。
检查其他命令时可省略测试程序专用的探测偏移:
```bash
sudo ./exec_image_inspector --timeout-ms 3000 -- /bin/true
```
命令行格式如下:
```text
exec_image_inspector [--probe-offset BYTES] [--timeout-ms MS] [--verbose] -- COMMAND [ARG...]
```
`--timeout-ms` 接受 100 至 60000 毫秒,默认值为 5000 毫秒,超过时限后加载器会终止并回收命令。`--probe-offset` 比较指定位置的 8 字节直接读取与延迟读取,用于验证冷页场景。`--verbose` 输出 libbpf 诊断,便于排查 load 或 attach 问题。`--` 明确结束 inspector 选项解析,后面的参数全部属于被观察命令。
### 环境要求
| 要求 | 详情 |
|---|---|
| 内核版本 | Linux 6.19+(BPF task work 在 6.18 引入,file-backed dynptr 在 6.19 引入) |
| 内核配置 | `CONFIG_BPF=y`, `CONFIG_BPF_SYSCALL=y`, `CONFIG_BPF_JIT=y`, `CONFIG_BPF_LSM=y`, `CONFIG_SECURITY=y`, `CONFIG_DEBUG_INFO_BTF=y` |
| 活动 LSM | `/sys/kernel/security/lsm` 中包含 `bpf` |
| 架构 | 已在 x86_64 上测试 |
| 权限 | root |
## 适用范围
本工具观察自身创建的一个直接子进程,单个 map 槽位适合这种 CLI 场景。如果要扩展到并发服务,可以改为按 task 分配状态、设置准入上限并回收待处理回调。超时清理向该子进程发送 SIGKILL,调用方可以添加进程组管理和外部信号处理来覆盖更复杂的场景。
## 总结
本教程展示了如何使用 BPF task work 和 file-backed dynptr 检查 exec 后真正安装的可执行镜像。相比读取 `/proc/<pid>/exe` 或在 tracepoint 中获取路径,这种方案可以在可睡眠上下文中读取文件内容,即使目标字节位于冷页也能完成。
工具的核心思路是把识别目标进程的时刻(不可睡眠的 LSM hook)与读取文件内容的时刻(可睡眠的 task work 回调)分开。阻塞子进程握手消除了 attach 竞态,有界命令运行和最终 drain 确保所有事件都能到达用户态,显式资源释放保证清理完整。这些内核功能组合成了一个可复现的单命令工具,同时为并发场景留出扩展空间。
> 如果你想深入了解 eBPF,请查看我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或访问我们的网站 <https://eunomia.dev/tutorials/>。
## 参考资料
- [BPF task-work 基础实现](https://github.com/torvalds/linux/commit/5c8fd7e2b5b0a527cf88740da122166695382a78)
- [bpf_task_work_schedule_signal kfunc](https://github.com/torvalds/linux/commit/38aa7003e369802f81a078f6673d10d97013f04f)
- [File-backed dynptr 基础实现](https://github.com/torvalds/linux/commit/8d8771dc03e48300e80b43744dd3c320ccaf746a)
- [File dynptr kfunc 与 helper](https://github.com/torvalds/linux/commit/e3e36edb1b8f0e6975c68acd2e1202ec0397fd75)
- [可睡眠 file-dynptr dispatch](https://github.com/torvalds/linux/commit/2c52e8943a437af6093d8b0f0920f1764f0e5f64)
- [Kernel kfunc 文档](https://github.com/torvalds/linux/blob/v7.1/Documentation/bpf/kfuncs.rst)
- [File-reader BPF selftest](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/progs/file_reader.c)
- [BPF task-work selftest](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/progs/task_work.c)

View File

@@ -0,0 +1,29 @@
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __EXEC_IMAGE_INSPECTOR_BPF_EXPERIMENTAL_H
#define __EXEC_IMAGE_INSPECTOR_BPF_EXPERIMENTAL_H
/*
* These Linux 6.18/6.19 declarations are not present in the repository's
* older generated UAPI and vmlinux headers. Keep them local until those
* vendored headers are regenerated.
*/
struct bpf_task_work {
__u64 opaque;
} __attribute__((aligned(8)));
typedef int (*bpf_task_work_callback_t)(struct bpf_map *map, void *key,
void *value);
extern int bpf_task_work_schedule_signal(struct task_struct *task,
struct bpf_task_work *work,
void *map__map,
bpf_task_work_callback_t callback) __ksym;
extern struct file *bpf_get_task_exe_file(struct task_struct *task) __ksym;
extern void bpf_put_file(struct file *file) __ksym;
extern int bpf_path_d_path(const struct path *path, char *buf,
__u64 buf__sz) __ksym;
extern int bpf_dynptr_from_file(struct file *file, __u32 flags,
struct bpf_dynptr *ptr__uninit) __ksym;
extern int bpf_dynptr_file_discard(struct bpf_dynptr *dynptr) __ksym;
#endif /* __EXEC_IMAGE_INSPECTOR_BPF_EXPERIMENTAL_H */

View File

@@ -0,0 +1,121 @@
digraph exec_image_flow {
graph [
rankdir=TB,
bgcolor="white",
pad="0.25",
nodesep="0.34",
ranksep="0.46",
splines=spline,
size="10,7.5",
ratio=compress,
newrank=true,
fontname="DejaVu Sans"
];
node [
shape=box,
style="rounded,filled",
fillcolor="#F7FAFC",
color="#4A5568",
fontcolor="#1A202C",
fontname="DejaVu Sans",
fontsize=12,
margin="0.16,0.10"
];
edge [
color="#4A5568",
fontcolor="#2D3748",
fontname="DejaVu Sans",
fontsize=10,
arrowsize=0.75
];
fork [
label="User space\nfork child\nchild waits on pipe",
fillcolor="#EBF8FF",
color="#3182CE"
];
prepare [
label="target_tgid + probe_offset\nload LSM + ring buffer",
fillcolor="#EBF8FF",
color="#3182CE"
];
exec [
label="release pipe\nchild calls execvp",
fillcolor="#EBF8FF",
color="#3182CE"
];
hook [
label="bprm_committed_creds\nmatch target_tgid",
fillcolor="#FFFAF0",
color="#DD6B20"
];
probe [
label="probe_offset set?",
shape=diamond,
style="filled",
fillcolor="#FFFAF0",
color="#DD6B20",
margin="0.10,0.05"
];
direct [
label="non-sleepable direct probe\nstore result",
fillcolor="#FFFAF0",
color="#DD6B20"
];
pending [
label="pending[0]\nscheduled_ns + direct result",
shape=cylinder,
fillcolor="#FFF5F5",
color="#C05621"
];
schedule [
label="bpf_task_work_schedule_signal\ninspect_executable",
fillcolor="#FAF5FF",
color="#805AD5"
];
callback [
label="sleepable task-work callback\nrecord callback latency",
fillcolor="#FAF5FF",
color="#805AD5"
];
file [
label="bpf_get_task_exe_file\nresolve path",
fillcolor="#FAF5FF",
color="#805AD5"
];
read [
label="file-backed dynptr\nread ELF header + marker\ndiscard dynptr + put file",
fillcolor="#FAF5FF",
color="#805AD5"
];
event [
label="ring buffer event\npath + ELF + probe + latency",
fillcolor="#F0FFF4",
color="#38A169"
];
collect [
label="User space\npoll + waitpid\nfinal drain + stats",
fillcolor="#EBF8FF",
color="#3182CE"
];
{ rank=same; fork; prepare; exec; }
{ rank=same; hook; probe; direct; }
{ rank=same; pending; schedule; }
{ rank=same; callback; file; read; }
{ rank=same; event; collect; }
fork -> prepare [label="parent knows child TGID"];
prepare -> exec [label="attach completes"];
exec -> hook [label="each committed exec"];
hook -> probe;
probe -> direct [label="yes"];
probe -> pending [label="no"];
direct -> pending;
pending -> schedule;
schedule -> callback [label="task work runs"];
callback -> file;
file -> read;
read -> event;
event -> collect;
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 205 KiB

View File

@@ -0,0 +1,198 @@
// SPDX-License-Identifier: GPL-2.0
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include "bpf_experimental.h"
#include "exec_image_inspector.h"
char LICENSE[] SEC("license") = "GPL";
#define ENOENT 2
#define EI_CLASS 4
#define EI_DATA 5
#define ELFCLASS32 1
#define ELFCLASS64 2
#define ELFDATA2LSB 1
#define ELFDATA2MSB 2
const volatile __u32 target_tgid;
const volatile __u32 probe_offset;
struct inspector_stats stats;
struct {
__uint(type, BPF_MAP_TYPE_RINGBUF);
__uint(max_entries, 256 * 1024);
} events SEC(".maps");
struct exec_work {
__u64 scheduled_ns;
int direct_probe_error;
struct bpf_task_work work;
};
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 1);
__type(key, __u32);
__type(value, struct exec_work);
} pending SEC(".maps");
static __u16 read_elf_u16(const unsigned char *header, int offset, __u8 data)
{
if (data == ELFDATA2MSB)
return ((__u16)header[offset] << 8) | header[offset + 1];
return header[offset] | ((__u16)header[offset + 1] << 8);
}
static int probe_file_without_sleep(struct file *file)
{
unsigned char sample[EXEC_PROBE_LEN];
struct bpf_dynptr dynptr;
int err;
if (!probe_offset)
return 0;
__sync_fetch_and_add(&stats.direct_probes, 1);
if (!file) {
err = -ENOENT;
goto record;
}
err = bpf_dynptr_from_file(file, 0, &dynptr);
if (err) {
bpf_dynptr_file_discard(&dynptr);
goto record;
}
err = bpf_dynptr_read(sample, sizeof(sample), &dynptr, probe_offset, 0);
bpf_dynptr_file_discard(&dynptr);
record:
if (err)
__sync_fetch_and_add(&stats.direct_probe_errors, 1);
return err;
}
static int inspect_executable(struct bpf_map *map, void *key, void *value)
{
unsigned char header[64] = {};
struct exec_work *work = value;
struct exec_event event = {};
struct task_struct *task;
struct bpf_dynptr dynptr;
struct file *file;
__u64 pid_tgid;
int err;
(void)map;
(void)key;
__sync_fetch_and_add(&stats.callbacks, 1);
pid_tgid = bpf_get_current_pid_tgid();
event.pid = (__u32)pid_tgid;
event.tgid = pid_tgid >> 32;
event.latency_ns = bpf_ktime_get_ns() - work->scheduled_ns;
event.direct_probe_error = work->direct_probe_error;
event.probe_offset = probe_offset;
bpf_get_current_comm(event.comm, sizeof(event.comm));
task = bpf_get_current_task_btf();
file = bpf_get_task_exe_file(task);
if (!file) {
event.header_error = -ENOENT;
__sync_fetch_and_add(&stats.header_errors, 1);
if (probe_offset) {
event.deferred_probe_error = -ENOENT;
__sync_fetch_and_add(&stats.deferred_probes, 1);
__sync_fetch_and_add(&stats.deferred_probe_errors, 1);
}
goto emit;
}
err = bpf_path_d_path(&file->f_path, event.path, sizeof(event.path));
if (err < 0) {
event.path_error = err;
__sync_fetch_and_add(&stats.path_errors, 1);
}
err = bpf_dynptr_from_file(file, 0, &dynptr);
if (err) {
bpf_dynptr_file_discard(&dynptr);
event.header_error = err;
__sync_fetch_and_add(&stats.header_errors, 1);
if (probe_offset) {
event.deferred_probe_error = err;
__sync_fetch_and_add(&stats.deferred_probes, 1);
__sync_fetch_and_add(&stats.deferred_probe_errors, 1);
}
goto put_file;
}
err = bpf_dynptr_read(header, sizeof(header), &dynptr, 0, 0);
if (err) {
event.header_error = err;
__sync_fetch_and_add(&stats.header_errors, 1);
}
if (probe_offset) {
__sync_fetch_and_add(&stats.deferred_probes, 1);
err = bpf_dynptr_read(event.probe_bytes, sizeof(event.probe_bytes),
&dynptr, probe_offset, 0);
event.deferred_probe_error = err;
if (err)
__sync_fetch_and_add(&stats.deferred_probe_errors, 1);
}
bpf_dynptr_file_discard(&dynptr);
if (!event.header_error && header[0] == 0x7f && header[1] == 'E' &&
header[2] == 'L' && header[3] == 'F') {
event.is_elf = 1;
event.elf_class = header[EI_CLASS];
event.elf_data = header[EI_DATA];
event.elf_type = read_elf_u16(header, 16, event.elf_data);
event.elf_machine = read_elf_u16(header, 18, event.elf_data);
}
put_file:
bpf_put_file(file);
emit:
if (bpf_ringbuf_output(&events, &event, sizeof(event), 0))
__sync_fetch_and_add(&stats.dropped, 1);
return 0;
}
SEC("lsm/bprm_committed_creds")
void BPF_PROG(schedule_exec_inspection, struct linux_binprm *bprm)
{
struct task_struct *task;
struct exec_work *work;
__u64 pid_tgid;
__u32 key = 0, tgid;
int err;
pid_tgid = bpf_get_current_pid_tgid();
tgid = pid_tgid >> 32;
if (target_tgid && tgid != target_tgid)
return;
__sync_fetch_and_add(&stats.matched, 1);
work = bpf_map_lookup_elem(&pending, &key);
if (!work) {
__sync_fetch_and_add(&stats.schedule_errors, 1);
return;
}
work->scheduled_ns = bpf_ktime_get_ns();
work->direct_probe_error = probe_file_without_sleep(bprm->file);
task = bpf_get_current_task_btf();
err = bpf_task_work_schedule_signal(task, &work->work, &pending,
inspect_executable);
if (err) {
__sync_fetch_and_add(&stats.schedule_errors, 1);
return;
}
__sync_fetch_and_add(&stats.scheduled, 1);
}

View File

@@ -0,0 +1,536 @@
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <errno.h>
#include <getopt.h>
#include <limits.h>
#include <signal.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <time.h>
#include <unistd.h>
#include <bpf/libbpf.h>
#include "exec_image_inspector.h"
#include "exec_image_inspector.skel.h"
struct environment {
unsigned long long probe_offset;
unsigned int timeout_ms;
bool verbose;
char **command;
};
struct child_process {
pid_t pid;
int release_fd;
bool released;
bool reaped;
int status;
};
struct event_context {
unsigned int seen;
};
static struct environment env = {
.timeout_ms = 5000,
};
static int libbpf_print_fn(enum libbpf_print_level level, const char *format,
va_list args)
{
if (level == LIBBPF_DEBUG && !env.verbose)
return 0;
return vfprintf(stderr, format, args);
}
static void usage(const char *program)
{
fprintf(stderr,
"Usage: %s [--probe-offset BYTES] [--timeout-ms MS] [--verbose] "
"-- COMMAND [ARG...]\n\n"
"Inspect the executable image installed by one command.\n\n"
"Options:\n"
" -p, --probe-offset BYTES also compare direct/deferred file reads\n"
" -t, --timeout-ms MS bound the command, 100-60000 "
"(default: 5000)\n"
" -v, --verbose print libbpf diagnostics\n"
" -h, --help show this help\n",
program);
}
static int parse_u64(const char *value, unsigned long long maximum,
unsigned long long *result)
{
char *end = NULL;
unsigned long long parsed;
errno = 0;
parsed = strtoull(value, &end, 10);
if (errno || end == value || *end || parsed > maximum)
return -EINVAL;
*result = parsed;
return 0;
}
static int parse_probe_offset(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, UINT_MAX - EXEC_PROBE_LEN, &parsed)) {
fprintf(stderr, "invalid probe offset: %s\n", value);
return -EINVAL;
}
env.probe_offset = parsed;
return 0;
}
static int parse_timeout(const char *value)
{
unsigned long long parsed;
if (parse_u64(value, 60000, &parsed) || parsed < 100) {
fprintf(stderr, "invalid timeout in milliseconds: %s\n", value);
return -EINVAL;
}
env.timeout_ms = parsed;
return 0;
}
static int parse_option(int option, const char *program)
{
switch (option) {
case 'p':
return parse_probe_offset(optarg);
case 't':
return parse_timeout(optarg);
case 'v':
env.verbose = true;
return 0;
case 'h':
usage(program);
exit(0);
default:
return -EINVAL;
}
}
static int parse_args(int argc, char **argv)
{
static const struct option options[] = {
{ "probe-offset", required_argument, NULL, 'p' },
{ "timeout-ms", required_argument, NULL, 't' },
{ "verbose", no_argument, NULL, 'v' },
{ "help", no_argument, NULL, 'h' },
{},
};
int error, option;
while ((option = getopt_long(argc, argv, "+p:t:vh", options, NULL)) != -1) {
error = parse_option(option, argv[0]);
if (error)
return error;
}
if (optind == argc) {
fprintf(stderr, "COMMAND is required\n");
return -EINVAL;
}
env.command = &argv[optind];
return 0;
}
static long long monotonic_milliseconds(void)
{
struct timespec timestamp;
if (clock_gettime(CLOCK_MONOTONIC, &timestamp))
return -errno;
return timestamp.tv_sec * 1000LL + timestamp.tv_nsec / 1000000;
}
static int start_blocked_child(struct child_process *child)
{
int pipe_fds[2];
pid_t pid;
if (pipe(pipe_fds))
return -errno;
pid = fork();
if (pid < 0) {
int error = -errno;
close(pipe_fds[0]);
close(pipe_fds[1]);
return error;
}
if (pid == 0) {
char release;
ssize_t count;
close(pipe_fds[1]);
do {
count = read(pipe_fds[0], &release, sizeof(release));
} while (count < 0 && errno == EINTR);
close(pipe_fds[0]);
if (count != sizeof(release))
_exit(126);
/* Intentional argv execution; no shell parses the supplied arguments. */
execvp(env.command[0], env.command); /* Flawfinder: ignore */
fprintf(stderr, "failed to execute %s: %s\n", env.command[0],
strerror(errno));
_exit(127);
}
close(pipe_fds[0]);
child->pid = pid;
child->release_fd = pipe_fds[1];
return 0;
}
static int release_child(struct child_process *child)
{
char release = 1;
ssize_t count;
do {
count = write(child->release_fd, &release, sizeof(release));
} while (count < 0 && errno == EINTR);
close(child->release_fd);
child->release_fd = -1;
if (count != sizeof(release))
return count < 0 ? -errno : -EIO;
child->released = true;
return 0;
}
static int child_exit_code(int status)
{
if (WIFEXITED(status))
return WEXITSTATUS(status);
if (WIFSIGNALED(status))
return 128 + WTERMSIG(status);
return 125;
}
static int reap_child(struct child_process *child, int options)
{
pid_t result;
if (child->reaped)
return 1;
do {
result = waitpid(child->pid, &child->status, options);
} while (result < 0 && errno == EINTR);
if (result < 0)
return -errno;
if (result == 0)
return 0;
child->reaped = true;
return 1;
}
static int drain_events(struct ring_buffer *ring_buffer)
{
int error;
for (;;) {
error = ring_buffer__poll(ring_buffer, 0);
if (error == -EINTR)
continue;
if (error < 0) {
fprintf(stderr, "ring-buffer drain failed: %s\n",
strerror(-error));
return error;
}
if (!error)
return 0;
}
}
static const char *elf_class_name(unsigned char value)
{
switch (value) {
case 1:
return "ELF32";
case 2:
return "ELF64";
default:
return "UNKNOWN";
}
}
static const char *elf_data_name(unsigned char value)
{
switch (value) {
case 1:
return "LSB";
case 2:
return "MSB";
default:
return "UNKNOWN";
}
}
static const char *elf_type_name(unsigned short value)
{
switch (value) {
case 2:
return "ET_EXEC";
case 3:
return "ET_DYN";
default:
return "OTHER";
}
}
static const char *elf_machine_name(unsigned short value)
{
switch (value) {
case 3:
return "EM_386";
case 62:
return "EM_X86_64";
case 183:
return "EM_AARCH64";
default:
return "OTHER";
}
}
static int handle_event(void *context, void *data, size_t size)
{
const struct exec_event *event = data;
struct event_context *events = context;
unsigned int index;
if (size < sizeof(*event)) {
fprintf(stderr, "short ring-buffer event: %zu bytes\n", size);
return 0;
}
events->seen++;
printf("EXEC pid=%u tgid=%u comm=%.*s path=%.*s is_elf=%u "
"class=%s endian=%s type=%s(%u) machine=%s(%u) "
"header_error=%d path_error=%d latency_us=%llu\n",
event->pid, event->tgid, EXEC_COMM_LEN, event->comm,
EXEC_PATH_LEN, event->path, event->is_elf,
elf_class_name(event->elf_class), elf_data_name(event->elf_data),
elf_type_name(event->elf_type), event->elf_type,
elf_machine_name(event->elf_machine), event->elf_machine,
event->header_error, event->path_error,
event->latency_ns / 1000);
if (event->probe_offset) {
printf("PROBE offset=%llu direct_error=%d deferred_error=%d bytes=",
event->probe_offset, event->direct_probe_error,
event->deferred_probe_error);
for (index = 0; index < EXEC_PROBE_LEN; index++)
printf("%02x", event->probe_bytes[index]);
putchar('\n');
}
fflush(stdout);
return 0;
}
static void stop_child(struct child_process *child)
{
if (child->reaped || child->pid <= 0)
return;
if (!child->released && child->release_fd >= 0) {
close(child->release_fd);
child->release_fd = -1;
} else {
kill(child->pid, SIGKILL);
}
(void)reap_child(child, 0);
}
static int setup_inspector(const struct child_process *child,
struct event_context *events,
struct exec_image_inspector_bpf **skeleton,
struct ring_buffer **ring_buffer)
{
struct exec_image_inspector_bpf *skel;
struct ring_buffer *ring;
int error;
skel = exec_image_inspector_bpf__open();
if (!skel) {
fprintf(stderr, "failed to open BPF skeleton\n");
return -ENOMEM;
}
*skeleton = skel;
skel->rodata->target_tgid = child->pid;
skel->rodata->probe_offset = env.probe_offset;
error = exec_image_inspector_bpf__load(skel);
if (error) {
fprintf(stderr, "failed to load BPF object: %s\n", strerror(-error));
return error;
}
error = exec_image_inspector_bpf__attach(skel);
if (error) {
fprintf(stderr, "failed to attach bprm_committed_creds LSM hook: %s\n",
strerror(-error));
return error;
}
ring = ring_buffer__new(bpf_map__fd(skel->maps.events), handle_event,
events, NULL);
if (!ring) {
fprintf(stderr, "failed to create ring buffer: %s\n", strerror(errno));
return errno ? -errno : -ENOMEM;
}
*ring_buffer = ring;
return 0;
}
static int reap_timed_out_child(struct child_process *child)
{
int error;
if (child->reaped)
return 0;
fprintf(stderr, "command exceeded timeout; sending SIGKILL\n");
kill(child->pid, SIGKILL);
error = reap_child(child, 0);
if (error < 0) {
fprintf(stderr, "waitpid after timeout failed: %s\n",
strerror(-error));
return error;
}
return 0;
}
static int wait_for_command(struct ring_buffer *ring_buffer,
struct child_process *child,
const struct event_context *events)
{
long long deadline, now;
int error;
printf("READY target_tgid=%d probe_offset=%llu timeout_ms=%u command=%s\n",
child->pid, env.probe_offset, env.timeout_ms, env.command[0]);
fflush(stdout);
error = release_child(child);
if (error) {
fprintf(stderr, "failed to release command process: %s\n",
strerror(-error));
return error;
}
now = monotonic_milliseconds();
if (now < 0) {
fprintf(stderr, "failed to read monotonic clock: %s\n",
strerror((int)-now));
return (int)now;
}
deadline = now + env.timeout_ms;
for (;;) {
error = ring_buffer__poll(ring_buffer, 50);
if (error == -EINTR)
continue;
if (error < 0) {
fprintf(stderr, "ring-buffer poll failed: %s\n", strerror(-error));
return error;
}
error = reap_child(child, WNOHANG);
if (error < 0) {
fprintf(stderr, "waitpid failed: %s\n", strerror(-error));
return error;
}
if (child->reaped && events->seen)
break;
now = monotonic_milliseconds();
if (now < 0)
return (int)now;
if (now >= deadline)
break;
if (child->reaped && !events->seen)
continue;
}
error = reap_timed_out_child(child);
if (error)
return error;
error = drain_events(ring_buffer);
if (error)
return error;
return child_exit_code(child->status);
}
static int report_result(const struct exec_image_inspector_bpf *skel,
const struct event_context *events, int command_exit)
{
struct inspector_stats final_stats = skel->bss->stats;
printf("SUMMARY matched=%llu scheduled=%llu schedule_errors=%llu "
"callbacks=%llu header_errors=%llu path_errors=%llu "
"direct_probes=%llu direct_probe_errors=%llu "
"deferred_probes=%llu deferred_probe_errors=%llu dropped=%llu "
"events=%u command_exit=%d\n",
final_stats.matched, final_stats.scheduled,
final_stats.schedule_errors, final_stats.callbacks,
final_stats.header_errors, final_stats.path_errors,
final_stats.direct_probes, final_stats.direct_probe_errors,
final_stats.deferred_probes, final_stats.deferred_probe_errors,
final_stats.dropped, events->seen, command_exit);
if (!events->seen) {
fprintf(stderr, "no executable image event was observed\n");
return 1;
}
if (command_exit) {
fprintf(stderr, "command exited with status %d\n", command_exit);
return command_exit;
}
return 0;
}
int main(int argc, char **argv)
{
struct exec_image_inspector_bpf *skel = NULL;
struct child_process child = { .release_fd = -1 };
struct event_context events = {};
struct ring_buffer *ring_buffer = NULL;
int command_exit, error, result = 1;
error = parse_args(argc, argv);
if (error) {
usage(argv[0]);
return 2;
}
libbpf_set_print(libbpf_print_fn);
error = start_blocked_child(&child);
if (error) {
fprintf(stderr, "failed to create command process: %s\n",
strerror(-error));
return 1;
}
error = setup_inspector(&child, &events, &skel, &ring_buffer);
if (error)
goto cleanup;
command_exit = wait_for_command(ring_buffer, &child, &events);
if (command_exit < 0)
goto cleanup;
result = report_result(skel, &events, command_exit);
cleanup:
stop_child(&child);
ring_buffer__free(ring_buffer);
exec_image_inspector_bpf__destroy(skel);
return result;
}

View File

@@ -0,0 +1,43 @@
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
#ifndef __EXEC_IMAGE_INSPECTOR_H
#define __EXEC_IMAGE_INSPECTOR_H
#define EXEC_COMM_LEN 16
#define EXEC_PATH_LEN 256
#define EXEC_PROBE_LEN 8
struct exec_event {
unsigned int pid;
unsigned int tgid;
unsigned char is_elf;
unsigned char elf_class;
unsigned char elf_data;
unsigned char reserved;
unsigned short elf_type;
unsigned short elf_machine;
int header_error;
int path_error;
int direct_probe_error;
int deferred_probe_error;
unsigned long long latency_ns;
unsigned long long probe_offset;
char comm[EXEC_COMM_LEN];
char path[EXEC_PATH_LEN];
unsigned char probe_bytes[EXEC_PROBE_LEN];
};
struct inspector_stats {
unsigned long long matched;
unsigned long long scheduled;
unsigned long long schedule_errors;
unsigned long long callbacks;
unsigned long long header_errors;
unsigned long long path_errors;
unsigned long long direct_probes;
unsigned long long direct_probe_errors;
unsigned long long deferred_probes;
unsigned long long deferred_probe_errors;
unsigned long long dropped;
};
#endif /* __EXEC_IMAGE_INSPECTOR_H */

View File

@@ -0,0 +1,8 @@
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
#include <stdio.h>
int main(void)
{
puts("exec fixture completed");
return 0;
}

View File

@@ -0,0 +1,236 @@
#!/usr/bin/env python3
"""Deterministic KVM checks for the deferred executable image inspector."""
from __future__ import annotations
import os
import re
import shutil
import stat
import subprocess
import sys
import tempfile
MARKER = b"EIPROBE!"
PROBE_DISTANCE = 4 * 1024 * 1024
SUMMARY = re.compile(
r"SUMMARY matched=(\d+) scheduled=(\d+) schedule_errors=(\d+) "
r"callbacks=(\d+) header_errors=(\d+) path_errors=(\d+) "
r"direct_probes=(\d+) direct_probe_errors=(\d+) "
r"deferred_probes=(\d+) deferred_probe_errors=(\d+) dropped=(\d+) "
r"events=(\d+) command_exit=(\d+)"
)
def run(*command: str, check: bool = True) -> subprocess.CompletedProcess[str]:
return subprocess.run(
command,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
check=check,
timeout=15,
)
def parse_summary(output: str) -> dict[str, int]:
match = SUMMARY.search(output)
if not match:
raise AssertionError("missing summary:\n" + output)
names = (
"matched",
"scheduled",
"schedule_errors",
"callbacks",
"header_errors",
"path_errors",
"direct_probes",
"direct_probe_errors",
"deferred_probes",
"deferred_probe_errors",
"dropped",
"events",
"command_exit",
)
return dict(zip(names, map(int, match.groups())))
def create_probe_image(fixture: str, directory: str) -> tuple[str, int]:
image = os.path.join(directory, "exec_fixture_image")
shutil.copyfile(fixture, image)
os.chmod(image, stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR)
original_size = os.path.getsize(image)
probe_offset = ((original_size + 4095) // 4096) * 4096 + PROBE_DISTANCE
with open(image, "r+b", buffering=0) as executable:
executable.seek(probe_offset)
executable.write(MARKER)
os.fsync(executable.fileno())
if hasattr(os, "posix_fadvise"):
os.posix_fadvise(executable.fileno(), 0, 0, os.POSIX_FADV_DONTNEED)
return image, probe_offset
def test_missing_command(inspector: str) -> dict[str, int]:
missing = "/definitely/missing/exec-image-inspector-fixture"
result = run(
inspector,
"--timeout-ms", "500",
"--",
missing,
check=False,
)
assert result.returncode != 0, result.stdout
assert f"failed to execute {missing}" in result.stdout, result.stdout
assert "no executable image event was observed" in result.stdout, result.stdout
stats = parse_summary(result.stdout)
assert stats["matched"] == 0, stats
assert stats["scheduled"] == 0, stats
assert stats["callbacks"] == 0, stats
assert stats["events"] == 0, stats
assert stats["command_exit"] == 127, stats
return stats
def test_timeout_cleanup(inspector: str) -> dict[str, int]:
result = run(
inspector,
"--timeout-ms", "200",
"--",
"/bin/sleep", "2",
check=False,
)
assert result.returncode == 128 + 9, result.stdout
assert "command exceeded timeout; sending SIGKILL" in result.stdout, result.stdout
stats = parse_summary(result.stdout)
assert stats["matched"] == 1, stats
assert stats["scheduled"] == 1, stats
assert stats["callbacks"] == 1, stats
assert stats["events"] == 1, stats
assert stats["command_exit"] == 137, stats
return stats
def test_reexec_chain(inspector: str) -> tuple[dict[str, int], str]:
final_path = os.path.realpath("/bin/true")
result = run(
inspector,
"--timeout-ms", "1000",
"--",
"/bin/sh", "-c", "exec /bin/true",
check=False,
)
assert result.returncode == 0, result.stdout
exec_lines = [
line for line in result.stdout.splitlines() if line.startswith("EXEC ")
]
assert len(exec_lines) == 2, result.stdout
assert f"path={final_path}" in exec_lines[-1], exec_lines[-1]
stats = parse_summary(result.stdout)
assert stats["matched"] == 2, stats
assert stats["scheduled"] == 2, stats
assert stats["callbacks"] == 2, stats
assert stats["events"] == 2, stats
assert stats["command_exit"] == 0, stats
return stats, final_path
def test_deferred_probe(inspector: str, fixture: str) -> str:
with tempfile.TemporaryDirectory(prefix="exec-image-inspector-") as directory:
image, probe_offset = create_probe_image(fixture, directory)
result = run(
inspector,
"--probe-offset", str(probe_offset),
"--timeout-ms", "3000",
"--",
image,
check=False,
)
assert result.returncode == 0, result.stdout
exec_line = next(
(line for line in result.stdout.splitlines() if line.startswith("EXEC ")),
"",
)
assert exec_line, result.stdout
assert f"path={image}" in exec_line, exec_line
assert "is_elf=1" in exec_line, exec_line
assert "class=ELF64" in exec_line, exec_line
assert "endian=LSB" in exec_line, exec_line
assert "machine=EM_X86_64(62)" in exec_line, exec_line
assert "header_error=0 path_error=0" in exec_line, exec_line
probe_line = next(
(line for line in result.stdout.splitlines() if line.startswith("PROBE ")),
"",
)
assert probe_line, result.stdout
assert f"offset={probe_offset}" in probe_line, probe_line
assert "direct_error=-14" in probe_line, probe_line
assert "deferred_error=0" in probe_line, probe_line
assert f"bytes={MARKER.hex()}" in probe_line, probe_line
stats = parse_summary(result.stdout)
expected = {
"matched": 1,
"scheduled": 1,
"schedule_errors": 0,
"callbacks": 1,
"header_errors": 0,
"path_errors": 0,
"direct_probes": 1,
"direct_probe_errors": 1,
"deferred_probes": 1,
"deferred_probe_errors": 0,
"dropped": 0,
"events": 1,
"command_exit": 0,
}
assert stats == expected, stats
return result.stdout
def main() -> int:
if os.geteuid() != 0:
raise SystemExit("this integration test must run as root inside KVM")
inspector = os.path.abspath(
sys.argv[1] if len(sys.argv) > 1 else "./exec_image_inspector"
)
fixture = os.path.abspath(
sys.argv[2] if len(sys.argv) > 2 else "./tests/exec_fixture"
)
missing_stats = test_missing_command(inspector)
timeout_stats = test_timeout_cleanup(inspector)
reexec_stats, final_path = test_reexec_chain(inspector)
output = test_deferred_probe(inspector, fixture)
print(
"TEST-MISSING "
f"matched={missing_stats['matched']} events={missing_stats['events']} "
f"command_exit={missing_stats['command_exit']}"
)
print(
"TEST-TIMEOUT "
f"matched={timeout_stats['matched']} callbacks={timeout_stats['callbacks']} "
f"events={timeout_stats['events']} command_exit={timeout_stats['command_exit']}"
)
print(
"TEST-REEXEC "
f"matched={reexec_stats['matched']} callbacks={reexec_stats['callbacks']} "
f"events={reexec_stats['events']} command_exit={reexec_stats['command_exit']} "
f"final_path={final_path}"
)
print(output, end="")
print(
"PASS: missing-command, timeout cleanup, re-exec drain, ELF decode, "
"and deferred file read succeeded"
)
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -68,6 +68,7 @@ Networking:
- [lesson 42-xdp-loadbalancer](42-xdp-loadbalancer/README.md) XDP Load Balancer
- [lesson 46-xdp-test](46-xdp-test/README.md) Building a High-Performance XDP Packet Generator
- [lesson 50-tcx](50-tcx/README.md) Composable Traffic Control with TCX Links
- [lesson 53-egress-pacer](53-egress-pacer/README.md) Building an Egress Pacer with BPF Qdisc
Tracing:
@@ -79,6 +80,7 @@ Tracing:
- [lesson 39-nginx](39-nginx/README.md) Using eBPF to Trace Nginx Requests
- [lesson 40-mysql](40-mysql/README.md) Using eBPF to Trace MySQL Queries
- [lesson 48-energy](48-energy/README.md) Energy Monitoring for Process-Level Power Analysis
- [lesson 52-fsession-latency](52-fsession-latency/README.md) Tracing Slow vfs_read Calls with fsession
Security:
@@ -89,6 +91,8 @@ Security:
- [lesson 27-replace](27-replace/README.md) Transparent Text Replacement in File Reads
- [lesson 28-detach](28-detach/README.md) Running eBPF After Application Exits: The Lifecycle of eBPF Programs
- [lesson 34-syscall](34-syscall/README.md) Modifying System Call Arguments with eBPF
- [lesson 51-tcp-quarantine](51-tcp-quarantine/README.md) Precisely Isolating Established TCP Connections
- [lesson 54-exec-image-inspector](54-exec-image-inspector/README.md) Inspecting the Executable Image After exec
Features:

View File

@@ -68,6 +68,7 @@ GPU:
- [lesson 42-xdp-loadbalancer](42-xdp-loadbalancer/README.zh.md) eBPF 开发者教程: 简单的 XDP 负载均衡器
- [lesson 46-xdp-test](46-xdp-test/README.zh.md) eBPF 实例教程:构建高性能 XDP 数据包生成器
- [lesson 50-tcx](50-tcx/README.zh.md) eBPF 入门实践教程第五十篇:使用 TCX Link 实现可组合的流量控制
- [lesson 53-egress-pacer](53-egress-pacer/README.zh.md) eBPF 教程:用 BPF Qdisc 实现出口限速
追踪:
@@ -79,6 +80,7 @@ GPU:
- [lesson 39-nginx](39-nginx/README.zh.md) 使用 eBPF 跟踪 Nginx 请求
- [lesson 40-mysql](40-mysql/README.zh.md) 使用 eBPF 跟踪 MySQL 查询
- [lesson 48-energy](48-energy/README.zh.md) eBPF 教程:进程级能源监控与功耗分析
- [lesson 52-fsession-latency](52-fsession-latency/README.zh.md) eBPF 教程:使用 fsession 追踪慢速 vfs_read 调用
安全:
@@ -89,6 +91,8 @@ GPU:
- [lesson 27-replace](27-replace/README.zh.md) 替换任意程序读取或者写入的文本
- [lesson 28-detach](28-detach/README.zh.md) 在应用程序退出后运行 eBPF 程序:eBPF 程序的生命周期
- [lesson 34-syscall](34-syscall/README.zh.md) eBPF 开发实践:使用 eBPF 修改系统调用参数
- [lesson 51-tcp-quarantine](51-tcp-quarantine/README.zh.md) eBPF 教程:精准隔离已建立的 TCP 连接
- [lesson 54-exec-image-inspector](54-exec-image-inspector/README.zh.md) eBPF 教程:检查 exec 后真正安装的可执行镜像
特性:

View File

@@ -57,6 +57,10 @@ This table is generated from each tutorial's `.config` metadata. `Minimum kernel
| [eBPF Tutorial: Energy Monitoring for Process-Level Power Analysis](48-energy/README.md) | 5.8 | Required feature | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_TRACEPOINTS=y | None | Required | CI build |
| [eBPF Tutorial: Fixing Broken HID Devices Without Kernel Patches](49-hid/README.md) | 6.3 | Tutorial docs | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_HID_BPF=y, CONFIG_UHID=y\|m | None | Required | CI build |
| [eBPF Tutorial by Example 50: Composable Traffic Control with TCX Links](50-tcx/README.md) | 6.6 | Tutorial docs | x86_64, arm64 | Optional | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_NET_XGRESS=y | Network interface | Required | CI build |
| [eBPF Tutorial: Precisely Isolating Established TCP Connections](51-tcp-quarantine/README.md) | 6.5 | Tutorial docs | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_INET=y, CONFIG_PROC_FS=y | None | Required | CI runtime |
| [eBPF Tutorial: Tracing Slow vfs_read Calls with fsession](52-fsession-latency/README.md) | 7.0 | Tutorial docs | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y | None | Required | CI build |
| [eBPF Tutorial: Building an Egress Pacer with BPF Qdisc](53-egress-pacer/README.md) | 6.16 | Tutorial docs | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_NET_SCHED=y, CONFIG_NET_SCH_BPF=y | Network interface | Required | CI build |
| [eBPF Tutorial: Inspecting the Executable Image After exec](54-exec-image-inspector/README.md) | 6.19 | Required feature | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_LSM=y, CONFIG_SECURITY=y, CONFIG_DEBUG_INFO_BTF=y | None | Required | CI build |
| [eBPF Tutorial: cgroup-based Policy Control](cgroup/README.md) | 5.8 | Required feature | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_CGROUPS=y, CONFIG_CGROUP_BPF=y, CONFIG_CGROUP_DEVICE=y, CONFIG_DEBUG_INFO_BTF=y | None | Required | CI build |
| [eBPF Tutorial by Example: BPF Arena for Zero-Copy Shared Memory](features/bpf_arena/README.md) | 6.9 | Tutorial docs | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y | None | Required | CI build |
| [eBPF Tutorial: BPF Iterators for Kernel Data Export](features/bpf_iters/README.md) | 5.8 | Tutorial docs | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_DEBUG_INFO_BTF=y | None | Required | CI build |

View File

@@ -57,6 +57,10 @@
| [eBPF 教程:进程级能源监控与功耗分析](48-energy/README.zh.md) | 5.8 | 必需特性 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_TRACEPOINTS=y | 无 | 需要 | CI 构建测试 |
| [eBPF 教程:无需内核补丁修复故障的 HID 设备](49-hid/README.zh.md) | 6.3 | 教程文档 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_HID_BPF=y, CONFIG_UHID=y\|m | 无 | 需要 | CI 构建测试 |
| [eBPF 入门实践教程第五十篇:使用 TCX Link 实现可组合的流量控制](50-tcx/README.zh.md) | 6.6 | 教程文档 | x86_64, arm64 | 可选 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_NET_XGRESS=y | 网络接口 | 需要 | CI 构建测试 |
| [eBPF 教程:精准隔离已建立的 TCP 连接](51-tcp-quarantine/README.zh.md) | 6.5 | 教程文档 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_INET=y, CONFIG_PROC_FS=y | 无 | 需要 | CI 运行测试 |
| [eBPF 教程:使用 fsession 追踪慢速 vfs_read 调用](52-fsession-latency/README.zh.md) | 7.0 | 教程文档 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y | 无 | 需要 | CI 构建测试 |
| [eBPF 教程:用 BPF Qdisc 实现出口限速](53-egress-pacer/README.zh.md) | 6.16 | 教程文档 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_NET_SCHED=y, CONFIG_NET_SCH_BPF=y | 网络接口 | 需要 | CI 构建测试 |
| [eBPF 教程:检查 exec 后真正安装的可执行镜像](54-exec-image-inspector/README.zh.md) | 6.19 | 必需特性 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_LSM=y, CONFIG_SECURITY=y, CONFIG_DEBUG_INFO_BTF=y | 无 | 需要 | CI 构建测试 |
| [eBPF 实例教程:基于 cgroup 的策略控制](cgroup/README.zh.md) | 5.8 | 必需特性 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_CGROUPS=y, CONFIG_CGROUP_BPF=y, CONFIG_CGROUP_DEVICE=y, CONFIG_DEBUG_INFO_BTF=y | 无 | 需要 | CI 构建测试 |
| [eBPF 实例教程:BPF Arena 零拷贝共享内存](features/bpf_arena/README.zh.md) | 6.9 | 教程文档 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y | 无 | 需要 | CI 构建测试 |
| [eBPF 教程:BPF 迭代器用于内核数据导出](features/bpf_iters/README.zh.md) | 5.8 | 教程文档 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_DEBUG_INFO_BTF=y | 无 | 需要 | CI 构建测试 |