mirror of
https://github.com/eunomia-bpf/bpf-developer-tutorial.git
synced 2026-10-30 20:25:50 +08:00
docs: improve four advanced tutorial narratives
This commit is contained in:
LinuxDev9002
@@ -1,24 +1,36 @@
|
||||
# eBPF Tutorial by Example: Build a DNS-Derived IP Allowlist with cgroup BPF
|
||||
# eBPF Tutorial: Building a DNS-Derived IP Allowlist with cgroup BPF
|
||||
|
||||
Suppose a service may open HTTPS connections to `api.example.com`, while direct connections to every other address should be rejected. A static IP allowlist is a poor fit because DNS answers change and expire. A connect hook sees an IP address rather than the name that produced it. The missing piece is a short-lived link between the application's DNS exchange and its later `connect()` call.
|
||||
Ever tried to restrict a service so it can only connect to `api.example.com`? You might reach for an IP allowlist, but DNS answers change constantly and TTLs expire. By the time your application calls `connect()`, the kernel only sees an IP address—it has no idea which domain produced it. What's missing is a short-lived link between the DNS lookup and the subsequent connection.
|
||||
|
||||
This tutorial builds that link as a minimal working policy tool. It watches one domain through one configured DNS resolver, learns a direct IPv4 A record only from a matching query and response, and allows connections to one TCP port until the DNS TTL expires.
|
||||
This tutorial builds that link. We'll watch DNS traffic for a specific domain, learn IP addresses only from valid query-response pairs, and allow connections until the TTL expires. It's a minimal but complete policy tool that shows how cgroup BPF can coordinate across multiple kernel hooks.
|
||||
|
||||
> Complete source code: <https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/55-dns-egress>
|
||||
|
||||
## Where eBPF Enforces the Decision
|
||||
## Understanding the Problem: DNS and Connect Are Disconnected
|
||||
|
||||
eBPF runs verifier-checked programs at Linux kernel hooks and can retain state in maps or send selected events to user space. cgroup BPF makes those hooks follow a workload: packet programs can inspect traffic entering and leaving a cgroup, while a socket-address program can accept or reject an IPv4 connection before the kernel sends it.
|
||||
When your application resolves `api.example.com`, the DNS response contains an IP address and a TTL. But this information lives only in userspace—the resolver library caches it, your application calls `connect()` with the IP, and the kernel has no idea where that IP came from. This creates a fundamental security gap.
|
||||
|
||||
This example attaches three programs to the same cgroup. `cgroup_skb/egress` recognizes the configured DNS question and records a pending query. `cgroup_skb/ingress` accepts an address into the allowlist only after the response matches that query. `cgroup/connect4` makes the final decision for the protected TCP port. The ring buffer reports learned, allowed, denied, and expired decisions without participating in enforcement.
|
||||
Consider a container that should only talk to your backend API. Traditional firewalls can block destination IPs, but they can't enforce "only connect to IPs that came from resolving api.example.com within the last 60 seconds." The kernel-level policy and the application-level DNS resolution operate in complete isolation.
|
||||
|
||||
Follow one successful exchange. The application sends an A query for `lab.test`. The egress hook saves the resolver address, client address, client UDP port, and DNS transaction ID for five seconds. When the reply returns, the ingress hook reconstructs the same key, checks the response flags and question, and reads the first direct A answer. It stores the returned address with an expiration time derived from the TTL. A later `connect()` to that address and the protected port succeeds while the entry is live.
|
||||
cgroup BPF bridges this gap. By attaching programs to packet hooks and connect hooks on the same cgroup, we can observe DNS traffic and later enforce connection policy based on what we learned. The key insight is that cgroup BPF programs share state through maps, creating a trust chain from DNS query through response to eventual connection.
|
||||
|
||||
The query record is the trust boundary. An unsolicited response has no pending key, and a response with a different transaction ID searches for a different key. Neither can populate the allowlist. Expiration is checked again at connect time, so an address naturally stops working even if its LRU entry is still present.
|
||||
## How the Pieces Fit Together
|
||||
|
||||
## Shared Protocol and Event Types
|
||||
eBPF lets us run verified programs at multiple kernel hooks and share state between them through maps. cgroup BPF makes these hooks follow a workload: packet programs can inspect traffic entering and leaving a cgroup, while socket-address programs can accept or reject connections before they're established.
|
||||
|
||||
The shared header contains the small DNS layouts parsed by the BPF programs and the fixed event format consumed in user space.
|
||||
Our tool attaches three programs to the same cgroup:
|
||||
|
||||
1. **`cgroup_skb/egress`** watches outgoing DNS queries and records which ones we're expecting answers for
|
||||
2. **`cgroup_skb/ingress`** validates incoming DNS responses and learns IP addresses only from matching replies
|
||||
3. **`cgroup/connect4`** makes the final decision—allowing connections only to IPs learned from valid DNS responses, and only while their TTL is still valid
|
||||
|
||||
Let's walk through a successful exchange. Your application sends an A query for `lab.test`. The egress hook saves a correlation key containing the resolver address, client address, client UDP port, and DNS transaction ID. This record lives for five seconds. When the reply arrives, the ingress hook reconstructs the same key, verifies the response, and extracts the IP address along with its TTL. A later `connect()` to that address succeeds—but only while the TTL is valid.
|
||||
|
||||
The query record is the trust boundary here. An unsolicited response has no pending key to match. A response with the wrong transaction ID looks for a different key and finds nothing. Neither can pollute the allowlist. And even if an entry lingers in the LRU map, the connect hook re-checks expiration time, so addresses naturally stop working when their DNS time runs out.
|
||||
|
||||
## The Data Structures
|
||||
|
||||
Before diving into code, let's understand the data structures that make this work. The shared header defines DNS protocol layouts and the events we report to userspace:
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -69,11 +81,13 @@ struct dns_egress_event {
|
||||
#endif /* __DNS_EGRESS_H */
|
||||
```
|
||||
|
||||
The protocol structs are packed because they describe bytes on the wire. The event carries both the learned TTL and the absolute kernel expiration timestamp. The command prints the TTL, while the BPF side uses the monotonic timestamp for its decision.
|
||||
The protocol structs use the `packed` attribute because they describe bytes on the wire—no padding allowed. The `dns_header` maps directly to the 12-byte DNS header that starts every query and response. The `dns_question` struct follows the question name (which uses length-prefixed labels). The `dns_a_answer` expects the common compressed format where the name pointer is `0xc00c`, pointing back to the question section.
|
||||
|
||||
## The Three BPF Hooks
|
||||
Each event we send to userspace carries both the DNS TTL and the absolute expiration timestamp. User space prints the human-readable TTL, while the BPF side uses the monotonic timestamp for its decisions. This separation keeps the kernel logic clean—no time format conversions in BPF code.
|
||||
|
||||
Here is the complete kernel-side program.
|
||||
## The BPF Programs: Complete Implementation
|
||||
|
||||
Here's the complete kernel-side implementation. It's longer than some of our examples, but each piece has a clear job. Let's look at it in full, then walk through the key sections:
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: GPL-2.0
|
||||
@@ -367,17 +381,43 @@ int enforce_dns_policy(struct bpf_sock_addr *ctx)
|
||||
}
|
||||
```
|
||||
|
||||
`pending_queries` and `allowed_ips` are LRU hash maps with fixed capacities. The first map holds correlation state for at most five seconds. The second holds admitted IPv4 addresses for their DNS lifetime. LRU eviction keeps memory bounded; it also means this example favors recently active queries and addresses when either map reaches 1024 entries.
|
||||
### Understanding the Map Design
|
||||
|
||||
The egress path first validates IPv4, UDP, fragmentation state, resolver address, and resolver port. It then checks a standard one-question A query for the exact encoded name. Only after those checks does `record_dns_query()` insert the four-field correlation key. Packet observation itself returns `1`, so DNS traffic continues normally.
|
||||
The program uses three BPF maps, each serving a specific purpose in the trust chain. The `pending_queries` map is an LRU hash that holds correlation state for DNS queries. The key combines four fields—server IP, client IP, client port, and transaction ID—that uniquely identify a query/response pair. The value stores only the expiration timestamp because that's all we need to validate timing. LRU eviction ensures memory stays bounded even under DNS flood conditions.
|
||||
|
||||
On ingress, parsing starts with the reverse transport tuple. `parse_response_question()` adds the transaction ID, requires a live pending entry, and verifies a successful response containing the configured question. `parse_direct_a_answer()` accepts the compact `0xc00c` name pointer, class IN, type A, four-byte address, and a TTL from 1 to 86400 seconds. Once the answer passes, the pending query is consumed and the IP becomes eligible for connections.
|
||||
The `allowed_ips` map is also an LRU hash, but keyed simply by IPv4 address. The value contains the expiration timestamp, the original TTL in seconds (for logging), padding for alignment, and a flag to track whether we've already reported expiration. This flag prevents duplicate "expired" events when multiple threads race on a stale entry.
|
||||
|
||||
`enforce_dns_policy()` stays narrow. TCP connects to other destination ports pass immediately. A connect to the protected port looks up `ctx->user_ip4`; a live entry returns `1`, while a missing or expired entry returns `0`, which surfaces to the application as `EPERM`. A 64-bit compare-and-swap on `expired_reported` keeps the expiry notification to one event even when several threads race on the stale address. BPF atomic compare-and-exchange arrived in Linux 5.12, which sets this tool's minimum kernel version.
|
||||
The `events` ring buffer sends notifications to userspace. At 256KB, it can hold thousands of events without blocking the kernel path. Ring buffers are the modern replacement for perf buffers—they're more efficient and provide better ordering guarantees.
|
||||
|
||||
## Loading the Policy and Exercising Its Trust Chain
|
||||
### The Egress Path: Recording Pending Queries
|
||||
|
||||
The complete user-space program configures the read-only BPF data, attaches all three programs to one cgroup, and consumes ring-buffer events.
|
||||
When a packet leaves the cgroup, `record_dns_query` runs. The function first validates the transport layer through `parse_query_transport`. This function loads the IP header, checks for IPv4 with UDP protocol, rejects fragmented packets (which would require reassembly we don't implement), and verifies the destination matches our configured resolver. If all checks pass, it calculates where DNS data begins and populates the correlation key.
|
||||
|
||||
The `parse_dns_query` function then validates the DNS layer. It checks that the flags indicate a standard query (not a response), that exactly one question exists, and that the question name matches our configured domain. The question type and class must be A (address) and IN (internet). Only after all validation passes does it extract the transaction ID and insert into `pending_queries`.
|
||||
|
||||
Notice that `record_dns_query` always returns 1. This tells the kernel to continue processing the packet normally—we're observing, not blocking. DNS queries flow through unchanged.
|
||||
|
||||
### The Ingress Path: Learning from Responses
|
||||
|
||||
The ingress program `learn_dns_answer` reverses the perspective. Now we're looking at packets arriving from the resolver, so `parse_response_transport` checks that the source (not destination) matches the resolver IP and port. The correlation key gets populated with the same fields, but from the response's viewpoint.
|
||||
|
||||
The critical security check happens in `pending_query_is_live`. This function looks up the correlation key in `pending_queries`. If no entry exists—meaning we never saw a matching query—the response is rejected. If an entry exists but has expired, we delete it and reject the response. Only responses that match a live pending query proceed.
|
||||
|
||||
After confirming we have a legitimate response, `parse_response_question` validates the DNS header. It verifies this is a successful response (flags indicate "response" and "no error"), contains exactly one question matching our domain, and has at least one answer. `parse_direct_a_answer` then extracts the first A record, requiring the common `0xc00c` compressed name format, correct type and class, and a sane TTL between 1 and 86400 seconds.
|
||||
|
||||
When validation passes, the pending query gets deleted (it's been consumed), and the IP address gets added to `allowed_ips` with an expiration based on the DNS TTL. The `emit_event` call sends a `DNS_LEARNED` notification to userspace.
|
||||
|
||||
### The Connect Path: Enforcing Policy
|
||||
|
||||
The `enforce_dns_policy` function attaches to `cgroup/connect4`, which runs before every IPv4 TCP connect. The function first applies filtering: if we're targeting a specific process and this isn't it, allow the connection. If it's not TCP or not the protected port, allow the connection. These early returns minimize overhead for irrelevant traffic.
|
||||
|
||||
For connections that need policy enforcement, we look up the destination IP in `allowed_ips`. If found and not expired, we emit `DNS_ALLOWED` and return 1 (allow). If found but expired, we use an atomic compare-and-swap on `expired_reported` to emit exactly one `DNS_EXPIRED` event even under concurrent access. BPF atomic compare-and-exchange arrived in Linux 5.12, which sets this tool's minimum kernel version.
|
||||
|
||||
If the IP isn't in the map or is expired, we emit `DNS_DENIED` and return 0. The kernel translates return value 0 into `EPERM`, and the application's connect() fails immediately.
|
||||
|
||||
## The User-Space Program
|
||||
|
||||
The user-space side configures the read-only BPF data, attaches all three programs to a cgroup, and processes ring-buffer events. It also includes a self-test demo mode that exercises the complete trust chain.
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
|
||||
@@ -1021,20 +1061,38 @@ cleanup:
|
||||
}
|
||||
```
|
||||
|
||||
The domain is converted to DNS label form before the BPF object is loaded. For example, `lab.test` becomes `\x03lab\x04test\x00`, the exact byte sequence compared in the packet hooks. The configured resolver and ports are written into skeleton `rodata`, which makes them constants from the verifier's point of view.
|
||||
### Understanding the User-Space Control Flow
|
||||
|
||||
Normal mode keeps the links alive and prints policy decisions until the duration ends or a signal arrives. Demo mode runs the whole trust chain with loopback sockets. It first proves that a connection is blocked, sends an unsolicited answer and a wrong-ID answer, admits a correctly correlated one-second answer, then waits for that TTL to expire. These are useful checks because a simple “parse every DNS response” implementation would pass the happy path while remaining easy to poison.
|
||||
The user-space program follows a clear initialization sequence. First, `parse_options` handles command-line arguments, validating inputs and setting defaults. Demo mode automatically configures loopback addresses and non-standard ports to avoid conflicts with real DNS and web traffic.
|
||||
|
||||
## Build and Run
|
||||
The `encode_qname` function converts a domain name like `lab.test` into DNS label format: `\x03lab\x04test\x00`. Each label starts with a length byte followed by the label content. The final zero byte terminates the name. This encoding happens once at startup and gets written into the BPF skeleton's `rodata` section, where the verifier treats it as a constant.
|
||||
|
||||
Build the example with the repository's vendored libbpf and bpftool:
|
||||
The `prepare_runtime` function ties everything together. It opens the target cgroup directory, opens the BPF skeleton, configures all the `rodata` values (resolver IP, ports, domain name), loads the BPF programs, and attaches each program to the cgroup. The three separate links allow independent attachment and detachment. Finally, it creates a ring buffer consumer that calls `handle_event` for each notification from the kernel.
|
||||
|
||||
Normal mode enters `poll_policy_events`, which loops on the ring buffer until the duration expires or a signal arrives. Each event gets printed with the IP address, PID, TTL, and event type. Demo mode instead runs `run_demo`, which exercises the complete trust chain with synthetic DNS traffic and TCP connections.
|
||||
|
||||
### The Demo Mode: Proving the Security Properties
|
||||
|
||||
Demo mode serves as both a functional test and a demonstration of the security model. It runs entirely on loopback, using non-standard ports (15353 for DNS, 19090 for TCP) to avoid interfering with real services.
|
||||
|
||||
The test sequence starts by verifying that connections are blocked before any DNS traffic. Then it sends an unsolicited DNS response—a response that arrives without a preceding query. The BPF program should reject this because there's no matching entry in `pending_queries`. The demo verifies the connection is still blocked.
|
||||
|
||||
Next, it sends a legitimate DNS query and receives a response with the wrong transaction ID. The BPF program should also reject this because the transaction ID is part of the correlation key. Again, the demo verifies the connection remains blocked.
|
||||
|
||||
Finally, it sends a response with the correct transaction ID and a 1-second TTL. Now the connection should succeed. After waiting 1.3 seconds (longer than the TTL), the demo verifies the connection is blocked again.
|
||||
|
||||
This sequence proves that the tool correctly implements query-response correlation, rejects spoofing attempts, honors TTLs, and properly expires allowlist entries.
|
||||
|
||||
## Compilation and Execution
|
||||
|
||||
Build the example:
|
||||
|
||||
```bash
|
||||
cd src/55-dns-egress
|
||||
make
|
||||
```
|
||||
|
||||
Attach it to a service cgroup, watch one domain through its actual resolver, and protect TCP port 443:
|
||||
Attach it to a service cgroup and watch one domain through its resolver:
|
||||
|
||||
```bash
|
||||
sudo ./dns_egress \
|
||||
@@ -1044,13 +1102,13 @@ sudo ./dns_egress \
|
||||
--port 443
|
||||
```
|
||||
|
||||
The cgroup must contain the workload whose DNS packets and connections should share the policy state. TCP port 443 and DNS port 53 are the defaults; `--dns-port` selects another resolver port, and `--duration` adds a time limit. The built-in demo needs no external DNS server:
|
||||
The cgroup must contain the workload whose DNS packets and connections should share the policy state. TCP 443 and DNS 53 are defaults; `--dns-port` selects another resolver port, and `--duration` adds a time limit. The built-in demo needs no external DNS server:
|
||||
|
||||
```bash
|
||||
sudo ./dns_egress --demo
|
||||
```
|
||||
|
||||
A real demo run looks like this:
|
||||
Example output:
|
||||
|
||||
```text
|
||||
dns-egress attached cgroup=/sys/fs/cgroup domain=lab.test resolver=127.0.0.1 tcp_port=19090 dns_port=15353
|
||||
@@ -1068,25 +1126,25 @@ event=denied pid=1246 ip=127.0.0.1 ttl=1
|
||||
demo step=expired-answer result=blocked
|
||||
```
|
||||
|
||||
The three early `denied` events show that neither receiving DNS-shaped traffic nor seeing the right name is sufficient. `learned` appears only for the correlated response, `allowed` covers its live TTL, and `expired` is immediately followed by the denied connect.
|
||||
The three early `denied` events show that merely receiving DNS-shaped traffic or seeing the right domain name isn't enough. `learned` appears only for the correlated response, `allowed` covers its live TTL, and `expired` is immediately followed by a denied connect.
|
||||
|
||||
## Requirements
|
||||
|
||||
| Requirement | Details |
|
||||
|---|---|
|
||||
| Kernel | Linux 5.12 or newer; the newest dependency is BPF atomic compare-and-exchange |
|
||||
| Kernel | Linux 5.12+ (BPF atomic compare-and-exchange) |
|
||||
| Kernel config | `CONFIG_BPF`, `CONFIG_BPF_SYSCALL`, `CONFIG_BPF_JIT`, `CONFIG_CGROUP_BPF`, `CONFIG_DEBUG_INFO_BTF`, `CONFIG_INET` |
|
||||
| cgroup | cgroup v2, with the target workload placed below the attached directory |
|
||||
| Privileges | Root, or an equivalent set of BPF and network-administration capabilities |
|
||||
| Architecture and hardware | x86-64 is the declared and tested target; no special network hardware |
|
||||
| cgroup | cgroup v2, with workload placed below the attached directory |
|
||||
| Privileges | Root, or equivalent BPF and network capabilities |
|
||||
| Architecture | x86-64 tested; no special network hardware required |
|
||||
|
||||
## Scope
|
||||
## What This Example Doesn't Cover
|
||||
|
||||
The tool deliberately implements one exact domain, one resolver, one protected TCP port, IPv4 UDP DNS, and the first direct A answer. It understands the common compressed owner name `0xc00c`. CNAME chains, several answer layouts, TCP DNS, IPv6, DoH, and DoT need additional parsers or observation points. This compact scope keeps the important policy property visible: an IP enters the allowlist through a recent matching query and leaves it through DNS time.
|
||||
The tool deliberately implements one exact domain, one resolver, one protected TCP port, IPv4 UDP DNS, and the first direct A answer. It understands the common `0xc00c` compressed owner name. CNAME chains, alternate answer layouts, TCP DNS, IPv6, DoH, and DoT would need additional parsers or observation points. This compact scope keeps the important property visible: an IP enters the allowlist through a recent matching query and leaves through DNS time.
|
||||
|
||||
## Summary
|
||||
|
||||
This example turns an observed DNS result into a time-bounded connect policy. The egress and ingress hooks establish a trustworthy query-response relation, the TTL controls the address lifetime, and the connect hook enforces the result for the workload's protected port.
|
||||
This example turns observed DNS results into a time-bounded connect policy. The egress and ingress hooks establish a trustworthy query-response relation, the TTL controls address lifetime, and the connect hook enforces the result for the protected port.
|
||||
|
||||
> If you'd like to dive deeper into eBPF, check out our tutorial repository at <https://github.com/eunomia-bpf/bpf-developer-tutorial> or visit our website at <https://eunomia.dev/tutorials/>.
|
||||
|
||||
|
||||
@@ -1,24 +1,36 @@
|
||||
# eBPF 实战教程：用 cgroup BPF 构建 DNS 派生的 IP 允许列表
|
||||
# eBPF 实战教程：用 cgroup BPF 构建基于 DNS 的 IP 允许列表
|
||||
|
||||
假设一个服务只能通过 HTTPS 访问 `api.example.com`，其余目的地址都需要拒绝。静态 IP 允许列表很难完成这件事，因为 DNS 应答会变化，也会过期；到了 `connect()` 阶段，内核看到的又只是一个 IP 地址，并不知道它来自哪个域名。这里缺少的是一条短期关联，把应用刚刚完成的 DNS 解析和随后的连接连起来。
|
||||
假设你想让一个服务只能连接 `api.example.com`，其他地址一律拒绝。静态 IP 白名单听起来可行，但 DNS 应答会变化，TTL 也会过期。到了 `connect()` 阶段，内核只看到一个 IP 地址，根本不知道它来自哪个域名。这里缺少的是一条短期关联，把应用的 DNS 查询和随后的连接联系起来。
|
||||
|
||||
本课会把这条关联做成一个最小可运行的策略工具。它通过指定的 DNS 解析器观察一个域名，只从匹配的查询与响应中学习 IPv4 A 记录，并在 DNS TTL 有效期间允许访问一个 TCP 端口。
|
||||
本课就来构建这条关联。我们会观察特定域名的 DNS 流量，只从有效的查询响应对中学习 IP 地址，并在 TTL 有效期间允许连接。这是一个最小但完整的策略工具，展示了 cgroup BPF 如何在多个内核 hook 之间协调工作。
|
||||
|
||||
> 完整源代码：<https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/55-dns-egress>
|
||||
|
||||
## eBPF 在哪里执行策略
|
||||
## 理解问题：DNS 和 Connect 相互隔离
|
||||
|
||||
eBPF 可以让经过验证器检查的程序运行在 Linux 内核 hook 上，通过 map 保留状态，也可以把选出的事件交给用户态。cgroup BPF 让这些 hook 跟随一组工作负载：packet program 可以观察 cgroup 的出入流量，socket-address program 则能在内核发起 IPv4 连接之前允许或拒绝这次请求。
|
||||
当你的应用解析 `api.example.com` 时，DNS 响应包含一个 IP 地址和一个 TTL。但这些信息只存在于用户空间——解析器库缓存它，应用用这个 IP 调用 `connect()`，而内核完全不知道这个 IP 是怎么来的。这就形成了一个根本性的安全缺口。
|
||||
|
||||
这个例子在同一个 cgroup 上挂载三个程序。`cgroup_skb/egress` 识别指定的 DNS 问题并记录待匹配查询，`cgroup_skb/ingress` 只从与查询对应的响应中接纳地址，`cgroup/connect4` 对受保护的 TCP 端口作最终判断。ring buffer 负责报告 learned、allowed、denied 和 expired 事件，并不参与策略结果。
|
||||
考虑一个只应该访问你后端 API 的容器。传统防火墙可以阻止目标 IP，但无法执行"只允许连接最近 60 秒内从 api.example.com 解析出来的 IP"这样的策略。内核级策略和应用级 DNS 解析完全隔离运行。
|
||||
|
||||
先跟着一次成功的解析走一遍。应用发送 `lab.test` 的 A 查询，egress hook 保存解析器地址、客户端地址、客户端 UDP 端口和 DNS transaction ID，这份记录有效 5 秒。响应回来后，ingress hook 用反向报文重建同一个 key，检查响应标志与 question，再读取第一个直接 A 应答。通过检查的地址会和 TTL 推导出的过期时间一起进入允许列表，随后发往这个地址和受保护端口的 `connect()` 可以在 TTL 内成功。
|
||||
cgroup BPF 弥合了这个缺口。通过在同一个 cgroup 上挂载 packet hook 和 connect hook，我们可以观察 DNS 流量，然后根据观察到的结果执行连接策略。关键洞察是：cgroup BPF 程序通过 map 共享状态，从 DNS 查询到响应再到最终连接形成一条信任链。
|
||||
|
||||
待匹配查询就是这里的信任边界。未经请求的响应没有 pending key，transaction ID 不同的响应也会查到另一个 key，因此两者都无法写入允许列表。connect hook 还会再次比较过期时间，即使 LRU map 中仍保留着旧条目，地址也会按 DNS 时间自然失效。
|
||||
## 整体架构
|
||||
|
||||
## DNS 协议与事件结构
|
||||
eBPF 可以在多个内核 hook 上运行经过验证的程序，并通过 map 在它们之间共享状态。cgroup BPF 让这些 hook 跟随工作负载：packet program 可以观察 cgroup 的出入流量，socket-address program 则能在连接建立之前接受或拒绝请求。
|
||||
|
||||
共享头文件包含 BPF 程序需要解析的少量 DNS 布局，以及用户态接收的固定事件格式。
|
||||
我们的工具在同一个 cgroup 上挂载三个程序：
|
||||
|
||||
1. **`cgroup_skb/egress`** 监控出站 DNS 查询，记录我们正在等待响应的请求
|
||||
2. **`cgroup_skb/ingress`** 验证入站 DNS 响应，只从匹配的回复中学习 IP 地址
|
||||
3. **`cgroup/connect4`** 做最终决策——只允许连接到从有效 DNS 响应中学到的 IP，且仅在 TTL 有效期内
|
||||
|
||||
先跟着一次成功的解析走一遍。应用发送 `lab.test` 的 A 查询，egress hook 保存一个关联 key，包含解析器地址、客户端地址、客户端 UDP 端口和 DNS 事务 ID。这份记录有效期 5 秒。响应到达后，ingress hook 重建同一个 key，验证响应内容，并提取 IP 地址和 TTL。之后发往这个地址的 `connect()` 可以成功——但只在 TTL 有效期内。
|
||||
|
||||
待匹配查询就是这里的信任边界。未经请求的响应没有待匹配的 key。事务 ID 错误的响应会查找另一个 key 而找不到。两者都无法污染允许列表。即使条目还留在 LRU map 中，connect hook 也会重新检查过期时间，地址会随 DNS 时间自然失效。
|
||||
|
||||
## 数据结构详解
|
||||
|
||||
在深入代码之前，先理解让这一切工作的数据结构。共享头文件定义了 DNS 协议布局和我们报告给用户空间的事件：
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -69,11 +81,13 @@ struct dns_egress_event {
|
||||
#endif /* __DNS_EGRESS_H */
|
||||
```
|
||||
|
||||
这些协议结构直接描述线上字节，因此使用 packed 布局。事件同时携带 DNS TTL 和内核单调时钟下的绝对过期时间，命令行展示 TTL，BPF 程序则用时间戳作策略判断。
|
||||
协议结构使用 `packed` 属性，因为它们直接描述线上字节——不允许任何填充。`dns_header` 直接映射到每个查询和响应开头的 12 字节 DNS 头部。`dns_question` 结构跟在问题名（使用长度前缀标签）后面。`dns_a_answer` 期望常见的压缩格式，其中 name 指针是 `0xc00c`，指回问题部分。
|
||||
|
||||
## 三个 BPF hook
|
||||
我们发送给用户空间的每个事件同时携带 DNS TTL 和绝对过期时间戳。用户空间打印人类可读的 TTL，而 BPF 侧使用单调时间戳做决策。这种分离保持了内核逻辑的简洁——BPF 代码中不需要时间格式转换。
|
||||
|
||||
下面是完整的内核态程序。
|
||||
## BPF 程序：完整实现
|
||||
|
||||
下面是完整的内核态实现。虽然比某些例子长，但每个部分都有明确的职责。让我们先看完整代码，然后逐步分析关键部分：
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: GPL-2.0
|
||||
@@ -367,17 +381,43 @@ int enforce_dns_policy(struct bpf_sock_addr *ctx)
|
||||
}
|
||||
```
|
||||
|
||||
`pending_queries` 和 `allowed_ips` 都是固定容量的 LRU hash map。前者保存最多 5 秒的查询关联状态，后者保存处于 DNS 生命周期内的 IPv4 地址。LRU 淘汰把内存开销限制在确定范围内；当条目超过 1024 个时，最近活跃的查询和地址也会得到优先保留。
|
||||
### 理解 Map 设计
|
||||
|
||||
egress 路径先验证 IPv4、UDP、分片状态、解析器地址和端口，再检查报文是不是指定名称的单 question A 查询。只有这些字段全部成立，`record_dns_query()` 才会插入四字段关联 key。程序返回 `1`，DNS 报文仍然按正常路径继续传输。
|
||||
程序使用三个 BPF map，每个在信任链中都有特定用途。`pending_queries` map 是一个 LRU hash，保存 DNS 查询的关联状态。key 组合了四个字段——服务器 IP、客户端 IP、客户端端口和事务 ID——它们唯一标识一个查询/响应对。value 只存储过期时间戳，因为这就是我们验证时序所需的全部。LRU 淘汰确保即使在 DNS 洪水条件下内存也保持有界。
|
||||
|
||||
ingress 路径从反向 transport tuple 开始解析。`parse_response_question()` 补上 transaction ID，要求 pending entry 仍然有效，并验证这是包含指定 question 的成功响应。`parse_direct_a_answer()` 接受常见的 `0xc00c` 压缩 name pointer、IN class、A type、4 字节地址，以及 1 到 86400 秒的 TTL。应答通过以后，pending query 会被消费，解析出的 IP 才获得连接资格。
|
||||
`allowed_ips` map 也是 LRU hash，但只用 IPv4 地址作为 key。value 包含过期时间戳、以秒为单位的原始 TTL（用于日志）、对齐填充，以及一个标记是否已报告过期的 flag。这个 flag 防止多个线程在竞争访问过期条目时产生重复的 "expired" 事件。
|
||||
|
||||
`enforce_dns_policy()` 的范围很窄。发往其他目的端口的 TCP 连接直接通过；发往受保护端口的连接使用 `ctx->user_ip4` 查表，有效条目返回 `1`，缺失或过期条目返回 `0`，应用会收到 `EPERM`。`expired_reported` 上的 64 位 compare-and-swap 让多个线程同时访问旧地址时只产生一次 expired 通知。BPF atomic compare-and-exchange 在 Linux 5.12 引入，这也确定了工具的最低内核版本。
|
||||
`events` ring buffer 向用户空间发送通知。256KB 的大小可以容纳数千个事件而不会阻塞内核路径。Ring buffer 是 perf buffer 的现代替代品——它们更高效，提供更好的顺序保证。
|
||||
|
||||
## 加载策略并验证完整信任链
|
||||
### 出站路径：记录待处理查询
|
||||
|
||||
用户态程序会配置只读 BPF 数据，把三个程序附加到一个 cgroup，再消费 ring buffer 事件。
|
||||
当报文离开 cgroup 时，`record_dns_query` 运行。函数首先通过 `parse_query_transport` 验证传输层。这个函数加载 IP 头，检查是否为使用 UDP 协议的 IPv4，拒绝分片报文（需要我们没有实现的重组），并验证目标是否匹配配置的解析器。如果所有检查通过，它计算 DNS 数据的起始位置并填充关联 key。
|
||||
|
||||
然后 `parse_dns_query` 函数验证 DNS 层。它检查 flags 是否表示标准查询（不是响应），是否恰好有一个问题，以及问题名是否匹配我们配置的域名。问题类型和类必须是 A（地址）和 IN（互联网）。只有所有验证都通过后，它才提取事务 ID 并插入 `pending_queries`。
|
||||
|
||||
注意 `record_dns_query` 始终返回 1。这告诉内核继续正常处理报文——我们是在观察，不是阻断。DNS 查询原样流过。
|
||||
|
||||
### 入站路径：从响应中学习
|
||||
|
||||
入站程序 `learn_dns_answer` 反转了视角。现在我们看的是从解析器到达的报文，所以 `parse_response_transport` 检查源（不是目标）是否匹配解析器 IP 和端口。关联 key 用相同的字段填充，但从响应的视角来看。
|
||||
|
||||
关键的安全检查发生在 `pending_query_is_live`。这个函数在 `pending_queries` 中查找关联 key。如果没有条目存在——意味着我们从未看到匹配的查询——响应被拒绝。如果条目存在但已过期，我们删除它并拒绝响应。只有匹配活跃待处理查询的响应才能继续。
|
||||
|
||||
确认我们有合法响应后，`parse_response_question` 验证 DNS 头。它验证这是成功响应（flags 表示"响应"且"无错误"），恰好包含一个匹配我们域名的问题，且至少有一个应答。`parse_direct_a_answer` 然后提取第一个 A 记录，要求常见的 `0xc00c` 压缩 name 格式、正确的 type 和 class，以及 1 到 86400 秒之间的合理 TTL。
|
||||
|
||||
验证通过后，待处理查询被删除（已被消费），IP 地址以基于 DNS TTL 的过期时间添加到 `allowed_ips`。`emit_event` 调用向用户空间发送 `DNS_LEARNED` 通知。
|
||||
|
||||
### 连接路径：执行策略
|
||||
|
||||
`enforce_dns_policy` 函数挂载到 `cgroup/connect4`，在每个 IPv4 TCP 连接之前运行。函数首先应用过滤：如果我们针对特定进程且这不是它，允许连接。如果不是 TCP 或不是受保护端口，允许连接。这些提前返回最小化了无关流量的开销。
|
||||
|
||||
对于需要策略执行的连接，我们在 `allowed_ips` 中查找目标 IP。如果找到且未过期，我们发出 `DNS_ALLOWED` 并返回 1（允许）。如果找到但已过期，我们在 `expired_reported` 上使用原子 compare-and-swap，即使在并发访问下也只发出一个 `DNS_EXPIRED` 事件。BPF atomic compare-and-exchange 在 Linux 5.12 引入，这也确定了工具的最低内核版本。
|
||||
|
||||
如果 IP 不在 map 中或已过期，我们发出 `DNS_DENIED` 并返回 0。内核将返回值 0 转换为 `EPERM`，应用的 connect() 立即失败。
|
||||
|
||||
## 用户态程序
|
||||
|
||||
用户态程序配置只读 BPF 数据，把三个程序附加到 cgroup，并处理 ring buffer 事件。它还包含一个自测的 demo 模式，验证完整的信任链。
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
|
||||
@@ -1021,20 +1061,38 @@ cleanup:
|
||||
}
|
||||
```
|
||||
|
||||
域名会在 BPF object 加载前编码为 DNS label 格式，例如 `lab.test` 会变成 `\x03lab\x04test\x00`，这正是 packet hook 逐字节比较的内容。解析器与端口写入 skeleton 的 `rodata`，验证器可以把它们视为常量。
|
||||
### 理解用户态控制流程
|
||||
|
||||
普通模式会保持三个 link 存活，直到运行时间结束或收到退出信号。demo 模式用 loopback socket 走完信任链：先证明初始连接会被拒绝，再分别发送未经请求的应答和错误 ID 应答，随后接纳一个 TTL 为 1 秒的正确应答，最后等待它过期。这些检查很重要，因为“解析所有 DNS 响应”的简单实现也能通过成功路径，却很容易被伪造响应污染。
|
||||
用户态程序遵循清晰的初始化序列。首先，`parse_options` 处理命令行参数，验证输入并设置默认值。Demo 模式自动配置 loopback 地址和非标准端口，以避免与真实 DNS 和 Web 流量冲突。
|
||||
|
||||
`encode_qname` 函数将域名如 `lab.test` 转换为 DNS label 格式：`\x03lab\x04test\x00`。每个 label 以长度字节开头，后跟 label 内容。最后的零字节终止名称。这个编码在启动时只发生一次，并写入 BPF skeleton 的 `rodata` 部分，验证器将其视为常量。
|
||||
|
||||
`prepare_runtime` 函数把一切串联起来。它打开目标 cgroup 目录，打开 BPF skeleton，配置所有 `rodata` 值（解析器 IP、端口、域名），加载 BPF 程序，并将每个程序附加到 cgroup。三个独立的 link 允许独立的附加和分离。最后，它创建一个 ring buffer 消费者，为每个内核通知调用 `handle_event`。
|
||||
|
||||
普通模式进入 `poll_policy_events`，在 ring buffer 上循环直到持续时间结束或收到信号。每个事件都会打印 IP 地址、PID、TTL 和事件类型。Demo 模式则运行 `run_demo`，用合成的 DNS 流量和 TCP 连接验证完整的信任链。
|
||||
|
||||
### Demo 模式：证明安全属性
|
||||
|
||||
Demo 模式既是功能测试，也是安全模型的演示。它完全在 loopback 上运行，使用非标准端口（DNS 15353，TCP 19090）以避免干扰真实服务。
|
||||
|
||||
测试序列首先验证在任何 DNS 流量之前连接被阻止。然后发送一个未经请求的 DNS 响应——一个没有前置查询就到达的响应。BPF 程序应该拒绝它，因为 `pending_queries` 中没有匹配的条目。Demo 验证连接仍然被阻止。
|
||||
|
||||
接下来，它发送一个合法的 DNS 查询并收到一个事务 ID 错误的响应。BPF 程序也应该拒绝它，因为事务 ID 是关联 key 的一部分。同样，demo 验证连接保持阻止状态。
|
||||
|
||||
最后，它发送一个事务 ID 正确、TTL 为 1 秒的响应。现在连接应该成功。等待 1.3 秒（超过 TTL）后，demo 验证连接再次被阻止。
|
||||
|
||||
这个序列证明了工具正确实现了查询响应关联、拒绝欺骗尝试、遵守 TTL，并正确使允许列表条目过期。
|
||||
|
||||
## 编译和运行
|
||||
|
||||
使用仓库内置的 libbpf 与 bpftool 构建示例：
|
||||
构建示例：
|
||||
|
||||
```bash
|
||||
cd src/55-dns-egress
|
||||
make
|
||||
```
|
||||
|
||||
将它挂到服务 cgroup，通过服务实际使用的解析器观察一个域名，并保护 TCP 443 端口：
|
||||
将它挂到服务 cgroup，通过解析器观察一个域名：
|
||||
|
||||
```bash
|
||||
sudo ./dns_egress \
|
||||
@@ -1044,13 +1102,13 @@ sudo ./dns_egress \
|
||||
--port 443
|
||||
```
|
||||
|
||||
这个 cgroup 需要包含目标工作负载，让它的 DNS 报文和连接共享同一份策略状态。TCP 443 和 DNS 53 是默认端口，`--dns-port` 可以选择其他解析器端口，`--duration` 则为运行时间设置上限。内置 demo 不依赖外部 DNS 服务器：
|
||||
cgroup 需要包含目标工作负载，让它的 DNS 报文和连接共享同一份策略状态。TCP 443 和 DNS 53 是默认端口；`--dns-port` 选择其他解析器端口，`--duration` 设置时间上限。内置 demo 不需要外部 DNS 服务器：
|
||||
|
||||
```bash
|
||||
sudo ./dns_egress --demo
|
||||
```
|
||||
|
||||
一次真实运行的输出如下：
|
||||
示例输出：
|
||||
|
||||
```text
|
||||
dns-egress attached cgroup=/sys/fs/cgroup domain=lab.test resolver=127.0.0.1 tcp_port=19090 dns_port=15353
|
||||
@@ -1068,25 +1126,25 @@ event=denied pid=1246 ip=127.0.0.1 ttl=1
|
||||
demo step=expired-answer result=blocked
|
||||
```
|
||||
|
||||
前三个 denied 说明，仅仅收到 DNS 格式的流量或者看到正确域名还不足以学习地址。只有关联正确的响应会产生 learned，allowed 覆盖它的有效 TTL，expired 之后的连接则立刻回到 denied。
|
||||
前三个 `denied` 说明仅仅收到 DNS 格式的流量或看到正确的域名还不够。只有关联正确的响应才会产生 `learned`，`allowed` 覆盖它的有效 TTL，`expired` 之后的连接立刻回到 denied。
|
||||
|
||||
## 环境要求
|
||||
|
||||
| 要求 | 说明 |
|
||||
|---|---|
|
||||
| 内核 | Linux 5.12 或更高版本，最新依赖来自 BPF atomic compare-and-exchange |
|
||||
| 内核 | Linux 5.12+（BPF atomic compare-and-exchange） |
|
||||
| 内核配置 | `CONFIG_BPF`、`CONFIG_BPF_SYSCALL`、`CONFIG_BPF_JIT`、`CONFIG_CGROUP_BPF`、`CONFIG_DEBUG_INFO_BTF`、`CONFIG_INET` |
|
||||
| cgroup | cgroup v2，目标工作负载位于挂载目录之下 |
|
||||
| 权限 | root，或者等价的 BPF 与网络管理 capability |
|
||||
| 架构与硬件 | 当前声明并完成测试的目标是 x86-64，不需要特殊网卡 |
|
||||
| cgroup | cgroup v2，工作负载位于挂载目录之下 |
|
||||
| 权限 | root，或等价的 BPF 与网络 capability |
|
||||
| 架构 | x86-64 已测试；不需要特殊网卡 |
|
||||
|
||||
## 实现范围
|
||||
## 本例未涵盖的内容
|
||||
|
||||
这个工具处理一个精确域名、一个解析器、一个受保护的 TCP 端口、IPv4 UDP DNS 和第一个直接 A 应答，并识别常见的 `0xc00c` owner name。CNAME 链、多种 answer 布局、TCP DNS、IPv6、DoH 和 DoT 需要更多解析逻辑或新的观察位置。这个紧凑范围保留了策略最关键的性质：IP 通过最近一次匹配查询进入允许列表，再按照 DNS 时间退出。
|
||||
工具刻意只实现一个精确域名、一个解析器、一个受保护的 TCP 端口、IPv4 UDP DNS 和第一个直接 A 应答。它识别常见的 `0xc00c` 压缩 owner name。CNAME 链、其他 answer 布局、TCP DNS、IPv6、DoH 和 DoT 需要额外的解析器或新的观察点。这个紧凑范围保留了重要属性的可见性：IP 通过最近一次匹配查询进入允许列表，再通过 DNS 时间退出。
|
||||
|
||||
## 总结
|
||||
|
||||
这个例子把观察到的 DNS 结果变成有时间边界的 connect 策略。egress 与 ingress hook 建立可信的查询响应关联，TTL 管理地址生命周期，connect hook 则对工作负载的受保护端口执行结果。
|
||||
这个例子把观察到的 DNS 结果变成有时间边界的 connect 策略。egress 和 ingress hook 建立可信的查询响应关联，TTL 控制地址生命周期，connect hook 对受保护端口执行结果。
|
||||
|
||||
> 如果你想深入了解 eBPF，请查看我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或访问我们的网站 <https://eunomia.dev/tutorials/>。
|
||||
|
||||
@@ -1094,6 +1152,6 @@ demo step=expired-answer result=blocked
|
||||
|
||||
- [BPF ring buffer](https://docs.kernel.org/bpf/ringbuf.html)
|
||||
- [BPF LRU hash map](https://docs.kernel.org/bpf/map_hash.html)
|
||||
- [BPF atomic compare-and-exchange 引入 commit](https://github.com/torvalds/linux/commit/5ffa25502b5ab3d639829a2d1e316cff7f59a41e)
|
||||
- [BPF atomic compare-and-exchange commit](https://github.com/torvalds/linux/commit/5ffa25502b5ab3d639829a2d1e316cff7f59a41e)
|
||||
- [Control Group v2](https://docs.kernel.org/admin-guide/cgroup-v2.html)
|
||||
- [RFC 1035：Domain Names — Implementation and Specification](https://www.rfc-editor.org/rfc/rfc1035.html)
|
||||
|
||||
@@ -1,24 +1,51 @@
|
||||
# eBPF Tutorial by Example: Index TC Flows in Two Refcounted rbtrees
|
||||
# eBPF Tutorial: Building a Top-Flow Monitor with Dual Rbtree Indexing
|
||||
|
||||
A flow monitor has two different lookup problems. Every packet needs to find its five-tuple quickly so the counters can be updated. At reporting time, the same records need to appear in traffic order so the busiest flows come first. One ordering cannot answer both questions efficiently, while two independent copies would let counters and lifetimes drift apart.
|
||||
Ever tried building a network flow monitor and hit this wall? Every packet needs lightning-fast lookup to update counters, but at report time you want flows sorted by traffic volume. One index can't do both efficiently, and keeping two separate copies means counters drift apart. What if you could have both views of the same data, always in sync?
|
||||
|
||||
This tutorial builds a TC egress flow index around one object with two views. Each IPv4 TCP or UDP flow is stored in an identity rbtree and a byte-ranked rbtree at the same time. BPF object allocation creates the record, a BPF refcount gives both trees ownership, and rbtree traversal produces top-flow output without rebuilding the index in user space.
|
||||
This tutorial builds a TC egress flow monitor that solves exactly this problem. We'll index each IPv4 TCP or UDP flow in two red-black trees simultaneously: one keyed by five-tuple for fast packet lookup, another sorted by bytes for instant top-flow output. The magic ingredient is BPF refcounting, which lets both trees own the same flow record without duplication.
|
||||
|
||||
> Complete source code: <https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/56-tc-flow-index>
|
||||
|
||||
## Why the Same Object Can Belong to Two Trees
|
||||
## The Dual-Index Challenge
|
||||
|
||||
eBPF runs verifier-checked programs on kernel data paths such as TC and can keep structured state between invocations. Linux 6.4 added `bpf_refcount_acquire()`, which lets a dynamically allocated BPF object gain another owning reference. Linux 6.16 added `bpf_rbtree_root()`, `bpf_rbtree_left()`, and `bpf_rbtree_right()`, making ordered search and traversal available to BPF programs and completing the feature set used here.
|
||||
Network monitoring tools face a fundamental tension. When a packet arrives, you need to find the matching flow record by its five-tuple (source IP, destination IP, source port, destination port, protocol) in microseconds. But when the user asks "show me the top 10 flows," you need those same records sorted by traffic volume.
|
||||
|
||||
Each `flow_entry` embeds one `bpf_refcount` and two different `bpf_rb_node` fields. `identity_root` orders `by_identity` by source address, destination address, source port, destination port, and protocol. `traffic_root` orders `by_traffic` by descending bytes, then packets, then the five-tuple for a deterministic tie break. Both nodes lead back to the same counters and timestamps through `container_of()`.
|
||||
Traditional solutions either maintain two independent data structures (which can drift out of sync) or rebuild the ranking on demand (which burns CPU and delays output). Neither feels right.
|
||||
|
||||
Follow the first packet of a new flow. The TC program searches the identity tree and finds no match. It allocates a `flow_entry`, then calls `bpf_refcount_acquire()` to obtain a second owning reference. The first `bpf_rbtree_add()` transfers one reference to the identity tree, and the second call transfers the other to the traffic tree. Once the critical section ends, no raw pointer escapes; the verifier can prove that both collections own the object.
|
||||
The approach we take here is different. Each flow record lives in memory exactly once, but it participates in two different orderings through two embedded tree nodes. When you update the byte counter, both views see the change immediately because they point to the same object. This is possible thanks to three recent eBPF features:
|
||||
|
||||
The next packet finds the entry by identity, but adding bytes changes its traffic rank. The program removes only `by_traffic`; removal returns that tree's owning reference. It updates the shared object and inserts the traffic node again at its new position. The identity node stays in place, so lookup order remains valid throughout the update.
|
||||
- **BPF object allocation** (`bpf_obj_new`): Create dynamically allocated structures in BPF programs
|
||||
- **BPF refcounting** (`bpf_refcount_acquire`): Give multiple owners shared access to one object
|
||||
- **Rbtree traversal** (`bpf_rbtree_root/left/right`): Search and walk trees without removing nodes
|
||||
|
||||
## Shared Flow and Cursor Types
|
||||
Linux 6.4 introduced refcounting, and 6.16 completed the picture with tree traversal. Together, they enable data structures that were previously impossible in BPF.
|
||||
|
||||
The shared header defines the five-tuple, the snapshot returned from BPF, and the cursor used to continue an ordered walk.
|
||||
## How One Object Joins Two Trees
|
||||
|
||||
Let's trace what happens when a new flow appears. The TC program sees a packet with five-tuple (10.0.0.1, 10.0.0.2, 50000, 80, TCP). It searches the identity tree by key and finds nothing. Time to create a new entry.
|
||||
|
||||
First, `bpf_obj_new()` allocates a `flow_entry`. This structure embeds a `bpf_refcount` and two `bpf_rb_node` fields, one for each tree. Right after allocation, the object has exactly one owning reference.
|
||||
|
||||
Next, `bpf_refcount_acquire()` creates a second owning reference to the same object. Now we have two references pointing to one piece of memory. The first `bpf_rbtree_add()` transfers one reference to the identity tree. The second call transfers the other to the traffic tree. At this point, both trees jointly own the flow entry, and the verifier knows no raw pointers escaped.
|
||||
|
||||
When a subsequent packet for this flow arrives, we find the existing entry in the identity tree. But adding bytes changes the traffic ranking. The program removes only the traffic-tree node (which returns its owning reference), updates the counters, and reinserts at the new position. The identity node never moves, so lookup remains valid throughout.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The implementation splits into three files:
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `tc_flow_index.h` | Shared structures: five-tuple, snapshot, and cursor |
|
||||
| `bpf_experimental.h` | Kfunc declarations for BPF graph APIs |
|
||||
| `tc_flow_index.bpf.c` | BPF program: TC hook and snapshot syscall |
|
||||
| `tc_flow_index.c` | User-space loader, demo traffic, and output |
|
||||
|
||||
The BPF side handles packet parsing, tree maintenance, and cursor-based snapshot iteration. The user side attaches the TC program, optionally generates test traffic, and queries the index through `BPF_PROG_TEST_RUN`.
|
||||
|
||||
## Shared Data Structures
|
||||
|
||||
The header file defines the flow key, snapshot result, and traversal cursor:
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -55,11 +82,11 @@ struct flow_cursor {
|
||||
#endif /* __TC_FLOW_INDEX_H */
|
||||
```
|
||||
|
||||
Addresses and ports remain in network byte order inside `flow_key`, which lets the BPF side copy packet fields directly. User space converts them only when formatting the table.
|
||||
The `flow_key` keeps addresses and ports in network byte order, so the BPF program can copy packet fields directly without conversion. User space converts to host order only when formatting output.
|
||||
|
||||
## Experimental kfunc Declarations
|
||||
## Experimental Kfunc Declarations
|
||||
|
||||
BPF graph APIs are kfuncs rather than stable UAPI helpers. The small compatibility header declares the exact functions used by the program and wraps the allocator calls with local BTF type IDs.
|
||||
BPF graph APIs use kfuncs rather than stable UAPI helpers. This compatibility header declares the functions our program uses:
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -104,11 +131,11 @@ bpf_rbtree_right(struct bpf_rb_root *root, struct bpf_rb_node *node) __ksym;
|
||||
#endif /* __TC_FLOW_INDEX_EXPERIMENTAL_H */
|
||||
```
|
||||
|
||||
`__contains(flow_entry, by_identity)` and its traffic counterpart tell the verifier which containing type and member belong to each root. The verifier combines that type relation with the neighboring spin lock to enforce graph ownership and critical-section rules. These kfunc interfaces can evolve between kernels, which is why the tutorial records a concrete minimum version and keeps the declarations beside the example.
|
||||
The `__contains(flow_entry, by_identity)` annotation tells the verifier which containing type and member belong to each root. Combined with the neighboring spin lock, this lets the verifier enforce ownership and critical-section rules. These interfaces may evolve between kernels, which is why we note a concrete minimum version below.
|
||||
|
||||
## Maintaining Both Indexes at TC Egress
|
||||
## The BPF Program
|
||||
|
||||
Here is the complete BPF program.
|
||||
Here's the complete BPF implementation. We'll walk through the key parts afterward.
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: GPL-2.0
|
||||
@@ -399,17 +426,23 @@ int snapshot_next(void *ctx)
|
||||
}
|
||||
```
|
||||
|
||||
`index_lock`, `identity_root`, and `traffic_root` share one private map value. Every search or mutation holds that lock. A bounded 32-step binary-tree walk keeps verifier-visible control flow finite; with at most 4096 entries, the balanced rbtree needs far fewer levels.
|
||||
### Understanding the Flow Entry
|
||||
|
||||
Allocation happens after the first locked lookup. The program then locks again and repeats the lookup before insertion. This second check handles two CPUs seeing the same new flow: one object enters both trees, while the loser updates the existing entry and drops both references it allocated. Allocation, refcount, capacity, and re-ranking failures have separate counters so ownership problems remain visible.
|
||||
The `flow_entry` structure is the heart of this design. It embeds one `bpf_refcount` for ownership tracking and two `bpf_rb_node` fields for participation in both trees. The lock, `identity_root`, and `traffic_root` share a private map value, ensuring all access is serialized.
|
||||
|
||||
`parse_flow()` admits non-fragmented IPv4 TCP and UDP packets and builds the key from Ethernet, IP, and transport headers. `index_egress_flow()` always returns `TC_ACT_OK`; this tool observes and indexes traffic without changing packet delivery.
|
||||
### The Packet Path
|
||||
|
||||
The `snapshot_next` program reads the traffic tree under the same lock. With an empty cursor it returns `bpf_rbtree_first()`. Later calls search for the first entry after the previous `(bytes, packets, key)` tuple. The result and cursor live in BSS, so user space can request one ranked entry at a time without receiving a per-packet event stream.
|
||||
When `index_egress_flow` runs, `parse_flow()` extracts the five-tuple from non-fragmented IPv4 TCP and UDP packets. The main function always returns `TC_ACT_OK` because we're observing, not filtering.
|
||||
|
||||
## Attaching TC and Reading a Stable Snapshot
|
||||
The `update_flow()` function first searches under lock. If the flow exists, it removes the traffic node, updates counters, and reinserts. If not, it allocates outside the lock, then locks again for a second lookup. This double-check handles two CPUs discovering the same new flow simultaneously: one wins the insertion race, the other updates the existing entry and drops its unused references.
|
||||
|
||||
The user-space program attaches the classifier, generates optional demo traffic, detaches before the snapshot, then calls the snapshot program through `BPF_PROG_TEST_RUN`.
|
||||
### The Snapshot Mechanism
|
||||
|
||||
The `snapshot_next` syscall program reads the traffic tree under the same lock. With an empty cursor, it returns `bpf_rbtree_first()`. Subsequent calls search for the first entry after the previous position. Since the cursor and result live in BSS, user space can fetch one ranked entry at a time without receiving a per-packet event stream.
|
||||
|
||||
## The User-Space Program
|
||||
|
||||
The loader attaches the TC program, optionally generates demo traffic, detaches before reading, then queries through `BPF_PROG_TEST_RUN`:
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
|
||||
@@ -730,11 +763,9 @@ cleanup:
|
||||
}
|
||||
```
|
||||
|
||||
Detaching before output freezes packet updates, so the cursor walks one stable traffic ordering. `snapshot_next()` copies each result from BSS and advances the cursor until `found` becomes zero or `--top` entries have been printed. The index itself, including both trees and their object ownership, stays in BPF memory until the skeleton is destroyed.
|
||||
Detaching before output freezes packet updates, so the cursor walks a stable traffic ordering. The demo mode creates three UDP flows over loopback: 2 packets at 100 bytes, 4 at 300 bytes, and 6 at 700 bytes. With headers, these become 284, 1368, and 4452 bytes total, producing a deterministic ranking.
|
||||
|
||||
Demo mode opens three different UDP socket pairs and sends 2 × 100-byte, 4 × 300-byte, and 6 × 700-byte payloads over loopback. Ethernet, IPv4, and UDP headers make their observed totals 284, 1368, and 4452 bytes. This creates a deterministic ranking while exercising repeated remove-update-reinsert operations.
|
||||
|
||||
## Build and Run
|
||||
## Building and Running
|
||||
|
||||
Build the tool:
|
||||
|
||||
@@ -743,19 +774,19 @@ cd src/56-tc-flow-index
|
||||
make
|
||||
```
|
||||
|
||||
Index egress traffic on an interface for 30 seconds and print its ten busiest flows:
|
||||
Monitor real egress traffic for 30 seconds:
|
||||
|
||||
```bash
|
||||
sudo ./tc_flow_index --interface eth0 --duration 30 --top 10
|
||||
```
|
||||
|
||||
Run the deterministic loopback demo:
|
||||
Run the deterministic demo:
|
||||
|
||||
```bash
|
||||
sudo ./tc_flow_index --demo --top 3
|
||||
```
|
||||
|
||||
A real demo run shows the two-tree ordering directly:
|
||||
Expected demo output:
|
||||
|
||||
```text
|
||||
Indexing IPv4 TCP/UDP egress flows on lo for 0 seconds.
|
||||
@@ -768,25 +799,30 @@ SOURCE DESTINATION PROTO PACKETS BYTES COMM
|
||||
observed_packets=12 indexed_flows=3 dropped_new=0 allocation_failures=0 refcount_failures=0 rank_update_failures=0
|
||||
```
|
||||
|
||||
The table is already descending by `BYTES` when it reaches user space. Twelve packets created three identity entries, and all ownership-related counters remain zero.
|
||||
The table arrives pre-sorted by bytes. Twelve packets created three identity entries, and all ownership counters remain zero, confirming correct dual-tree management.
|
||||
|
||||
## Requirements
|
||||
|
||||
| Requirement | Details |
|
||||
|---|---|
|
||||
| Kernel | Linux 6.16 or newer for refcounted BPF objects plus searchable rbtree kfuncs |
|
||||
| Kernel config | `CONFIG_BPF`, `CONFIG_BPF_SYSCALL`, `CONFIG_BPF_JIT`, `CONFIG_DEBUG_INFO_BTF`, `CONFIG_NET_SCHED`, `CONFIG_NET_CLS_BPF` |
|
||||
| Privileges | Root, or equivalent BPF and network-administration capabilities |
|
||||
| Interface | Any interface that can host a `clsact` egress program |
|
||||
| Architecture and hardware | x86-64 is the declared and tested target; an ordinary network interface is sufficient |
|
||||
|-------------|---------|
|
||||
| Kernel | Linux 6.16+ (refcounted BPF objects + rbtree search kfuncs) |
|
||||
| Config | `CONFIG_BPF`, `CONFIG_BPF_SYSCALL`, `CONFIG_BPF_JIT`, `CONFIG_DEBUG_INFO_BTF`, `CONFIG_NET_SCHED`, `CONFIG_NET_CLS_BPF` |
|
||||
| Privileges | Root or CAP_BPF + CAP_NET_ADMIN |
|
||||
| Interface | Any interface supporting clsact egress |
|
||||
| Architecture | Tested on x86-64 |
|
||||
|
||||
## Scope
|
||||
## Extending the Design
|
||||
|
||||
The index covers non-fragmented IPv4 TCP and UDP egress traffic and remembers the command name observed when a flow is first created. It has a fixed 4096-flow capacity and keeps entries until the program exits, which suits bounded experiments and short observation windows. A long-running service can add idle expiry and remove both nodes before dropping their references.
|
||||
This example covers non-fragmented IPv4 TCP/UDP egress with a fixed 4096-entry capacity. For production use, consider:
|
||||
|
||||
- **Idle expiry**: Remove flows that haven't seen traffic for N seconds, freeing both tree nodes before dropping references
|
||||
- **Ingress indexing**: Add a second TC program for incoming packets
|
||||
- **IPv6 support**: Extend the flow key structure
|
||||
- **Real-time streaming**: Use a ring buffer alongside the trees for per-packet events
|
||||
|
||||
## Summary
|
||||
|
||||
This example uses BPF object ownership to maintain two useful orderings over one flow record. The identity tree handles per-packet updates, the traffic tree supplies top-flow traversal, and refcounts let both collections share the object without duplicating its state.
|
||||
This tutorial showed how BPF object ownership enables sophisticated data structures that weren't possible before. The identity tree handles fast five-tuple lookups, the traffic tree provides instant top-flow rankings, and refcounting lets both share one object without state duplication. The technique generalizes to any scenario where you need multiple orderings over the same dataset.
|
||||
|
||||
> If you'd like to dive deeper into eBPF, check out our tutorial repository at <https://github.com/eunomia-bpf/bpf-developer-tutorial> or visit our website at <https://eunomia.dev/tutorials/>.
|
||||
|
||||
|
||||
@@ -1,24 +1,51 @@
|
||||
# eBPF 实战教程：用两棵 refcounted rbtree 索引 TC 流量
|
||||
# eBPF 实战教程：用双红黑树索引构建 Top-Flow 监控器
|
||||
|
||||
流量监控会遇到两种完全不同的查询。每个报文到来时，需要按五元组快速找到原有 flow 并更新计数；输出报告时，又希望同一批记录按流量大小排列，让最繁忙的 flow 出现在前面。一种排序很难同时高效回答两个问题，两份独立副本又会让计数和生命周期逐渐分叉。
|
||||
做过流量监控的人可能都遇到过这个问题：每个报文到达时，需要按五元组快速找到对应的 flow 记录更新计数器；但输出报告时，又希望同一批数据按流量大小排序，让最繁忙的 flow 排在前面。一份索引很难同时满足两种需求，而维护两份独立副本又会让计数器逐渐失去同步。有没有办法让两种视图共享同一份数据，始终保持一致？
|
||||
|
||||
本课围绕一个对象的两种视图构建 TC egress flow index。每条 IPv4 TCP 或 UDP 流会同时进入 identity rbtree 和按字节数排序的 traffic rbtree，BPF object allocator 创建记录，BPF refcount 让两棵树共同持有对象，rbtree traversal 则直接产出 top-flow 结果，用户态无需重建索引。
|
||||
本教程构建的 TC egress 流量监控器正是为了解决这个问题。我们把每条 IPv4 TCP 或 UDP 流同时索引到两棵红黑树中：一棵按五元组键值排序，用于快速报文查找；另一棵按字节数排序，用于即时输出 top-flow 结果。核心技术是 BPF 引用计数，它让两棵树能够共同拥有同一条 flow 记录，无需复制数据。
|
||||
|
||||
> 完整源代码：<https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/56-tc-flow-index>
|
||||
|
||||
## 为什么同一个对象可以进入两棵树
|
||||
## 双索引的挑战
|
||||
|
||||
eBPF 可以让经过验证器检查的程序运行在 TC 等内核数据路径上，并在多次调用之间保留结构化状态。Linux 6.4 引入 `bpf_refcount_acquire()`，动态分配的 BPF object 可以获得新的 owning reference。Linux 6.16 又加入 `bpf_rbtree_root()`、`bpf_rbtree_left()` 和 `bpf_rbtree_right()`，BPF 程序开始能够搜索并遍历有序树，也补齐了本例使用的整组能力。
|
||||
网络监控工具面临一个根本性的矛盾。报文到达时，需要在微秒级别内按五元组（源 IP、目的 IP、源端口、目的端口、协议）找到对应的 flow 记录。但当用户查询"显示前 10 条流量最大的 flow"时，又需要这些记录按流量大小排序。
|
||||
|
||||
每个 `flow_entry` 内嵌一个 `bpf_refcount` 和两个不同的 `bpf_rb_node`。`identity_root` 通过 `by_identity` 按源地址、目的地址、源端口、目的端口和协议排序；`traffic_root` 通过 `by_traffic` 按 bytes、packets 和五元组排序，其中 bytes 与 packets 都是降序，五元组负责稳定处理并列项。两个 node 最终都能通过 `container_of()` 回到同一份计数和时间戳。
|
||||
传统方案要么维护两份独立的数据结构（可能出现不一致），要么在查询时重新排序（消耗 CPU 并延迟输出）。两种方案都不够理想。
|
||||
|
||||
先看一条新 flow 的第一个报文。TC 程序搜索 identity tree，没有发现匹配项，于是分配一个 `flow_entry`，再调用 `bpf_refcount_acquire()` 取得第二个 owning reference。第一次 `bpf_rbtree_add()` 把一个 reference 交给 identity tree，第二次调用把另一个交给 traffic tree。临界区结束以后没有裸指针逃逸，验证器可以证明两棵 collection 都拥有这个对象。
|
||||
我们采用的方法完全不同。每条 flow 记录在内存中只存在一份，但通过两个内嵌的树节点参与两种不同的排序。更新字节计数器时，两种视图立即看到变化，因为它们指向同一个对象。这得益于三个最新的 eBPF 特性：
|
||||
|
||||
后续报文会在 identity tree 找到条目，但 bytes 增加后，原来的 traffic 顺序已经失效。程序只从 traffic tree 移除 `by_traffic`，remove 返回这棵树持有的 owning reference；计数更新以后，同一个 traffic node 会重新插入正确位置。identity node 始终留在原位，因此查找顺序在整个更新期间保持有效。
|
||||
- **BPF 对象分配**（`bpf_obj_new`）：在 BPF 程序中创建动态分配的结构体
|
||||
- **BPF 引用计数**（`bpf_refcount_acquire`）：让多个所有者共享同一个对象
|
||||
- **红黑树遍历**（`bpf_rbtree_root/left/right`）：不移除节点也能搜索和遍历树
|
||||
|
||||
## 共享的 flow 与 cursor 类型
|
||||
Linux 6.4 引入了引用计数，6.16 又添加了树遍历功能。两者结合，使得以前在 BPF 中不可能实现的数据结构成为现实。
|
||||
|
||||
共享头文件定义五元组、BPF 返回的 snapshot，以及继续有序遍历所需的 cursor。
|
||||
## 一个对象如何加入两棵树
|
||||
|
||||
我们来追踪一条新 flow 出现时会发生什么。TC 程序看到一个五元组为 (10.0.0.1, 10.0.0.2, 50000, 80, TCP) 的报文。它在 identity 树中按键值搜索，没有找到匹配项。是时候创建新条目了。
|
||||
|
||||
首先，`bpf_obj_new()` 分配一个 `flow_entry`。这个结构体内嵌一个 `bpf_refcount` 和两个 `bpf_rb_node` 字段，分别对应两棵树。刚分配时，对象恰好有一个 owning reference。
|
||||
|
||||
接下来，`bpf_refcount_acquire()` 为同一个对象创建第二个 owning reference。现在我们有两个引用指向同一块内存。第一次 `bpf_rbtree_add()` 把一个引用交给 identity 树。第二次调用把另一个交给 traffic 树。此时，两棵树共同拥有这个 flow entry，验证器知道没有裸指针逃逸。
|
||||
|
||||
当这条 flow 的后续报文到达时，我们在 identity 树中找到已有条目。但增加字节数会改变 traffic 排名。程序只移除 traffic 树节点（返回其 owning reference），更新计数器，然后在新位置重新插入。identity 节点始终不动，所以查找在整个过程中保持有效。
|
||||
|
||||
## 架构概览
|
||||
|
||||
实现分为三个文件：
|
||||
|
||||
| 文件 | 作用 |
|
||||
|------|------|
|
||||
| `tc_flow_index.h` | 共享结构体：五元组、快照、游标 |
|
||||
| `bpf_experimental.h` | BPF graph API 的 kfunc 声明 |
|
||||
| `tc_flow_index.bpf.c` | BPF 程序：TC hook 和快照系统调用 |
|
||||
| `tc_flow_index.c` | 用户态加载器、演示流量和输出 |
|
||||
|
||||
BPF 端负责报文解析、树维护和基于游标的快照迭代。用户端挂载 TC 程序，可选生成测试流量，并通过 `BPF_PROG_TEST_RUN` 查询索引。
|
||||
|
||||
## 共享数据结构
|
||||
|
||||
头文件定义 flow key、快照结果和遍历游标：
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -55,11 +82,11 @@ struct flow_cursor {
|
||||
#endif /* __TC_FLOW_INDEX_H */
|
||||
```
|
||||
|
||||
`flow_key` 中的地址和端口保留 network byte order，BPF 侧可以直接复制报文字段，只有用户态格式化表格时才作转换。
|
||||
`flow_key` 中的地址和端口保留网络字节序，BPF 程序可以直接从报文复制字段。只有用户态格式化输出时才转换为主机字节序。
|
||||
|
||||
## 实验性 kfunc 声明
|
||||
|
||||
BPF graph API 使用 kfunc，而不是稳定 UAPI helper。这个兼容头文件声明程序实际调用的函数，并用本地 BTF type ID 包装 object allocator。
|
||||
BPF graph API 使用 kfunc 而非稳定的 UAPI helper。这个兼容头文件声明了程序使用的函数：
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -104,11 +131,11 @@ bpf_rbtree_right(struct bpf_rb_root *root, struct bpf_rb_node *node) __ksym;
|
||||
#endif /* __TC_FLOW_INDEX_EXPERIMENTAL_H */
|
||||
```
|
||||
|
||||
`__contains(flow_entry, by_identity)` 和 traffic tree 对应的声明会告诉验证器，每个 root 关联的 containing type 与 node member。验证器再把这层类型关系和相邻 spin lock 结合起来，检查 graph ownership 与临界区规则。kfunc 接口可能随内核演进，因此本课明确记录最低版本，并把声明放在例子旁边。
|
||||
`__contains(flow_entry, by_identity)` 注解告诉验证器每个 root 对应的 containing type 和 member。结合相邻的 spin lock，验证器可以强制检查 ownership 和临界区规则。这些接口可能随内核版本演进，因此我们在下文注明了最低版本要求。
|
||||
|
||||
## 在 TC egress 维护两份索引
|
||||
## BPF 程序
|
||||
|
||||
下面是完整 BPF 程序。
|
||||
下面是完整的 BPF 实现，之后我们会逐一讲解关键部分。
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: GPL-2.0
|
||||
@@ -399,17 +426,23 @@ int snapshot_next(void *ctx)
|
||||
}
|
||||
```
|
||||
|
||||
`index_lock`、`identity_root` 和 `traffic_root` 位于同一个 private map value，所有搜索与修改都持有这把锁。二叉树搜索最多执行 32 步，让验证器看到确定的控制流；索引最多保存 4096 个 entry，平衡 rbtree 的实际深度远小于这个上限。
|
||||
### 理解 Flow Entry
|
||||
|
||||
第一次加锁查找失败以后，程序会在锁外分配对象，再次加锁并重新查找。这次 double check 用于处理两个 CPU 同时发现新 flow 的情况：胜者把对象交给两棵树，另一方更新已有条目，并释放自己刚分配的两个 reference。allocation、refcount、capacity 和 re-ranking failure 都有独立计数，ownership 问题可以直接从输出中看到。
|
||||
`flow_entry` 结构体是整个设计的核心。它内嵌一个 `bpf_refcount` 用于 ownership 跟踪，以及两个 `bpf_rb_node` 字段用于参与两棵树。lock、`identity_root` 和 `traffic_root` 共享一个 private map value，确保所有访问都是串行化的。
|
||||
|
||||
`parse_flow()` 接收未分片的 IPv4 TCP 与 UDP 报文，从 Ethernet、IP 和 transport header 组装 key。`index_egress_flow()` 始终返回 `TC_ACT_OK`，工具只观察并建立索引，不会改变报文传输。
|
||||
### 报文处理路径
|
||||
|
||||
`snapshot_next` 程序在同一把锁下读取 traffic tree。空 cursor 从 `bpf_rbtree_first()` 开始，之后则搜索前一个 `(bytes, packets, key)` 之后的第一项。result 与 cursor 都放在 BSS，用户态每次取一个有序 entry，不需要接收每报文事件流。
|
||||
当 `index_egress_flow` 运行时，`parse_flow()` 从非分片的 IPv4 TCP 和 UDP 报文中提取五元组。主函数始终返回 `TC_ACT_OK`，因为我们只是观察，不是过滤。
|
||||
|
||||
## 挂载 TC 并读取稳定 snapshot
|
||||
`update_flow()` 函数首先在锁内搜索。如果 flow 存在，它移除 traffic 节点、更新计数器并重新插入。如果不存在，则在锁外分配，然后重新加锁做第二次查找。这个 double-check 处理了两个 CPU 同时发现新 flow 的情况：一个赢得插入竞争，另一个更新已有条目并释放自己未使用的引用。
|
||||
|
||||
用户态程序挂载 classifier，按需生成 demo 流量，在 snapshot 之前解除挂载，再通过 `BPF_PROG_TEST_RUN` 调用 snapshot program。
|
||||
### 快照机制
|
||||
|
||||
`snapshot_next` syscall 程序在同一把锁下读取 traffic 树。空游标时返回 `bpf_rbtree_first()`，后续调用则搜索上一个位置之后的第一个条目。由于游标和结果都在 BSS 中，用户态可以逐个获取排序好的条目，无需接收每报文事件流。
|
||||
|
||||
## 用户态程序
|
||||
|
||||
加载器挂载 TC 程序，可选生成演示流量，读取前先解挂，然后通过 `BPF_PROG_TEST_RUN` 查询：
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
|
||||
@@ -730,9 +763,7 @@ cleanup:
|
||||
}
|
||||
```
|
||||
|
||||
输出前解除挂载会冻结所有 packet update，cursor 因此可以遍历一份稳定的 traffic ordering。`snapshot_next()` 从 BSS 复制结果并推进 cursor，直到 `found` 变成 0，或者已经打印 `--top` 指定的数量。包括两棵树和 object ownership 在内的索引仍保存在 BPF 内存中，直到 skeleton 被销毁。
|
||||
|
||||
demo 模式创建三组 UDP socket pair，在 loopback 上分别发送 2 × 100 字节、4 × 300 字节和 6 × 700 字节 payload。算上 Ethernet、IPv4 与 UDP header，最终观测到 284、1368 和 4452 字节。这组流量会稳定产生三档排名，同时反复执行 traffic node 的 remove、update 和 reinsert。
|
||||
输出前解挂会冻结报文更新，让游标可以遍历一份稳定的 traffic 排序。演示模式创建三条 loopback 上的 UDP 流：2 个报文 100 字节、4 个报文 300 字节、6 个报文 700 字节。加上协议头后，分别是 284、1368 和 4452 字节，产生确定性的排名。
|
||||
|
||||
## 编译和运行
|
||||
|
||||
@@ -743,19 +774,19 @@ cd src/56-tc-flow-index
|
||||
make
|
||||
```
|
||||
|
||||
在指定接口上索引 30 秒 egress 流量，并打印最繁忙的 10 条 flow：
|
||||
监控真实 egress 流量 30 秒：
|
||||
|
||||
```bash
|
||||
sudo ./tc_flow_index --interface eth0 --duration 30 --top 10
|
||||
```
|
||||
|
||||
运行可重复的 loopback demo：
|
||||
运行确定性演示：
|
||||
|
||||
```bash
|
||||
sudo ./tc_flow_index --demo --top 3
|
||||
```
|
||||
|
||||
一次真实 demo 会直接展示双树排序结果：
|
||||
预期演示输出：
|
||||
|
||||
```text
|
||||
Indexing IPv4 TCP/UDP egress flows on lo for 0 seconds.
|
||||
@@ -768,25 +799,30 @@ SOURCE DESTINATION PROTO PACKETS BYTES COMM
|
||||
observed_packets=12 indexed_flows=3 dropped_new=0 allocation_failures=0 refcount_failures=0 rank_update_failures=0
|
||||
```
|
||||
|
||||
表格到达用户态时已经按照 `BYTES` 降序排列，12 个报文创建 3 个 identity entry，所有 ownership 相关失败计数都是 0。
|
||||
表格到达用户态时已按字节数降序排列。12 个报文创建了 3 个 identity 条目，所有 ownership 计数器都是 0，确认双树管理正确无误。
|
||||
|
||||
## 环境要求
|
||||
|
||||
| 要求 | 说明 |
|
||||
|---|---|
|
||||
| 内核 | Linux 6.16 或更高版本，需要 refcounted BPF object 与可搜索的 rbtree kfunc |
|
||||
| 内核配置 | `CONFIG_BPF`、`CONFIG_BPF_SYSCALL`、`CONFIG_BPF_JIT`、`CONFIG_DEBUG_INFO_BTF`、`CONFIG_NET_SCHED`、`CONFIG_NET_CLS_BPF` |
|
||||
| 权限 | root，或者等价的 BPF 与网络管理 capability |
|
||||
| 网络接口 | 可以挂载 `clsact` egress program 的接口 |
|
||||
| 架构与硬件 | 当前声明并完成测试的目标是 x86-64，普通网络接口即可 |
|
||||
|------|------|
|
||||
| 内核 | Linux 6.16+（需要 refcounted BPF object + rbtree search kfunc） |
|
||||
| 配置 | `CONFIG_BPF`、`CONFIG_BPF_SYSCALL`、`CONFIG_BPF_JIT`、`CONFIG_DEBUG_INFO_BTF`、`CONFIG_NET_SCHED`、`CONFIG_NET_CLS_BPF` |
|
||||
| 权限 | root 或 CAP_BPF + CAP_NET_ADMIN |
|
||||
| 网络接口 | 支持 clsact egress 的任意接口 |
|
||||
| 架构 | 已在 x86-64 上测试 |
|
||||
|
||||
## 实现范围
|
||||
## 扩展方向
|
||||
|
||||
这个索引覆盖未分片的 IPv4 TCP 与 UDP egress 流量，并保留 flow 首次创建时观察到的 command name。容量固定为 4096 条 flow，entry 会一直保留到程序退出，适合有边界的实验和短时间观察。长期运行的服务可以继续加入 idle expiry，在释放 reference 之前从两棵树移除对应 node。
|
||||
本示例覆盖非分片的 IPv4 TCP/UDP egress 流量，容量固定为 4096 条。对于生产环境，可以考虑：
|
||||
|
||||
- **空闲过期**：移除 N 秒内没有流量的 flow，先释放两棵树的节点再 drop reference
|
||||
- **入口索引**：为入站报文添加第二个 TC 程序
|
||||
- **IPv6 支持**：扩展 flow key 结构
|
||||
- **实时流式输出**：在树之外添加 ring buffer 用于每报文事件
|
||||
|
||||
## 总结
|
||||
|
||||
这个例子用 BPF object ownership 在同一条 flow 记录上维护两种有用顺序。identity tree 负责每报文更新，traffic tree 提供 top-flow traversal，refcount 则让两棵 collection 共享对象，同时保持状态只有一份。
|
||||
本教程展示了 BPF object ownership 如何实现以前不可能的复杂数据结构。identity 树负责快速的五元组查找，traffic 树提供即时的 top-flow 排名，而引用计数让两者共享同一个对象，无需状态复制。这个技术可以推广到任何需要对同一数据集维护多种排序的场景。
|
||||
|
||||
> 如果你想深入了解 eBPF，请查看我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或访问我们的网站 <https://eunomia.dev/tutorials/>。
|
||||
|
||||
|
||||
@@ -1,26 +1,52 @@
|
||||
# eBPF Tutorial by Example: Profile memcg Reclaim Before an OOM Kill
|
||||
# eBPF Tutorial: Profiling Memory Reclaim Before an OOM Kill
|
||||
|
||||
An OOM log answers the last question in a memory failure: which task did the kernel choose to kill? It says much less about the work that led there. Operators still need to know how often the target memory cgroup entered reclaim, how long those attempts took, which kernel paths consumed the time, and whether the victim actually exited.
|
||||
Have you ever had a container or service suddenly killed by the Linux OOM killer, leaving you with nothing but a cryptic log message? The kernel tells you *which* process it chose as a victim, but says almost nothing about the memory pressure that led there. How many times did the system try to reclaim memory? How long did each attempt take? Which kernel code paths consumed all that time?
|
||||
|
||||
This tutorial builds `oom-watch` to preserve that missing history. It profiles memcg reclaim as a latency distribution and a set of sampled kernel stacks, then attaches the accumulated profile to the selected OOM victim and follows that victim to process exit.
|
||||
This tutorial builds `oom-watch`, an eBPF tool that captures what happens *before* the kill. It profiles every memcg reclaim attempt as a latency histogram and a set of sampled kernel stacks, then attaches this accumulated profile to the OOM victim and tracks the process until it exits.
|
||||
|
||||
> Complete source code: <https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/57-oom-watch>
|
||||
|
||||
## Reclaim Is the Story Before the Kill
|
||||
## The Missing Piece in OOM Debugging
|
||||
|
||||
When a memory cgroup approaches its limit, an allocation may enter reclaim and scan that memcg for pages it can free. Several short attempts can succeed partially, or repeated reclaim can make little progress until the OOM killer selects a victim. Looking only at `oom/mark_victim` loses both the time distribution and the call paths of those attempts.
|
||||
When a memory cgroup approaches its limit, any allocation might trigger reclaim. The kernel scans the cgroup for pages it can free. Sometimes a few short attempts succeed. Other times, reclaim keeps running with little progress until the OOM killer finally steps in.
|
||||
|
||||
eBPF runs verifier-checked programs at kernel events and uses maps to carry state between them. Linux 7.1 added the target `mem_cgroup` to the `mm_vmscan_memcg_reclaim_begin` and `mm_vmscan_memcg_reclaim_end` tracepoints. That argument matters because the current task can reclaim a different cgroup through `memory.reclaim`; attribution now follows the memcg being scanned instead of the task that triggered the work.
|
||||
Looking only at the `oom/mark_victim` tracepoint loses all this context. You see the victim, but not the struggle that came before. Operators need answers to questions like:
|
||||
|
||||
The second dependency is victim lookup. `oom/mark_victim` reports the selected thread ID, while diagnosis also needs its thread-group ID and cgroup. `bpf_task_from_pid()` arrived in Linux 6.2, and regular tracepoint programs gained access to this tracing kfunc class in Linux 6.12. The newer vmscan tracepoint signature therefore sets the complete tool's minimum kernel at Linux 7.1.
|
||||
- How many reclaim cycles happened before the kill?
|
||||
- Were they quick microsecond scans, or long millisecond stalls?
|
||||
- Did reclaim come from allocations inside the cgroup, or from external proactive reclaim via `memory.reclaim`?
|
||||
- Which kernel functions dominated the reclaim time?
|
||||
|
||||
Follow one reclaim interval. At the begin tracepoint, the BPF program records the monotonic start time and target cgroup under the current `pid_tgid`. According to `--sample-every`, it also captures a kernel stack ID. The matching end tracepoint computes duration, increments a power-of-two microsecond bucket, and adds reclaimed pages. A second map groups sampled intervals by `(cgroup_id, stack_id)` and accumulates sample count, total time, maximum time, and reclaimed pages for each path.
|
||||
`oom-watch` answers all of these. It hooks into the kernel's vmscan tracepoints to measure every reclaim interval, samples kernel stacks to show where time was spent, and when OOM selects a victim, it snapshots the accumulated profile right alongside the victim information.
|
||||
|
||||
When OOM selects a victim, the program resolves that TID to its TGID and cgroup, copies the cgroup's reclaim profile into a ring-buffer event, and stores victim state keyed by TID. `sched_process_exit` later consumes that state and reports the exit code. The result connects activity accumulated before the kill with the exact thread the kernel marked and the process lifecycle that followed.
|
||||
## Why eBPF for Memory Profiling?
|
||||
|
||||
## Profile and Event Layouts
|
||||
Traditional monitoring approaches have serious limitations for this use case. Polling `/proc/meminfo` or cgroup stats misses short-lived reclaim events. `perf` can capture stack traces but requires careful setup and post-processing. Neither approach easily connects reclaim activity to a specific OOM event.
|
||||
|
||||
The shared header defines the cgroup-level histogram, per-stack aggregate, and the event that carries an OOM snapshot.
|
||||
eBPF changes the game. Programs run directly in the kernel, triggered by events with nanosecond precision. Maps carry state between events, letting us build histograms and correlate begin/end pairs. The ring buffer delivers events to userspace with minimal overhead. And because eBPF programs are verified before loading, there's no risk of crashing the kernel.
|
||||
|
||||
For `oom-watch`, we use several tracepoints:
|
||||
|
||||
- `mm_vmscan_memcg_reclaim_begin`: Fires when reclaim starts for a specific memory cgroup
|
||||
- `mm_vmscan_memcg_reclaim_end`: Fires when that reclaim interval completes
|
||||
- `oom/mark_victim`: Fires when the OOM killer selects a victim
|
||||
- `sched/sched_process_exit`: Fires when the victim process exits
|
||||
|
||||
Linux 7.1 added a crucial feature: the vmscan tracepoints now include the target `mem_cgroup` being scanned. This matters because a process in one cgroup can trigger reclaim in a different cgroup through `memory.reclaim`. With the target cgroup in the tracepoint, we can correctly attribute the work.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The tool has three main components working together.
|
||||
|
||||
The **header file** defines shared data structures: a per-cgroup reclaim profile containing latency buckets and counters, a per-stack aggregate tracking samples and timing, and an event structure for OOM notifications.
|
||||
|
||||
The **BPF program** runs in the kernel. On reclaim begin, it records the start time and optionally captures a kernel stack. On reclaim end, it computes duration, updates the histogram, and stores the stack aggregate. When OOM marks a victim, it looks up the victim's cgroup, copies the accumulated profile into an event, and sends it to userspace. It also saves victim state so it can report when the process exits.
|
||||
|
||||
The **userspace program** loads kernel symbols for stack symbolization, manages the optional cgroup filter, processes events from the ring buffer, and includes a self-contained demo mode that triggers an OOM to verify everything works.
|
||||
|
||||
## The Profile Data Structures
|
||||
|
||||
The shared header defines what we're measuring:
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -79,11 +105,13 @@ struct oom_watch_event {
|
||||
#endif /* __OOM_WATCH_H */
|
||||
```
|
||||
|
||||
`latency_slots` contains 20 base-2 microsecond ranges. Bucket 0 covers 0–1 µs, bucket 1 covers 2–3 µs, then 4–7 µs and so on; the final bucket contains every interval of at least 524288 µs. The event embeds a complete `reclaim_profile`, so the numbers printed with a victim describe the profile as it stood when that victim was selected.
|
||||
The `latency_slots` array holds a power-of-two histogram with 20 buckets. Bucket 0 covers 0-1 microseconds, bucket 1 covers 2-3 microseconds, then 4-7 microseconds, and so on. The final bucket catches everything over half a second. This logarithmic distribution captures both fast reclaim cycles and rare slow ones efficiently.
|
||||
|
||||
## Profiling Reclaim and Tracking the Victim
|
||||
When an OOM event fires, it embeds a complete copy of the cgroup's profile. The numbers printed with the victim show exactly how things stood at the moment of selection.
|
||||
|
||||
Here is the complete BPF program.
|
||||
## The BPF Program
|
||||
|
||||
Here's the complete BPF code. We'll walk through how it works after the listing:
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: GPL-2.0
|
||||
@@ -398,776 +426,34 @@ int capture_victim_exit(void *ctx)
|
||||
}
|
||||
```
|
||||
|
||||
`active_reclaims` joins begin and end by `pid_tgid`. The begin callback records the target cgroup ID from `memcg->css.cgroup->kn->id`, rather than using the current task's cgroup. Comparing those two IDs produces `cross_cgroup_reclaims`, which makes proactive reclaim from another cgroup visible.
|
||||
The program uses several BPF maps to maintain state. The `active_reclaims` map joins begin and end events by storing the start timestamp keyed by `pid_tgid`. The `profiles` map accumulates per-cgroup statistics. The `stack_traces` map stores deduplicated kernel stacks, while `stack_profiles` aggregates timing data for each unique stack.
|
||||
|
||||
Stack sampling happens at begin, before the reclaim work runs. `sample_every=1` captures every interval; a larger value reduces stack-map and unwinding cost while the latency histogram still counts every matched interval. The value `2` in the low eight flag bits skips two tracing frames, while `bpf_get_stackid()` deduplicates the remaining stack in `stack_traces`. `stack_profiles` attaches timing and reclaimed-page totals to each ID. Atomic updates let reclaim from several CPUs contribute to one cgroup profile.
|
||||
At reclaim begin, we extract the target cgroup ID from the `mem_cgroup` parameter, not from the current task's cgroup. This distinction matters for cross-cgroup reclaim. If someone calls `memory.reclaim` on a cgroup from outside, we correctly attribute the work to the target. The `cross_cgroup_reclaims` counter tracks how often this happens.
|
||||
|
||||
`profile_reclaim_end()` removes the active state after accounting. It updates total and maximum latency, chooses the histogram bucket, and updates the sampled stack aggregate. Separate counters expose active-state insertion failures and stack-capture failures, while a difference between begin and end counts reveals an interval that could not be paired.
|
||||
Stack sampling uses the `--sample-every` setting. With the default value of 1, we capture every interval. Higher values reduce overhead while still counting all intervals in the histogram. The `BPF_F_FAST_STACK_CMP` flag speeds up stack deduplication, and the `2` in the flags skips two tracing frames to get cleaner stacks.
|
||||
|
||||
The OOM path handles process identity carefully. The tracepoint's `ctx->pid` is retained as `victim_tid`; `bpf_task_from_pid()` supplies the task's TGID and cgroup ID, and `bpf_task_release()` releases the referenced task pointer. Victim state stays keyed by TID because `sched_process_exit` runs in the context of that exact thread. This also covers a multithreaded process whose leader and selected victim have different IDs.
|
||||
The OOM handler has to resolve process identity carefully. The tracepoint gives us a thread ID, but we also need the thread group ID (PID from userspace perspective) and the cgroup. We use `bpf_task_from_pid()` to look up the task, read what we need, then release the reference. Victim state is keyed by TID because `sched_process_exit` fires in that thread's context.
|
||||
|
||||
## Symbolizing and Presenting the Profile
|
||||
## Userspace: Symbols and Presentation
|
||||
|
||||
The user-space program loads kernel symbols, ranks stack aggregates, manages the optional cgroup filter, and provides a self-contained OOM demonstration.
|
||||
The userspace code handles several jobs: loading kernel symbols for stack symbolization, setting up the BPF program with any cgroup filter, processing events from the ring buffer, and ranking stacks by total time spent. It also includes a demo mode that creates a memory-limited cgroup and triggers an OOM to verify the tool works.
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <getopt.h>
|
||||
#include <poll.h>
|
||||
#include <pthread.h>
|
||||
#include <signal.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/mman.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/wait.h>
|
||||
#include <time.h>
|
||||
#include <unistd.h>
|
||||
#include <bpf/bpf.h>
|
||||
#include <bpf/libbpf.h>
|
||||
#include "oom_watch.h"
|
||||
#include "oom_watch.skel.h"
|
||||
The full userspace code is quite long, so we'll highlight the key parts. On startup, it reads `/proc/kallsyms`, sorts symbols by address, and later uses binary search to resolve each stack frame. When symbol addresses are restricted (common on production systems), it still prints raw addresses - the measurements stay accurate.
|
||||
|
||||
struct options {
|
||||
const char *cgroup_path;
|
||||
unsigned int duration_seconds;
|
||||
unsigned int sample_every;
|
||||
bool demo;
|
||||
};
|
||||
Stack groups are ranked by cumulative reclaim time. This ranking surfaces both frequently-called paths and rare slow intervals. After an OOM event, the top five stacks are printed with their timing statistics.
|
||||
|
||||
static volatile sig_atomic_t stop;
|
||||
static int victim_events;
|
||||
static int exit_events;
|
||||
static unsigned long long observed_cgroup_id;
|
||||
static unsigned int observed_victim_pid;
|
||||
static unsigned int observed_victim_tid;
|
||||
static unsigned long long observed_reclaims;
|
||||
static unsigned long long observed_cross_cgroup_reclaims;
|
||||
static unsigned long long observed_stack_samples;
|
||||
The demo mode creates a cgroup with `memory.max=32 MiB`, no swap, and grouped OOM behavior. A worker process faults 24 MiB then pauses, letting the parent request 8 MiB through `memory.reclaim` from outside the cgroup. This exercises cross-cgroup attribution. The worker then continues faulting a 128 MiB mapping until OOM kills it. The process leader exits before the second stage, testing that we handle TGID and victim TID independently.
|
||||
|
||||
struct kernel_symbol {
|
||||
unsigned long long address;
|
||||
char *name;
|
||||
};
|
||||
## Compilation and Execution
|
||||
|
||||
struct kernel_symbols {
|
||||
struct kernel_symbol *items;
|
||||
size_t count;
|
||||
size_t capacity;
|
||||
};
|
||||
|
||||
struct runtime_context {
|
||||
int profiles_fd;
|
||||
int stack_profiles_fd;
|
||||
int stack_traces_fd;
|
||||
struct kernel_symbols symbols;
|
||||
};
|
||||
|
||||
struct oom_runtime {
|
||||
struct oom_watch_bpf *skel;
|
||||
struct ring_buffer *ring;
|
||||
struct runtime_context context;
|
||||
};
|
||||
|
||||
struct selected_cgroup {
|
||||
char demo_path[256];
|
||||
const char *path;
|
||||
struct stat metadata;
|
||||
bool demo_created;
|
||||
bool memory_enabled_by_demo;
|
||||
};
|
||||
|
||||
struct demo_process {
|
||||
pid_t child;
|
||||
int ready_pipe[2];
|
||||
int continue_pipe[2];
|
||||
int status;
|
||||
};
|
||||
|
||||
struct ranked_stack {
|
||||
struct reclaim_stack_key key;
|
||||
struct reclaim_stack_profile profile;
|
||||
};
|
||||
|
||||
struct allocation_context {
|
||||
int ready_fd;
|
||||
int continue_fd;
|
||||
};
|
||||
|
||||
static struct allocation_context allocation_context;
|
||||
|
||||
static void handle_signal(int signal_number)
|
||||
{
|
||||
(void)signal_number;
|
||||
stop = 1;
|
||||
}
|
||||
|
||||
static unsigned long long monotonic_ns(void)
|
||||
{
|
||||
struct timespec now;
|
||||
|
||||
clock_gettime(CLOCK_MONOTONIC, &now);
|
||||
return (unsigned long long)now.tv_sec * 1000000000ULL + now.tv_nsec;
|
||||
}
|
||||
|
||||
static int write_text(const char *path, const char *text)
|
||||
{
|
||||
int fd = open(path, O_WRONLY | O_CLOEXEC);
|
||||
ssize_t length = strlen(text);
|
||||
int err = 0;
|
||||
|
||||
if (fd < 0)
|
||||
return -1;
|
||||
if (write(fd, text, length) != length)
|
||||
err = -1;
|
||||
close(fd);
|
||||
return err;
|
||||
}
|
||||
|
||||
static int memory_controller_enabled(bool *enabled)
|
||||
{
|
||||
char controllers[4096];
|
||||
ssize_t length;
|
||||
int fd;
|
||||
|
||||
fd = open("/sys/fs/cgroup/cgroup.subtree_control",
|
||||
O_RDONLY | O_CLOEXEC);
|
||||
if (fd < 0)
|
||||
return -1;
|
||||
length = read(fd, controllers, sizeof(controllers) - 1);
|
||||
close(fd);
|
||||
if (length < 0)
|
||||
return -1;
|
||||
controllers[length] = '\0';
|
||||
*enabled = strstr(controllers, "memory") != NULL;
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int compare_symbols(const void *left, const void *right)
|
||||
{
|
||||
const struct kernel_symbol *a = left;
|
||||
const struct kernel_symbol *b = right;
|
||||
|
||||
return a->address < b->address ? -1 : a->address > b->address ? 1 : 0;
|
||||
}
|
||||
|
||||
static int load_kernel_symbols(struct kernel_symbols *symbols)
|
||||
{
|
||||
char name[256];
|
||||
char type;
|
||||
unsigned long long address;
|
||||
FILE *file = fopen("/proc/kallsyms", "r");
|
||||
|
||||
if (!file)
|
||||
return -1;
|
||||
while (fscanf(file, "%llx %c %255s%*[^\n]\n", &address, &type,
|
||||
name) == 3) {
|
||||
struct kernel_symbol *item;
|
||||
|
||||
(void)type;
|
||||
if (symbols->count == symbols->capacity) {
|
||||
size_t capacity = symbols->capacity ? symbols->capacity * 2 : 4096;
|
||||
void *items = realloc(symbols->items,
|
||||
capacity * sizeof(*symbols->items));
|
||||
|
||||
if (!items)
|
||||
goto error;
|
||||
symbols->items = items;
|
||||
symbols->capacity = capacity;
|
||||
}
|
||||
item = &symbols->items[symbols->count++];
|
||||
item->address = address;
|
||||
item->name = strdup(name);
|
||||
if (!item->name)
|
||||
goto error;
|
||||
}
|
||||
fclose(file);
|
||||
qsort(symbols->items, symbols->count, sizeof(*symbols->items),
|
||||
compare_symbols);
|
||||
return symbols->count ? 0 : -1;
|
||||
|
||||
error:
|
||||
fclose(file);
|
||||
return -1;
|
||||
}
|
||||
|
||||
static void free_kernel_symbols(struct kernel_symbols *symbols)
|
||||
{
|
||||
for (size_t i = 0; i < symbols->count; i++)
|
||||
free(symbols->items[i].name);
|
||||
free(symbols->items);
|
||||
}
|
||||
|
||||
static const struct kernel_symbol *find_kernel_symbol(
|
||||
const struct kernel_symbols *symbols, unsigned long long address)
|
||||
{
|
||||
size_t low = 0, high = symbols->count;
|
||||
|
||||
while (low < high) {
|
||||
size_t middle = low + (high - low) / 2;
|
||||
|
||||
if (symbols->items[middle].address <= address)
|
||||
low = middle + 1;
|
||||
else
|
||||
high = middle;
|
||||
}
|
||||
return low ? &symbols->items[low - 1] : NULL;
|
||||
}
|
||||
|
||||
static void insert_ranked_stack(struct ranked_stack top[5], size_t *count,
|
||||
const struct reclaim_stack_key *key,
|
||||
const struct reclaim_stack_profile *profile)
|
||||
{
|
||||
size_t position = 0;
|
||||
|
||||
while (position < *count &&
|
||||
top[position].profile.total_ns >= profile->total_ns)
|
||||
position++;
|
||||
if (position >= 5)
|
||||
return;
|
||||
if (*count < 5)
|
||||
(*count)++;
|
||||
for (size_t i = *count - 1; i > position; i--)
|
||||
top[i] = top[i - 1];
|
||||
top[position].key = *key;
|
||||
top[position].profile = *profile;
|
||||
}
|
||||
|
||||
static void print_reclaim_stacks(struct runtime_context *runtime,
|
||||
__u64 cgroup_id)
|
||||
{
|
||||
struct ranked_stack top[5] = {};
|
||||
struct reclaim_stack_key previous, next;
|
||||
bool have_previous = false;
|
||||
size_t count = 0;
|
||||
|
||||
while (!bpf_map_get_next_key(runtime->stack_profiles_fd,
|
||||
have_previous ? &previous : NULL, &next)) {
|
||||
struct reclaim_stack_profile profile;
|
||||
|
||||
if (next.cgroup_id == cgroup_id &&
|
||||
!bpf_map_lookup_elem(runtime->stack_profiles_fd, &next,
|
||||
&profile))
|
||||
insert_ranked_stack(top, &count, &next, &profile);
|
||||
previous = next;
|
||||
have_previous = true;
|
||||
}
|
||||
|
||||
for (size_t rank = 0; rank < count; rank++) {
|
||||
unsigned long long addresses[OOM_STACK_DEPTH] = {};
|
||||
|
||||
printf("reclaim_stack rank=%zu samples=%llu total_ms=%.3f "
|
||||
"max_ms=%.3f reclaimed_pages=%llu\n",
|
||||
rank + 1, top[rank].profile.samples,
|
||||
top[rank].profile.total_ns / 1000000.0,
|
||||
top[rank].profile.maximum_ns / 1000000.0,
|
||||
top[rank].profile.reclaimed_pages);
|
||||
if (bpf_map_lookup_elem(runtime->stack_traces_fd,
|
||||
&top[rank].key.stack_id, addresses))
|
||||
continue;
|
||||
for (size_t frame = 0; frame < OOM_STACK_DEPTH && addresses[frame];
|
||||
frame++) {
|
||||
const struct kernel_symbol *symbol =
|
||||
find_kernel_symbol(&runtime->symbols, addresses[frame]);
|
||||
|
||||
if (symbol && symbol->address)
|
||||
printf(" #%zu %s+0x%llx\n", frame, symbol->name,
|
||||
addresses[frame] - symbol->address);
|
||||
else
|
||||
printf(" #%zu 0x%llx\n", frame, addresses[frame]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static void print_reclaim_profile(struct runtime_context *runtime,
|
||||
__u64 cgroup_id,
|
||||
const struct reclaim_profile *profile)
|
||||
{
|
||||
printf("reclaim_profile cgroup_id=%llu cycles=%llu completed=%llu "
|
||||
"total_ms=%.3f max_ms=%.3f reclaimed_pages=%llu "
|
||||
"cross_cgroup=%llu stack_samples=%llu stack_failures=%llu\n",
|
||||
(unsigned long long)cgroup_id, profile->begin_count,
|
||||
profile->end_count, profile->total_reclaim_ns / 1000000.0,
|
||||
profile->maximum_reclaim_ns / 1000000.0,
|
||||
profile->reclaimed_pages, profile->cross_cgroup_reclaims,
|
||||
profile->stack_samples, profile->stack_failures);
|
||||
for (unsigned int bucket = 0; bucket < OOM_RECLAIM_BUCKETS; bucket++) {
|
||||
unsigned long long low, high;
|
||||
|
||||
if (!profile->latency_slots[bucket])
|
||||
continue;
|
||||
low = bucket ? 1ULL << bucket : 0;
|
||||
high = (1ULL << (bucket + 1)) - 1;
|
||||
if (bucket == OOM_RECLAIM_BUCKETS - 1)
|
||||
printf("reclaim_latency_us=>=%llu count=%llu\n", low,
|
||||
profile->latency_slots[bucket]);
|
||||
else
|
||||
printf("reclaim_latency_us=%llu-%llu count=%llu\n", low,
|
||||
high, profile->latency_slots[bucket]);
|
||||
}
|
||||
print_reclaim_stacks(runtime, cgroup_id);
|
||||
}
|
||||
|
||||
static void print_live_profiles(struct runtime_context *runtime)
|
||||
{
|
||||
__u64 previous, next;
|
||||
bool have_previous = false;
|
||||
|
||||
while (!bpf_map_get_next_key(runtime->profiles_fd,
|
||||
have_previous ? &previous : NULL, &next)) {
|
||||
struct reclaim_profile profile;
|
||||
|
||||
if (!bpf_map_lookup_elem(runtime->profiles_fd, &next, &profile))
|
||||
print_reclaim_profile(runtime, next, &profile);
|
||||
previous = next;
|
||||
have_previous = true;
|
||||
}
|
||||
}
|
||||
|
||||
static int handle_event(void *ctx, void *data, size_t size)
|
||||
{
|
||||
const struct oom_watch_event *event = data;
|
||||
struct runtime_context *runtime = ctx;
|
||||
|
||||
if (size != sizeof(*event))
|
||||
return 0;
|
||||
if (event->type == OOM_VICTIM_MARKED) {
|
||||
victim_events++;
|
||||
observed_cgroup_id = event->cgroup_id;
|
||||
observed_victim_pid = event->victim_pid;
|
||||
observed_reclaims = event->profile.begin_count;
|
||||
observed_victim_tid = event->victim_tid;
|
||||
observed_cross_cgroup_reclaims =
|
||||
event->profile.cross_cgroup_reclaims;
|
||||
observed_stack_samples = event->profile.stack_samples;
|
||||
printf("event=oom-victim pid=%u tid=%u comm=%s trigger_pid=%u cgroup_id=%llu "
|
||||
"anon_rss_kb=%llu file_rss_kb=%llu total_vm_kb=%llu "
|
||||
"reclaim_cycles=%llu cross_cgroup_reclaims=%llu "
|
||||
"reclaimed_pages=%llu\n",
|
||||
event->victim_pid, event->victim_tid, event->comm,
|
||||
event->triggering_tgid,
|
||||
(unsigned long long)event->cgroup_id,
|
||||
(unsigned long long)event->anon_rss_kb,
|
||||
(unsigned long long)event->file_rss_kb,
|
||||
(unsigned long long)event->total_vm_kb,
|
||||
(unsigned long long)event->profile.begin_count,
|
||||
(unsigned long long)event->profile.cross_cgroup_reclaims,
|
||||
(unsigned long long)event->profile.reclaimed_pages);
|
||||
print_reclaim_profile(runtime, event->cgroup_id, &event->profile);
|
||||
} else if (event->type == OOM_VICTIM_EXITED) {
|
||||
exit_events++;
|
||||
printf("event=victim-exit pid=%u tid=%u cgroup_id=%llu exit_code=%d\n",
|
||||
event->victim_pid, event->victim_tid,
|
||||
(unsigned long long)event->cgroup_id, event->exit_code);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int parse_uint(const char *text, unsigned int maximum,
|
||||
unsigned int *value)
|
||||
{
|
||||
char *end = NULL;
|
||||
unsigned long parsed;
|
||||
|
||||
errno = 0;
|
||||
parsed = strtoul(text, &end, 10);
|
||||
if (errno || !*text || *end || !parsed || parsed > maximum)
|
||||
return -1;
|
||||
*value = parsed;
|
||||
return 0;
|
||||
}
|
||||
|
||||
static void usage(const char *program)
|
||||
{
|
||||
printf("Usage: %s [--cgroup PATH] [--duration SEC] [--sample-every N]\n"
|
||||
" %s --demo [--sample-every N]\n", program, program);
|
||||
}
|
||||
|
||||
static int parse_options(int argc, char **argv, struct options *options)
|
||||
{
|
||||
static const struct option long_options[] = {
|
||||
{ "cgroup", required_argument, NULL, 'c' },
|
||||
{ "duration", required_argument, NULL, 'd' },
|
||||
{ "sample-every", required_argument, NULL, 's' },
|
||||
{ "demo", no_argument, NULL, 'D' },
|
||||
{ "help", no_argument, NULL, 'h' },
|
||||
{},
|
||||
};
|
||||
int option;
|
||||
|
||||
while ((option = getopt_long(argc, argv, "c:d:s:Dh", long_options,
|
||||
NULL)) != -1) {
|
||||
switch (option) {
|
||||
case 'c': options->cgroup_path = optarg; break;
|
||||
case 'd':
|
||||
if (parse_uint(optarg, 86400, &options->duration_seconds))
|
||||
return -1;
|
||||
break;
|
||||
case 's':
|
||||
if (parse_uint(optarg, 1000000, &options->sample_every))
|
||||
return -1;
|
||||
break;
|
||||
case 'D': options->demo = true; break;
|
||||
case 'h': usage(argv[0]); exit(0);
|
||||
default: return -1;
|
||||
}
|
||||
}
|
||||
return optind == argc && !(options->demo && options->cgroup_path) ? 0 : -1;
|
||||
}
|
||||
|
||||
static void *allocation_worker(void *argument)
|
||||
{
|
||||
struct allocation_context *context = argument;
|
||||
size_t first_stage = 24 * 1024 * 1024;
|
||||
size_t length = 128 * 1024 * 1024;
|
||||
unsigned char *memory;
|
||||
char byte = 'x';
|
||||
|
||||
memory = mmap(NULL, length, PROT_READ | PROT_WRITE,
|
||||
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
|
||||
if (memory == MAP_FAILED)
|
||||
_exit(4);
|
||||
for (size_t offset = 0; offset < first_stage; offset += 4096)
|
||||
memory[offset] = 0xa5;
|
||||
if (write(context->ready_fd, &byte, 1) != 1 ||
|
||||
read(context->continue_fd, &byte, 1) != 1)
|
||||
_exit(5);
|
||||
for (size_t offset = first_stage; offset < length; offset += 4096)
|
||||
memory[offset] = 0xa5;
|
||||
_exit(6);
|
||||
}
|
||||
|
||||
static void allocate_until_killed(const char *cgroup_path, int ready_fd,
|
||||
int continue_fd)
|
||||
{
|
||||
char procs_path[512];
|
||||
char pid_text[32];
|
||||
pthread_t worker;
|
||||
|
||||
snprintf(procs_path, sizeof(procs_path), "%s/cgroup.procs", cgroup_path);
|
||||
snprintf(pid_text, sizeof(pid_text), "%d", getpid());
|
||||
if (write_text(procs_path, pid_text))
|
||||
_exit(3);
|
||||
allocation_context.ready_fd = ready_fd;
|
||||
allocation_context.continue_fd = continue_fd;
|
||||
if (pthread_create(&worker, NULL, allocation_worker,
|
||||
&allocation_context))
|
||||
_exit(4);
|
||||
pthread_detach(worker);
|
||||
pthread_exit(NULL);
|
||||
}
|
||||
|
||||
static int configure_demo_cgroup(const char *path, bool *created,
|
||||
bool *enabled_by_demo)
|
||||
{
|
||||
bool memory_enabled;
|
||||
char file[512];
|
||||
|
||||
if (memory_controller_enabled(&memory_enabled))
|
||||
return -1;
|
||||
if (!memory_enabled) {
|
||||
if (write_text("/sys/fs/cgroup/cgroup.subtree_control", "+memory"))
|
||||
return -1;
|
||||
*enabled_by_demo = true;
|
||||
}
|
||||
if (mkdir(path, 0755))
|
||||
return -1;
|
||||
*created = true;
|
||||
snprintf(file, sizeof(file), "%s/memory.max", path);
|
||||
if (write_text(file, "33554432"))
|
||||
return -1;
|
||||
snprintf(file, sizeof(file), "%s/memory.swap.max", path);
|
||||
if (write_text(file, "0"))
|
||||
return -1;
|
||||
snprintf(file, sizeof(file), "%s/memory.oom.group", path);
|
||||
return write_text(file, "1");
|
||||
}
|
||||
|
||||
static int trigger_cross_cgroup_reclaim(const char *cgroup_path)
|
||||
{
|
||||
char reclaim_path[512];
|
||||
|
||||
snprintf(reclaim_path, sizeof(reclaim_path), "%s/memory.reclaim",
|
||||
cgroup_path);
|
||||
if (!write_text(reclaim_path, "8388608") || errno == EAGAIN)
|
||||
return 0;
|
||||
return -1;
|
||||
}
|
||||
|
||||
static int select_cgroup(const struct options *options,
|
||||
struct selected_cgroup *selected)
|
||||
{
|
||||
if (options->demo) {
|
||||
snprintf(selected->demo_path, sizeof(selected->demo_path),
|
||||
"/sys/fs/cgroup/ebpf-oom-watch-%d", getpid());
|
||||
if (configure_demo_cgroup(selected->demo_path,
|
||||
&selected->demo_created,
|
||||
&selected->memory_enabled_by_demo)) {
|
||||
fprintf(stderr, "failed to configure demo memory cgroup: %s\n",
|
||||
strerror(errno));
|
||||
return -1;
|
||||
}
|
||||
selected->path = selected->demo_path;
|
||||
} else {
|
||||
selected->path = options->cgroup_path;
|
||||
}
|
||||
if (!selected->path)
|
||||
return 0;
|
||||
if (!stat(selected->path, &selected->metadata))
|
||||
return 0;
|
||||
fprintf(stderr, "failed to stat cgroup %s: %s\n", selected->path,
|
||||
strerror(errno));
|
||||
return -1;
|
||||
}
|
||||
|
||||
static void cleanup_selected_cgroup(struct selected_cgroup *selected,
|
||||
int *result)
|
||||
{
|
||||
if (selected->demo_created && rmdir(selected->demo_path) && !*result)
|
||||
*result = 1;
|
||||
if (selected->memory_enabled_by_demo &&
|
||||
write_text("/sys/fs/cgroup/cgroup.subtree_control", "-memory") &&
|
||||
!*result)
|
||||
*result = 1;
|
||||
}
|
||||
|
||||
static int prepare_runtime(struct oom_runtime *runtime,
|
||||
const struct options *options,
|
||||
const struct selected_cgroup *selected)
|
||||
{
|
||||
runtime->skel = oom_watch_bpf__open();
|
||||
if (!runtime->skel)
|
||||
return -1;
|
||||
runtime->skel->rodata->target_cgroup_id =
|
||||
selected->path ? selected->metadata.st_ino : 0;
|
||||
runtime->skel->rodata->sample_every = options->sample_every;
|
||||
if (oom_watch_bpf__load(runtime->skel) ||
|
||||
oom_watch_bpf__attach(runtime->skel)) {
|
||||
fprintf(stderr, "failed to load and attach OOM watcher\n");
|
||||
return -1;
|
||||
}
|
||||
runtime->context.profiles_fd =
|
||||
bpf_map__fd(runtime->skel->maps.profiles);
|
||||
runtime->context.stack_profiles_fd =
|
||||
bpf_map__fd(runtime->skel->maps.stack_profiles);
|
||||
runtime->context.stack_traces_fd =
|
||||
bpf_map__fd(runtime->skel->maps.stack_traces);
|
||||
if (load_kernel_symbols(&runtime->context.symbols))
|
||||
fprintf(stderr, "warning: kernel symbols unavailable; printing raw stack addresses\n");
|
||||
runtime->ring = ring_buffer__new(
|
||||
bpf_map__fd(runtime->skel->maps.events), handle_event,
|
||||
&runtime->context, NULL);
|
||||
return runtime->ring ? 0 : -1;
|
||||
}
|
||||
|
||||
static void destroy_runtime(struct oom_runtime *runtime)
|
||||
{
|
||||
ring_buffer__free(runtime->ring);
|
||||
free_kernel_symbols(&runtime->context.symbols);
|
||||
oom_watch_bpf__destroy(runtime->skel);
|
||||
}
|
||||
|
||||
static void init_demo_process(struct demo_process *demo)
|
||||
{
|
||||
memset(demo, 0, sizeof(*demo));
|
||||
demo->child = -1;
|
||||
demo->ready_pipe[0] = -1;
|
||||
demo->ready_pipe[1] = -1;
|
||||
demo->continue_pipe[0] = -1;
|
||||
demo->continue_pipe[1] = -1;
|
||||
}
|
||||
|
||||
static void close_demo_pipe(int *fd)
|
||||
{
|
||||
if (*fd >= 0)
|
||||
close(*fd);
|
||||
*fd = -1;
|
||||
}
|
||||
|
||||
static void cleanup_demo_process(struct demo_process *demo)
|
||||
{
|
||||
if (demo->child > 0) {
|
||||
kill(demo->child, SIGKILL);
|
||||
waitpid(demo->child, NULL, 0);
|
||||
}
|
||||
close_demo_pipe(&demo->ready_pipe[0]);
|
||||
close_demo_pipe(&demo->ready_pipe[1]);
|
||||
close_demo_pipe(&demo->continue_pipe[0]);
|
||||
close_demo_pipe(&demo->continue_pipe[1]);
|
||||
}
|
||||
|
||||
static int start_demo_process(struct demo_process *demo,
|
||||
const char *cgroup_path)
|
||||
{
|
||||
struct pollfd ready = { .events = POLLIN };
|
||||
struct timespec leader_exit_delay = { .tv_nsec = 100000000 };
|
||||
char byte = 'x';
|
||||
|
||||
if (pipe(demo->ready_pipe) || pipe(demo->continue_pipe))
|
||||
return -1;
|
||||
demo->child = fork();
|
||||
if (demo->child < 0)
|
||||
return -1;
|
||||
if (!demo->child) {
|
||||
close(demo->ready_pipe[0]);
|
||||
close(demo->continue_pipe[1]);
|
||||
allocate_until_killed(cgroup_path, demo->ready_pipe[1],
|
||||
demo->continue_pipe[0]);
|
||||
}
|
||||
close_demo_pipe(&demo->ready_pipe[1]);
|
||||
close_demo_pipe(&demo->continue_pipe[0]);
|
||||
ready.fd = demo->ready_pipe[0];
|
||||
if (poll(&ready, 1, 5000) != 1 ||
|
||||
read(demo->ready_pipe[0], &byte, 1) != 1 ||
|
||||
trigger_cross_cgroup_reclaim(cgroup_path))
|
||||
return -1;
|
||||
nanosleep(&leader_exit_delay, NULL);
|
||||
if (write(demo->continue_pipe[1], &byte, 1) != 1)
|
||||
return -1;
|
||||
close_demo_pipe(&demo->ready_pipe[0]);
|
||||
close_demo_pipe(&demo->continue_pipe[1]);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int collect_demo_events(struct demo_process *demo,
|
||||
struct ring_buffer *ring)
|
||||
{
|
||||
for (int i = 0; i < 200; i++) {
|
||||
pid_t waited;
|
||||
|
||||
ring_buffer__poll(ring, 50);
|
||||
waited = waitpid(demo->child, &demo->status, WNOHANG);
|
||||
if (waited == demo->child) {
|
||||
demo->child = -1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
for (int i = 0; i < 10 && exit_events < 1; i++)
|
||||
ring_buffer__poll(ring, 50);
|
||||
return demo->child < 0 ? 0 : -1;
|
||||
}
|
||||
|
||||
static bool valid_demo_observation(const struct demo_process *demo,
|
||||
unsigned long long cgroup_id,
|
||||
const struct oom_watch_bpf *skel)
|
||||
{
|
||||
return WIFSIGNALED(demo->status) &&
|
||||
WTERMSIG(demo->status) == SIGKILL && victim_events == 1 &&
|
||||
exit_events == 1 && observed_cgroup_id == cgroup_id &&
|
||||
observed_victim_pid && observed_victim_tid &&
|
||||
observed_victim_pid != observed_victim_tid && observed_reclaims &&
|
||||
observed_cross_cgroup_reclaims && observed_stack_samples &&
|
||||
!skel->bss->dropped_victim_states &&
|
||||
!skel->bss->dropped_reclaim_states;
|
||||
}
|
||||
|
||||
static int run_demo(struct oom_runtime *runtime,
|
||||
const struct selected_cgroup *selected)
|
||||
{
|
||||
struct demo_process demo;
|
||||
int result = -1;
|
||||
|
||||
init_demo_process(&demo);
|
||||
if (start_demo_process(&demo, selected->path) ||
|
||||
collect_demo_events(&demo, runtime->ring))
|
||||
goto cleanup;
|
||||
printf("demo workload signaled=%d signal=%d\n",
|
||||
WIFSIGNALED(demo.status),
|
||||
WIFSIGNALED(demo.status) ? WTERMSIG(demo.status) : 0);
|
||||
if (!valid_demo_observation(&demo, selected->metadata.st_ino,
|
||||
runtime->skel))
|
||||
goto cleanup;
|
||||
printf("demo result=matched-profile-to-victim\n");
|
||||
result = 0;
|
||||
|
||||
cleanup:
|
||||
cleanup_demo_process(&demo);
|
||||
return result;
|
||||
}
|
||||
|
||||
static int watch_profiles(struct oom_runtime *runtime,
|
||||
unsigned int duration_seconds)
|
||||
{
|
||||
unsigned long long deadline = 0;
|
||||
|
||||
signal(SIGINT, handle_signal);
|
||||
signal(SIGTERM, handle_signal);
|
||||
if (duration_seconds)
|
||||
deadline = monotonic_ns() +
|
||||
(unsigned long long)duration_seconds * 1000000000ULL;
|
||||
while (!stop && (!deadline || monotonic_ns() < deadline)) {
|
||||
int result = ring_buffer__poll(runtime->ring, 100);
|
||||
|
||||
if (result < 0 && result != -EINTR) {
|
||||
fprintf(stderr, "ring buffer poll failed: %d\n", result);
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
print_live_profiles(&runtime->context);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
struct options options = { .sample_every = 1 };
|
||||
struct selected_cgroup selected = {};
|
||||
struct oom_runtime runtime = {};
|
||||
int err = 1;
|
||||
|
||||
setvbuf(stdout, NULL, _IONBF, 0);
|
||||
if (parse_options(argc, argv, &options)) {
|
||||
usage(argv[0]);
|
||||
return 2;
|
||||
}
|
||||
if (select_cgroup(&options, &selected) ||
|
||||
prepare_runtime(&runtime, &options, &selected))
|
||||
goto cleanup;
|
||||
|
||||
if (selected.path)
|
||||
printf("oom-watch tracing cgroup=%s cgroup_id=%llu\n",
|
||||
selected.path,
|
||||
(unsigned long long)selected.metadata.st_ino);
|
||||
else
|
||||
printf("oom-watch tracing all cgroups\n");
|
||||
|
||||
if ((options.demo && run_demo(&runtime, &selected)) ||
|
||||
(!options.demo && watch_profiles(&runtime,
|
||||
options.duration_seconds)))
|
||||
goto cleanup;
|
||||
printf("dropped_victim_states=%llu dropped_reclaim_states=%llu\n",
|
||||
(unsigned long long)runtime.skel->bss->dropped_victim_states,
|
||||
(unsigned long long)runtime.skel->bss->dropped_reclaim_states);
|
||||
err = 0;
|
||||
|
||||
cleanup:
|
||||
destroy_runtime(&runtime);
|
||||
cleanup_selected_cgroup(&selected, &err);
|
||||
return err;
|
||||
}
|
||||
```
|
||||
|
||||
At startup, the loader reads `/proc/kallsyms`, sorts symbols by address, and uses binary search to resolve each address from the BPF stack-trace map. If symbol addresses are hidden, the same profile is still printed with raw addresses. Stack groups are ranked by cumulative reclaim time, which highlights a path that is called often as well as one unusually slow interval.
|
||||
|
||||
Normal mode can trace one cgroup or all cgroups. When tracing stops, it walks the profile map and prints each live histogram even if no OOM happened. An OOM event prints the same profile immediately, followed by the top five stack groups and their frames.
|
||||
|
||||
Demo mode creates a cgroup with `memory.max=32 MiB`, no swap, and grouped OOM behavior. A worker first faults 24 MiB, pauses, and lets the parent request 8 MiB through `memory.reclaim`; that deliberate cross-cgroup request exercises target-memcg attribution. The worker then continues faulting a 128 MiB mapping until the cgroup OOM killer selects it. The process leader exits before that second stage, so the test also proves that TGID and victim TID are handled separately. Cleanup restores the memory controller state it found at startup.
|
||||
|
||||
## Build and Run
|
||||
|
||||
Build the profiler:
|
||||
Build the tool:
|
||||
|
||||
```bash
|
||||
cd src/57-oom-watch
|
||||
make
|
||||
```
|
||||
|
||||
Profile one service cgroup for 60 seconds and sample one kernel stack for every ten reclaim intervals:
|
||||
Profile a specific cgroup for 60 seconds, sampling one kernel stack for every ten reclaim intervals:
|
||||
|
||||
```bash
|
||||
sudo ./oom_watch \
|
||||
@@ -1182,7 +468,7 @@ Omit `--cgroup` to watch all cgroups. Omit `--duration` to run until interrupted
|
||||
sudo ./oom_watch --demo
|
||||
```
|
||||
|
||||
The variable PIDs, cgroup ID, addresses, and timings change between runs. The following excerpt comes from a real run; it keeps the complete profile totals and shortens the stack frames for readability:
|
||||
Here's output from a real demo run. PIDs, cgroup IDs, addresses, and timings vary between runs:
|
||||
|
||||
```text
|
||||
oom-watch tracing cgroup=/sys/fs/cgroup/ebpf-oom-watch-1262 cgroup_id=151
|
||||
@@ -1206,25 +492,27 @@ demo result=matched-profile-to-victim
|
||||
dropped_victim_states=0 dropped_reclaim_states=0
|
||||
```
|
||||
|
||||
The 44 completed intervals equal the 44 begin events, and all histogram counts sum to 44. Half were triggered from outside the target cgroup. The two ranked paths separate allocation charge reclaim from the explicit `memory.reclaim` request, while the exit event confirms that the marked TID ended with `SIGKILL`.
|
||||
Look at what this tells us: 44 reclaim cycles completed before the kill, with half triggered from outside the target cgroup. The histogram shows most cycles were fast (under 8 microseconds), but a few took longer. The two ranked stacks separate allocation-triggered reclaim from explicit `memory.reclaim` requests. The exit event confirms the victim received `SIGKILL`.
|
||||
|
||||
## Requirements
|
||||
|
||||
| Requirement | Details |
|
||||
|---|---|
|
||||
| Kernel | Linux 7.1 or newer for target-memcg vmscan tracepoints |
|
||||
| Kernel config | `CONFIG_BPF`, `CONFIG_BPF_SYSCALL`, `CONFIG_BPF_JIT`, `CONFIG_BPF_EVENTS`, `CONFIG_DEBUG_INFO_BTF`, `CONFIG_MEMCG`; `CONFIG_KALLSYMS` improves symbols |
|
||||
| cgroup | cgroup v2 with the memory controller; demo mode also needs writable cgroup administration |
|
||||
| Privileges | Root, or equivalent BPF, tracing, and cgroup-management capabilities |
|
||||
| Architecture and hardware | x86-64 is the declared and tested target; no special hardware |
|
||||
| Kernel | Linux 7.1 or newer (needs target-memcg vmscan tracepoints) |
|
||||
| Kernel config | `CONFIG_BPF`, `CONFIG_BPF_SYSCALL`, `CONFIG_BPF_JIT`, `CONFIG_BPF_EVENTS`, `CONFIG_DEBUG_INFO_BTF`, `CONFIG_MEMCG`; `CONFIG_KALLSYMS` improves symbol output |
|
||||
| cgroup | cgroup v2 with memory controller; demo mode needs cgroup admin access |
|
||||
| Privileges | Root or equivalent BPF and tracing capabilities |
|
||||
| Architecture | x86-64 tested; no special hardware needed |
|
||||
|
||||
## Scope
|
||||
## Implementation Scope
|
||||
|
||||
Profiles accumulate from attachment until exit and use bounded LRU maps: 4096 cgroup profiles, 4096 active intervals, 8192 stack aggregates, and 1024 unique stacks. One active interval is retained per `pid_tgid`, which matches the traced begin/end path. The tool captures kernel stacks rather than user stacks and treats symbolization as presentation, so restricted `kallsyms` changes names into addresses without changing the measurements.
|
||||
Profiles accumulate from program attach until exit, using bounded LRU maps: 4096 cgroup profiles, 4096 active intervals, 8192 stack aggregates, and 1024 unique stacks. One active interval is retained per `pid_tgid`, matching the begin/end tracing pattern. The tool captures kernel stacks rather than user stacks, and treats symbolization as presentation - restricted `kallsyms` changes names to addresses without affecting measurements.
|
||||
|
||||
## Summary
|
||||
|
||||
`oom-watch` turns the period before an OOM kill into evidence that can be inspected. It measures every matched memcg reclaim interval, samples and ranks the kernel paths, attributes work to the target cgroup, then joins that profile to victim selection and exit.
|
||||
`oom-watch` turns the chaos before an OOM kill into evidence you can examine. It measures every memcg reclaim interval, samples and ranks the kernel paths that consumed time, correctly attributes work to the target cgroup (even for cross-cgroup reclaim), and connects this profile to the victim selection and exit.
|
||||
|
||||
The next time a container dies and someone asks "what happened?", you'll have more than a one-line kernel log to show them.
|
||||
|
||||
> If you'd like to dive deeper into eBPF, check out our tutorial repository at <https://github.com/eunomia-bpf/bpf-developer-tutorial> or visit our website at <https://eunomia.dev/tutorials/>.
|
||||
|
||||
|
||||
@@ -1,26 +1,52 @@
|
||||
# eBPF 实战教程：在 OOM kill 之前分析 memcg reclaim
|
||||
# eBPF 实战教程：在 OOM Kill 之前分析内存回收
|
||||
|
||||
一条 OOM 日志回答了内存故障的最后一个问题：内核选择了哪个 task 作为 victim。此前发生的工作却很难从这条记录里看出来，排查时仍然需要知道目标 memory cgroup 进入了多少次 reclaim、每次花了多久、时间主要消耗在哪些内核路径，以及 victim 最终是否退出。
|
||||
你是否遇到过容器或服务突然被 Linux OOM killer 杀掉，只留下一行不明所以的内核日志？内核会告诉你它选择了*哪个*进程作为 victim，但对于导致这一结果的内存压力几乎只字未提。系统尝试了多少次内存回收？每次花了多久？哪些内核路径消耗了这些时间？
|
||||
|
||||
本课构建 `oom-watch` 保存这段缺失的历史。它把 memcg reclaim 聚合为延迟分布和采样内核栈，再把积累得到的 profile 关联到 OOM victim，并继续跟踪这条 victim 记录直到进程退出。
|
||||
本教程构建 `oom-watch`，一个 eBPF 工具，用于捕获 OOM kill *之前*发生的事情。它将每次 memcg 回收尝试记录为延迟直方图和采样的内核调用栈，然后将积累的 profile 附加到 OOM victim 上，并跟踪进程直到它退出。
|
||||
|
||||
> 完整源代码：<https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/57-oom-watch>
|
||||
|
||||
## reclaim 是 OOM kill 之前的故事
|
||||
## OOM 调试中缺失的一环
|
||||
|
||||
memory cgroup 接近上限时，一次内存分配可能进入 reclaim，扫描这个 memcg 中可以释放的页面。多次短暂尝试有时只能回收少量内存，也可能几乎没有进展，最终由 OOM killer 选择 victim。只观察 `oom/mark_victim` 会同时丢失这些尝试的延迟分布和调用路径。
|
||||
当 memory cgroup 接近其限制时，任何内存分配都可能触发回收。内核会扫描该 cgroup 中可释放的页面。有时几次短暂的尝试就能成功。有时，回收反复运行却进展甚微，直到 OOM killer 最终介入。
|
||||
|
||||
eBPF 可以让经过验证器检查的程序运行在内核事件上，再通过 map 连接不同的时刻。Linux 7.1 为 `mm_vmscan_memcg_reclaim_begin` 和 `mm_vmscan_memcg_reclaim_end` tracepoint 加入目标 `mem_cgroup`。这个参数很重要，因为当前 task 可以通过 `memory.reclaim` 回收另一个 cgroup；现在归因可以跟随真正被扫描的 memcg，而不是碰巧触发工作的 task。
|
||||
如果只观察 `oom/mark_victim` tracepoint，你会丢失所有这些上下文。你看到了 victim，却看不到之前的挣扎过程。运维人员需要这些问题的答案：
|
||||
|
||||
另一项依赖来自 victim lookup。`oom/mark_victim` 给出被选中线程的 ID，诊断还需要它的 thread-group ID 和 cgroup。`bpf_task_from_pid()` 在 Linux 6.2 引入，普通 tracepoint program 从 Linux 6.12 开始可以调用这类 tracing kfunc。更新的 vmscan tracepoint signature 最终把完整工具的最低内核版本定在 Linux 7.1。
|
||||
- OOM kill 之前发生了多少次回收周期？
|
||||
- 它们是微秒级的快速扫描，还是毫秒级的长时间停顿？
|
||||
- 回收是由 cgroup 内部的分配触发的，还是通过 `memory.reclaim` 从外部主动触发的？
|
||||
- 哪些内核函数占用了回收时间？
|
||||
|
||||
跟着一次 reclaim interval 走一遍。begin tracepoint 到来时，BPF 程序按当前 `pid_tgid` 保存单调时钟下的开始时间和目标 cgroup，并根据 `--sample-every` 捕获 kernel stack ID。对应的 end tracepoint 计算 duration，增加一个 2 倍区间的微秒直方图 bucket，再累计 reclaimed pages。另一张 map 按 `(cgroup_id, stack_id)` 聚合每条采样路径的样本数、总时间、最长时间和回收页数。
|
||||
`oom-watch` 可以回答所有这些问题。它通过 hook 内核的 vmscan tracepoint 来测量每个回收间隔，采样内核调用栈以展示时间花在了哪里，当 OOM 选择 victim 时，它会将累积的 profile 快照与 victim 信息一起输出。
|
||||
|
||||
OOM 选中 victim 后，程序把 TID 解析为 TGID 和 cgroup，把这个 cgroup 已积累的 reclaim profile 复制到 ring buffer event，并按 TID 保存 victim state。随后 `sched_process_exit` 消费这份状态并报告 exit code。最终结果会把 kill 之前的回收活动、内核标记的具体线程和之后的进程生命周期连在一起。
|
||||
## 为什么用 eBPF 做内存分析？
|
||||
|
||||
## profile 与 event 布局
|
||||
传统监控方法在这个场景下有严重的局限性。轮询 `/proc/meminfo` 或 cgroup 统计会错过短暂的回收事件。`perf` 可以捕获调用栈，但需要仔细配置和后处理。两种方法都不容易将回收活动与特定的 OOM 事件关联起来。
|
||||
|
||||
共享头文件定义 cgroup 级直方图、每 stack aggregate 和携带 OOM snapshot 的事件。
|
||||
eBPF 改变了这一切。程序直接在内核中运行，以纳秒级精度响应事件。Map 在事件之间传递状态，让我们可以构建直方图并关联 begin/end 对。Ring buffer 以最小的开销将事件传递给用户空间。而且因为 eBPF 程序在加载前经过验证，不会有崩溃内核的风险。
|
||||
|
||||
对于 `oom-watch`，我们使用几个 tracepoint：
|
||||
|
||||
- `mm_vmscan_memcg_reclaim_begin`：当特定 memory cgroup 的回收开始时触发
|
||||
- `mm_vmscan_memcg_reclaim_end`：当该回收间隔完成时触发
|
||||
- `oom/mark_victim`：当 OOM killer 选择 victim 时触发
|
||||
- `sched/sched_process_exit`：当 victim 进程退出时触发
|
||||
|
||||
Linux 7.1 添加了一个关键特性：vmscan tracepoint 现在包含被扫描的目标 `mem_cgroup`。这很重要，因为一个 cgroup 中的进程可以通过 `memory.reclaim` 触发另一个 cgroup 的回收。有了 tracepoint 中的目标 cgroup，我们可以正确归因这些工作。
|
||||
|
||||
## 架构概述
|
||||
|
||||
这个工具有三个主要组件协同工作。
|
||||
|
||||
**头文件**定义共享数据结构：包含延迟桶和计数器的 per-cgroup 回收 profile，跟踪采样和时间的 per-stack aggregate，以及 OOM 通知的事件结构。
|
||||
|
||||
**BPF 程序**在内核中运行。在回收开始时，它记录开始时间并可选地捕获内核调用栈。在回收结束时，它计算持续时间，更新直方图，并存储 stack aggregate。当 OOM 标记 victim 时，它查找 victim 的 cgroup，将累积的 profile 复制到事件中，并发送给用户空间。它还保存 victim 状态，以便在进程退出时进行报告。
|
||||
|
||||
**用户空间程序**加载内核符号用于调用栈符号化，管理可选的 cgroup 过滤器，处理来自 ring buffer 的事件，并包含一个自包含的 demo 模式，可以触发 OOM 来验证一切正常工作。
|
||||
|
||||
## Profile 数据结构
|
||||
|
||||
共享头文件定义了我们测量的内容：
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -79,11 +105,13 @@ struct oom_watch_event {
|
||||
#endif /* __OOM_WATCH_H */
|
||||
```
|
||||
|
||||
`latency_slots` 包含 20 个 base-2 微秒区间。bucket 0 覆盖 0–1 µs，bucket 1 覆盖 2–3 µs，之后依次是 4–7 µs 等范围，最后一个 bucket 收纳所有不小于 524288 µs 的 interval。event 内嵌完整 `reclaim_profile`，因此 victim 旁边打印的数字正是它被选中时的 profile snapshot。
|
||||
`latency_slots` 数组是一个 20 个桶的 2 次幂直方图。桶 0 覆盖 0-1 微秒，桶 1 覆盖 2-3 微秒，然后是 4-7 微秒，依此类推。最后一个桶捕获超过半秒的所有内容。这种对数分布可以高效地捕获快速回收周期和罕见的慢速周期。
|
||||
|
||||
## 分析 reclaim 并跟踪 victim
|
||||
当 OOM 事件触发时，它会嵌入 cgroup profile 的完整副本。与 victim 一起打印的数字准确显示了选择时的状态。
|
||||
|
||||
下面是完整 BPF 程序。
|
||||
## BPF 程序
|
||||
|
||||
下面是完整的 BPF 代码。我们将在代码之后解释它的工作原理：
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: GPL-2.0
|
||||
@@ -398,776 +426,34 @@ int capture_victim_exit(void *ctx)
|
||||
}
|
||||
```
|
||||
|
||||
`active_reclaims` 按 `pid_tgid` 关联 begin 与 end。begin callback 从 `memcg->css.cgroup->kn->id` 取得目标 cgroup ID，而不是读取当前 task 的 cgroup。两者比较产生 `cross_cgroup_reclaims`，另一个 cgroup 发起的 proactive reclaim 因此可以被直接观察。
|
||||
程序使用多个 BPF map 来维护状态。`active_reclaims` map 通过以 `pid_tgid` 为 key 存储开始时间戳来关联 begin 和 end 事件。`profiles` map 累积每个 cgroup 的统计信息。`stack_traces` map 存储去重后的内核调用栈，而 `stack_profiles` 为每个唯一调用栈聚合时间数据。
|
||||
|
||||
stack sampling 发生在 begin，也就是 reclaim 工作开始之前。`sample_every=1` 会捕获每个 interval，设置更大的值可以降低 stack map 与 unwinding 开销，同时 latency histogram 仍会统计所有匹配 interval。flag 低 8 位中的 `2` 会跳过两个 tracing frame，`bpf_get_stackid()` 再把剩余的相同调用栈合并到 `stack_traces`，`stack_profiles` 则为每个 ID 保存 timing 与 reclaimed-page 总计。原子更新让多个 CPU 的 reclaim 可以共同写入同一份 cgroup profile。
|
||||
在回收开始时，我们从 `mem_cgroup` 参数而不是当前 task 的 cgroup 中提取目标 cgroup ID。这个区别对于跨 cgroup 回收很重要。如果有人从外部对某个 cgroup 调用 `memory.reclaim`，我们可以正确地将工作归因到目标。`cross_cgroup_reclaims` 计数器跟踪这种情况发生的频率。
|
||||
|
||||
`profile_reclaim_end()` 完成统计后删除 active state。它会更新总延迟和最大延迟，选择 histogram bucket，再更新对应的 sampled stack aggregate。active-state 插入失败和 stack-capture 失败都有独立计数，begin 与 end count 的差值也能反映未完成配对的 interval。
|
||||
调用栈采样使用 `--sample-every` 设置。默认值为 1 时，我们捕获每个间隔。更大的值可以减少开销，同时仍然在直方图中计算所有间隔。`BPF_F_FAST_STACK_CMP` 标志加速调用栈去重，标志中的 `2` 跳过两个 tracing 帧以获得更清晰的调用栈。
|
||||
|
||||
OOM 路径需要谨慎处理进程 identity。tracepoint 中的 `ctx->pid` 保留为 `victim_tid`，`bpf_task_from_pid()` 提供 task 的 TGID 与 cgroup ID，`bpf_task_release()` 释放带引用的 task pointer。victim state 继续按 TID 存储，因为 `sched_process_exit` 会在同一个线程上下文中运行。即使多线程进程的 leader 和被选 victim ID 不同，这条关联仍然有效。
|
||||
OOM 处理程序必须仔细解析进程身份。tracepoint 给我们一个线程 ID，但我们还需要线程组 ID（用户空间视角的 PID）和 cgroup。我们使用 `bpf_task_from_pid()` 查找 task，读取所需信息，然后释放引用。victim 状态以 TID 为 key，因为 `sched_process_exit` 在该线程的上下文中触发。
|
||||
|
||||
## 符号化并展示 profile
|
||||
## 用户空间：符号化和展示
|
||||
|
||||
用户态程序加载 kernel symbol、排序 stack aggregate、管理可选 cgroup filter，并提供可重复的 OOM demo。
|
||||
用户空间代码负责几项工作：加载内核符号用于调用栈符号化，使用任何 cgroup 过滤器设置 BPF 程序，处理来自 ring buffer 的事件，以及按总耗时对调用栈进行排名。它还包含一个 demo 模式，可以创建一个内存受限的 cgroup 并触发 OOM 来验证工具是否正常工作。
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <getopt.h>
|
||||
#include <poll.h>
|
||||
#include <pthread.h>
|
||||
#include <signal.h>
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/mman.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/wait.h>
|
||||
#include <time.h>
|
||||
#include <unistd.h>
|
||||
#include <bpf/bpf.h>
|
||||
#include <bpf/libbpf.h>
|
||||
#include "oom_watch.h"
|
||||
#include "oom_watch.skel.h"
|
||||
完整的用户空间代码相当长，所以我们重点介绍关键部分。启动时，它读取 `/proc/kallsyms`，按地址排序符号，然后使用二分查找解析每个栈帧。当符号地址受限时（在生产系统上很常见），它仍然打印原始地址——测量结果保持准确。
|
||||
|
||||
struct options {
|
||||
const char *cgroup_path;
|
||||
unsigned int duration_seconds;
|
||||
unsigned int sample_every;
|
||||
bool demo;
|
||||
};
|
||||
调用栈组按累计回收时间排名。这种排名可以展示频繁调用的路径和罕见的慢速间隔。在 OOM 事件之后，打印前五个调用栈及其时间统计。
|
||||
|
||||
static volatile sig_atomic_t stop;
|
||||
static int victim_events;
|
||||
static int exit_events;
|
||||
static unsigned long long observed_cgroup_id;
|
||||
static unsigned int observed_victim_pid;
|
||||
static unsigned int observed_victim_tid;
|
||||
static unsigned long long observed_reclaims;
|
||||
static unsigned long long observed_cross_cgroup_reclaims;
|
||||
static unsigned long long observed_stack_samples;
|
||||
|
||||
struct kernel_symbol {
|
||||
unsigned long long address;
|
||||
char *name;
|
||||
};
|
||||
|
||||
struct kernel_symbols {
|
||||
struct kernel_symbol *items;
|
||||
size_t count;
|
||||
size_t capacity;
|
||||
};
|
||||
|
||||
struct runtime_context {
|
||||
int profiles_fd;
|
||||
int stack_profiles_fd;
|
||||
int stack_traces_fd;
|
||||
struct kernel_symbols symbols;
|
||||
};
|
||||
|
||||
struct oom_runtime {
|
||||
struct oom_watch_bpf *skel;
|
||||
struct ring_buffer *ring;
|
||||
struct runtime_context context;
|
||||
};
|
||||
|
||||
struct selected_cgroup {
|
||||
char demo_path[256];
|
||||
const char *path;
|
||||
struct stat metadata;
|
||||
bool demo_created;
|
||||
bool memory_enabled_by_demo;
|
||||
};
|
||||
|
||||
struct demo_process {
|
||||
pid_t child;
|
||||
int ready_pipe[2];
|
||||
int continue_pipe[2];
|
||||
int status;
|
||||
};
|
||||
|
||||
struct ranked_stack {
|
||||
struct reclaim_stack_key key;
|
||||
struct reclaim_stack_profile profile;
|
||||
};
|
||||
|
||||
struct allocation_context {
|
||||
int ready_fd;
|
||||
int continue_fd;
|
||||
};
|
||||
|
||||
static struct allocation_context allocation_context;
|
||||
|
||||
static void handle_signal(int signal_number)
|
||||
{
|
||||
(void)signal_number;
|
||||
stop = 1;
|
||||
}
|
||||
|
||||
static unsigned long long monotonic_ns(void)
|
||||
{
|
||||
struct timespec now;
|
||||
|
||||
clock_gettime(CLOCK_MONOTONIC, &now);
|
||||
return (unsigned long long)now.tv_sec * 1000000000ULL + now.tv_nsec;
|
||||
}
|
||||
|
||||
static int write_text(const char *path, const char *text)
|
||||
{
|
||||
int fd = open(path, O_WRONLY | O_CLOEXEC);
|
||||
ssize_t length = strlen(text);
|
||||
int err = 0;
|
||||
|
||||
if (fd < 0)
|
||||
return -1;
|
||||
if (write(fd, text, length) != length)
|
||||
err = -1;
|
||||
close(fd);
|
||||
return err;
|
||||
}
|
||||
|
||||
static int memory_controller_enabled(bool *enabled)
|
||||
{
|
||||
char controllers[4096];
|
||||
ssize_t length;
|
||||
int fd;
|
||||
|
||||
fd = open("/sys/fs/cgroup/cgroup.subtree_control",
|
||||
O_RDONLY | O_CLOEXEC);
|
||||
if (fd < 0)
|
||||
return -1;
|
||||
length = read(fd, controllers, sizeof(controllers) - 1);
|
||||
close(fd);
|
||||
if (length < 0)
|
||||
return -1;
|
||||
controllers[length] = '\0';
|
||||
*enabled = strstr(controllers, "memory") != NULL;
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int compare_symbols(const void *left, const void *right)
|
||||
{
|
||||
const struct kernel_symbol *a = left;
|
||||
const struct kernel_symbol *b = right;
|
||||
|
||||
return a->address < b->address ? -1 : a->address > b->address ? 1 : 0;
|
||||
}
|
||||
|
||||
static int load_kernel_symbols(struct kernel_symbols *symbols)
|
||||
{
|
||||
char name[256];
|
||||
char type;
|
||||
unsigned long long address;
|
||||
FILE *file = fopen("/proc/kallsyms", "r");
|
||||
|
||||
if (!file)
|
||||
return -1;
|
||||
while (fscanf(file, "%llx %c %255s%*[^\n]\n", &address, &type,
|
||||
name) == 3) {
|
||||
struct kernel_symbol *item;
|
||||
|
||||
(void)type;
|
||||
if (symbols->count == symbols->capacity) {
|
||||
size_t capacity = symbols->capacity ? symbols->capacity * 2 : 4096;
|
||||
void *items = realloc(symbols->items,
|
||||
capacity * sizeof(*symbols->items));
|
||||
|
||||
if (!items)
|
||||
goto error;
|
||||
symbols->items = items;
|
||||
symbols->capacity = capacity;
|
||||
}
|
||||
item = &symbols->items[symbols->count++];
|
||||
item->address = address;
|
||||
item->name = strdup(name);
|
||||
if (!item->name)
|
||||
goto error;
|
||||
}
|
||||
fclose(file);
|
||||
qsort(symbols->items, symbols->count, sizeof(*symbols->items),
|
||||
compare_symbols);
|
||||
return symbols->count ? 0 : -1;
|
||||
|
||||
error:
|
||||
fclose(file);
|
||||
return -1;
|
||||
}
|
||||
|
||||
static void free_kernel_symbols(struct kernel_symbols *symbols)
|
||||
{
|
||||
for (size_t i = 0; i < symbols->count; i++)
|
||||
free(symbols->items[i].name);
|
||||
free(symbols->items);
|
||||
}
|
||||
|
||||
static const struct kernel_symbol *find_kernel_symbol(
|
||||
const struct kernel_symbols *symbols, unsigned long long address)
|
||||
{
|
||||
size_t low = 0, high = symbols->count;
|
||||
|
||||
while (low < high) {
|
||||
size_t middle = low + (high - low) / 2;
|
||||
|
||||
if (symbols->items[middle].address <= address)
|
||||
low = middle + 1;
|
||||
else
|
||||
high = middle;
|
||||
}
|
||||
return low ? &symbols->items[low - 1] : NULL;
|
||||
}
|
||||
|
||||
static void insert_ranked_stack(struct ranked_stack top[5], size_t *count,
|
||||
const struct reclaim_stack_key *key,
|
||||
const struct reclaim_stack_profile *profile)
|
||||
{
|
||||
size_t position = 0;
|
||||
|
||||
while (position < *count &&
|
||||
top[position].profile.total_ns >= profile->total_ns)
|
||||
position++;
|
||||
if (position >= 5)
|
||||
return;
|
||||
if (*count < 5)
|
||||
(*count)++;
|
||||
for (size_t i = *count - 1; i > position; i--)
|
||||
top[i] = top[i - 1];
|
||||
top[position].key = *key;
|
||||
top[position].profile = *profile;
|
||||
}
|
||||
|
||||
static void print_reclaim_stacks(struct runtime_context *runtime,
|
||||
__u64 cgroup_id)
|
||||
{
|
||||
struct ranked_stack top[5] = {};
|
||||
struct reclaim_stack_key previous, next;
|
||||
bool have_previous = false;
|
||||
size_t count = 0;
|
||||
|
||||
while (!bpf_map_get_next_key(runtime->stack_profiles_fd,
|
||||
have_previous ? &previous : NULL, &next)) {
|
||||
struct reclaim_stack_profile profile;
|
||||
|
||||
if (next.cgroup_id == cgroup_id &&
|
||||
!bpf_map_lookup_elem(runtime->stack_profiles_fd, &next,
|
||||
&profile))
|
||||
insert_ranked_stack(top, &count, &next, &profile);
|
||||
previous = next;
|
||||
have_previous = true;
|
||||
}
|
||||
|
||||
for (size_t rank = 0; rank < count; rank++) {
|
||||
unsigned long long addresses[OOM_STACK_DEPTH] = {};
|
||||
|
||||
printf("reclaim_stack rank=%zu samples=%llu total_ms=%.3f "
|
||||
"max_ms=%.3f reclaimed_pages=%llu\n",
|
||||
rank + 1, top[rank].profile.samples,
|
||||
top[rank].profile.total_ns / 1000000.0,
|
||||
top[rank].profile.maximum_ns / 1000000.0,
|
||||
top[rank].profile.reclaimed_pages);
|
||||
if (bpf_map_lookup_elem(runtime->stack_traces_fd,
|
||||
&top[rank].key.stack_id, addresses))
|
||||
continue;
|
||||
for (size_t frame = 0; frame < OOM_STACK_DEPTH && addresses[frame];
|
||||
frame++) {
|
||||
const struct kernel_symbol *symbol =
|
||||
find_kernel_symbol(&runtime->symbols, addresses[frame]);
|
||||
|
||||
if (symbol && symbol->address)
|
||||
printf(" #%zu %s+0x%llx\n", frame, symbol->name,
|
||||
addresses[frame] - symbol->address);
|
||||
else
|
||||
printf(" #%zu 0x%llx\n", frame, addresses[frame]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static void print_reclaim_profile(struct runtime_context *runtime,
|
||||
__u64 cgroup_id,
|
||||
const struct reclaim_profile *profile)
|
||||
{
|
||||
printf("reclaim_profile cgroup_id=%llu cycles=%llu completed=%llu "
|
||||
"total_ms=%.3f max_ms=%.3f reclaimed_pages=%llu "
|
||||
"cross_cgroup=%llu stack_samples=%llu stack_failures=%llu\n",
|
||||
(unsigned long long)cgroup_id, profile->begin_count,
|
||||
profile->end_count, profile->total_reclaim_ns / 1000000.0,
|
||||
profile->maximum_reclaim_ns / 1000000.0,
|
||||
profile->reclaimed_pages, profile->cross_cgroup_reclaims,
|
||||
profile->stack_samples, profile->stack_failures);
|
||||
for (unsigned int bucket = 0; bucket < OOM_RECLAIM_BUCKETS; bucket++) {
|
||||
unsigned long long low, high;
|
||||
|
||||
if (!profile->latency_slots[bucket])
|
||||
continue;
|
||||
low = bucket ? 1ULL << bucket : 0;
|
||||
high = (1ULL << (bucket + 1)) - 1;
|
||||
if (bucket == OOM_RECLAIM_BUCKETS - 1)
|
||||
printf("reclaim_latency_us=>=%llu count=%llu\n", low,
|
||||
profile->latency_slots[bucket]);
|
||||
else
|
||||
printf("reclaim_latency_us=%llu-%llu count=%llu\n", low,
|
||||
high, profile->latency_slots[bucket]);
|
||||
}
|
||||
print_reclaim_stacks(runtime, cgroup_id);
|
||||
}
|
||||
|
||||
static void print_live_profiles(struct runtime_context *runtime)
|
||||
{
|
||||
__u64 previous, next;
|
||||
bool have_previous = false;
|
||||
|
||||
while (!bpf_map_get_next_key(runtime->profiles_fd,
|
||||
have_previous ? &previous : NULL, &next)) {
|
||||
struct reclaim_profile profile;
|
||||
|
||||
if (!bpf_map_lookup_elem(runtime->profiles_fd, &next, &profile))
|
||||
print_reclaim_profile(runtime, next, &profile);
|
||||
previous = next;
|
||||
have_previous = true;
|
||||
}
|
||||
}
|
||||
|
||||
static int handle_event(void *ctx, void *data, size_t size)
|
||||
{
|
||||
const struct oom_watch_event *event = data;
|
||||
struct runtime_context *runtime = ctx;
|
||||
|
||||
if (size != sizeof(*event))
|
||||
return 0;
|
||||
if (event->type == OOM_VICTIM_MARKED) {
|
||||
victim_events++;
|
||||
observed_cgroup_id = event->cgroup_id;
|
||||
observed_victim_pid = event->victim_pid;
|
||||
observed_reclaims = event->profile.begin_count;
|
||||
observed_victim_tid = event->victim_tid;
|
||||
observed_cross_cgroup_reclaims =
|
||||
event->profile.cross_cgroup_reclaims;
|
||||
observed_stack_samples = event->profile.stack_samples;
|
||||
printf("event=oom-victim pid=%u tid=%u comm=%s trigger_pid=%u cgroup_id=%llu "
|
||||
"anon_rss_kb=%llu file_rss_kb=%llu total_vm_kb=%llu "
|
||||
"reclaim_cycles=%llu cross_cgroup_reclaims=%llu "
|
||||
"reclaimed_pages=%llu\n",
|
||||
event->victim_pid, event->victim_tid, event->comm,
|
||||
event->triggering_tgid,
|
||||
(unsigned long long)event->cgroup_id,
|
||||
(unsigned long long)event->anon_rss_kb,
|
||||
(unsigned long long)event->file_rss_kb,
|
||||
(unsigned long long)event->total_vm_kb,
|
||||
(unsigned long long)event->profile.begin_count,
|
||||
(unsigned long long)event->profile.cross_cgroup_reclaims,
|
||||
(unsigned long long)event->profile.reclaimed_pages);
|
||||
print_reclaim_profile(runtime, event->cgroup_id, &event->profile);
|
||||
} else if (event->type == OOM_VICTIM_EXITED) {
|
||||
exit_events++;
|
||||
printf("event=victim-exit pid=%u tid=%u cgroup_id=%llu exit_code=%d\n",
|
||||
event->victim_pid, event->victim_tid,
|
||||
(unsigned long long)event->cgroup_id, event->exit_code);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int parse_uint(const char *text, unsigned int maximum,
|
||||
unsigned int *value)
|
||||
{
|
||||
char *end = NULL;
|
||||
unsigned long parsed;
|
||||
|
||||
errno = 0;
|
||||
parsed = strtoul(text, &end, 10);
|
||||
if (errno || !*text || *end || !parsed || parsed > maximum)
|
||||
return -1;
|
||||
*value = parsed;
|
||||
return 0;
|
||||
}
|
||||
|
||||
static void usage(const char *program)
|
||||
{
|
||||
printf("Usage: %s [--cgroup PATH] [--duration SEC] [--sample-every N]\n"
|
||||
" %s --demo [--sample-every N]\n", program, program);
|
||||
}
|
||||
|
||||
static int parse_options(int argc, char **argv, struct options *options)
|
||||
{
|
||||
static const struct option long_options[] = {
|
||||
{ "cgroup", required_argument, NULL, 'c' },
|
||||
{ "duration", required_argument, NULL, 'd' },
|
||||
{ "sample-every", required_argument, NULL, 's' },
|
||||
{ "demo", no_argument, NULL, 'D' },
|
||||
{ "help", no_argument, NULL, 'h' },
|
||||
{},
|
||||
};
|
||||
int option;
|
||||
|
||||
while ((option = getopt_long(argc, argv, "c:d:s:Dh", long_options,
|
||||
NULL)) != -1) {
|
||||
switch (option) {
|
||||
case 'c': options->cgroup_path = optarg; break;
|
||||
case 'd':
|
||||
if (parse_uint(optarg, 86400, &options->duration_seconds))
|
||||
return -1;
|
||||
break;
|
||||
case 's':
|
||||
if (parse_uint(optarg, 1000000, &options->sample_every))
|
||||
return -1;
|
||||
break;
|
||||
case 'D': options->demo = true; break;
|
||||
case 'h': usage(argv[0]); exit(0);
|
||||
default: return -1;
|
||||
}
|
||||
}
|
||||
return optind == argc && !(options->demo && options->cgroup_path) ? 0 : -1;
|
||||
}
|
||||
|
||||
static void *allocation_worker(void *argument)
|
||||
{
|
||||
struct allocation_context *context = argument;
|
||||
size_t first_stage = 24 * 1024 * 1024;
|
||||
size_t length = 128 * 1024 * 1024;
|
||||
unsigned char *memory;
|
||||
char byte = 'x';
|
||||
|
||||
memory = mmap(NULL, length, PROT_READ | PROT_WRITE,
|
||||
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
|
||||
if (memory == MAP_FAILED)
|
||||
_exit(4);
|
||||
for (size_t offset = 0; offset < first_stage; offset += 4096)
|
||||
memory[offset] = 0xa5;
|
||||
if (write(context->ready_fd, &byte, 1) != 1 ||
|
||||
read(context->continue_fd, &byte, 1) != 1)
|
||||
_exit(5);
|
||||
for (size_t offset = first_stage; offset < length; offset += 4096)
|
||||
memory[offset] = 0xa5;
|
||||
_exit(6);
|
||||
}
|
||||
|
||||
static void allocate_until_killed(const char *cgroup_path, int ready_fd,
|
||||
int continue_fd)
|
||||
{
|
||||
char procs_path[512];
|
||||
char pid_text[32];
|
||||
pthread_t worker;
|
||||
|
||||
snprintf(procs_path, sizeof(procs_path), "%s/cgroup.procs", cgroup_path);
|
||||
snprintf(pid_text, sizeof(pid_text), "%d", getpid());
|
||||
if (write_text(procs_path, pid_text))
|
||||
_exit(3);
|
||||
allocation_context.ready_fd = ready_fd;
|
||||
allocation_context.continue_fd = continue_fd;
|
||||
if (pthread_create(&worker, NULL, allocation_worker,
|
||||
&allocation_context))
|
||||
_exit(4);
|
||||
pthread_detach(worker);
|
||||
pthread_exit(NULL);
|
||||
}
|
||||
|
||||
static int configure_demo_cgroup(const char *path, bool *created,
|
||||
bool *enabled_by_demo)
|
||||
{
|
||||
bool memory_enabled;
|
||||
char file[512];
|
||||
|
||||
if (memory_controller_enabled(&memory_enabled))
|
||||
return -1;
|
||||
if (!memory_enabled) {
|
||||
if (write_text("/sys/fs/cgroup/cgroup.subtree_control", "+memory"))
|
||||
return -1;
|
||||
*enabled_by_demo = true;
|
||||
}
|
||||
if (mkdir(path, 0755))
|
||||
return -1;
|
||||
*created = true;
|
||||
snprintf(file, sizeof(file), "%s/memory.max", path);
|
||||
if (write_text(file, "33554432"))
|
||||
return -1;
|
||||
snprintf(file, sizeof(file), "%s/memory.swap.max", path);
|
||||
if (write_text(file, "0"))
|
||||
return -1;
|
||||
snprintf(file, sizeof(file), "%s/memory.oom.group", path);
|
||||
return write_text(file, "1");
|
||||
}
|
||||
|
||||
static int trigger_cross_cgroup_reclaim(const char *cgroup_path)
|
||||
{
|
||||
char reclaim_path[512];
|
||||
|
||||
snprintf(reclaim_path, sizeof(reclaim_path), "%s/memory.reclaim",
|
||||
cgroup_path);
|
||||
if (!write_text(reclaim_path, "8388608") || errno == EAGAIN)
|
||||
return 0;
|
||||
return -1;
|
||||
}
|
||||
|
||||
static int select_cgroup(const struct options *options,
|
||||
struct selected_cgroup *selected)
|
||||
{
|
||||
if (options->demo) {
|
||||
snprintf(selected->demo_path, sizeof(selected->demo_path),
|
||||
"/sys/fs/cgroup/ebpf-oom-watch-%d", getpid());
|
||||
if (configure_demo_cgroup(selected->demo_path,
|
||||
&selected->demo_created,
|
||||
&selected->memory_enabled_by_demo)) {
|
||||
fprintf(stderr, "failed to configure demo memory cgroup: %s\n",
|
||||
strerror(errno));
|
||||
return -1;
|
||||
}
|
||||
selected->path = selected->demo_path;
|
||||
} else {
|
||||
selected->path = options->cgroup_path;
|
||||
}
|
||||
if (!selected->path)
|
||||
return 0;
|
||||
if (!stat(selected->path, &selected->metadata))
|
||||
return 0;
|
||||
fprintf(stderr, "failed to stat cgroup %s: %s\n", selected->path,
|
||||
strerror(errno));
|
||||
return -1;
|
||||
}
|
||||
|
||||
static void cleanup_selected_cgroup(struct selected_cgroup *selected,
|
||||
int *result)
|
||||
{
|
||||
if (selected->demo_created && rmdir(selected->demo_path) && !*result)
|
||||
*result = 1;
|
||||
if (selected->memory_enabled_by_demo &&
|
||||
write_text("/sys/fs/cgroup/cgroup.subtree_control", "-memory") &&
|
||||
!*result)
|
||||
*result = 1;
|
||||
}
|
||||
|
||||
static int prepare_runtime(struct oom_runtime *runtime,
|
||||
const struct options *options,
|
||||
const struct selected_cgroup *selected)
|
||||
{
|
||||
runtime->skel = oom_watch_bpf__open();
|
||||
if (!runtime->skel)
|
||||
return -1;
|
||||
runtime->skel->rodata->target_cgroup_id =
|
||||
selected->path ? selected->metadata.st_ino : 0;
|
||||
runtime->skel->rodata->sample_every = options->sample_every;
|
||||
if (oom_watch_bpf__load(runtime->skel) ||
|
||||
oom_watch_bpf__attach(runtime->skel)) {
|
||||
fprintf(stderr, "failed to load and attach OOM watcher\n");
|
||||
return -1;
|
||||
}
|
||||
runtime->context.profiles_fd =
|
||||
bpf_map__fd(runtime->skel->maps.profiles);
|
||||
runtime->context.stack_profiles_fd =
|
||||
bpf_map__fd(runtime->skel->maps.stack_profiles);
|
||||
runtime->context.stack_traces_fd =
|
||||
bpf_map__fd(runtime->skel->maps.stack_traces);
|
||||
if (load_kernel_symbols(&runtime->context.symbols))
|
||||
fprintf(stderr, "warning: kernel symbols unavailable; printing raw stack addresses\n");
|
||||
runtime->ring = ring_buffer__new(
|
||||
bpf_map__fd(runtime->skel->maps.events), handle_event,
|
||||
&runtime->context, NULL);
|
||||
return runtime->ring ? 0 : -1;
|
||||
}
|
||||
|
||||
static void destroy_runtime(struct oom_runtime *runtime)
|
||||
{
|
||||
ring_buffer__free(runtime->ring);
|
||||
free_kernel_symbols(&runtime->context.symbols);
|
||||
oom_watch_bpf__destroy(runtime->skel);
|
||||
}
|
||||
|
||||
static void init_demo_process(struct demo_process *demo)
|
||||
{
|
||||
memset(demo, 0, sizeof(*demo));
|
||||
demo->child = -1;
|
||||
demo->ready_pipe[0] = -1;
|
||||
demo->ready_pipe[1] = -1;
|
||||
demo->continue_pipe[0] = -1;
|
||||
demo->continue_pipe[1] = -1;
|
||||
}
|
||||
|
||||
static void close_demo_pipe(int *fd)
|
||||
{
|
||||
if (*fd >= 0)
|
||||
close(*fd);
|
||||
*fd = -1;
|
||||
}
|
||||
|
||||
static void cleanup_demo_process(struct demo_process *demo)
|
||||
{
|
||||
if (demo->child > 0) {
|
||||
kill(demo->child, SIGKILL);
|
||||
waitpid(demo->child, NULL, 0);
|
||||
}
|
||||
close_demo_pipe(&demo->ready_pipe[0]);
|
||||
close_demo_pipe(&demo->ready_pipe[1]);
|
||||
close_demo_pipe(&demo->continue_pipe[0]);
|
||||
close_demo_pipe(&demo->continue_pipe[1]);
|
||||
}
|
||||
|
||||
static int start_demo_process(struct demo_process *demo,
|
||||
const char *cgroup_path)
|
||||
{
|
||||
struct pollfd ready = { .events = POLLIN };
|
||||
struct timespec leader_exit_delay = { .tv_nsec = 100000000 };
|
||||
char byte = 'x';
|
||||
|
||||
if (pipe(demo->ready_pipe) || pipe(demo->continue_pipe))
|
||||
return -1;
|
||||
demo->child = fork();
|
||||
if (demo->child < 0)
|
||||
return -1;
|
||||
if (!demo->child) {
|
||||
close(demo->ready_pipe[0]);
|
||||
close(demo->continue_pipe[1]);
|
||||
allocate_until_killed(cgroup_path, demo->ready_pipe[1],
|
||||
demo->continue_pipe[0]);
|
||||
}
|
||||
close_demo_pipe(&demo->ready_pipe[1]);
|
||||
close_demo_pipe(&demo->continue_pipe[0]);
|
||||
ready.fd = demo->ready_pipe[0];
|
||||
if (poll(&ready, 1, 5000) != 1 ||
|
||||
read(demo->ready_pipe[0], &byte, 1) != 1 ||
|
||||
trigger_cross_cgroup_reclaim(cgroup_path))
|
||||
return -1;
|
||||
nanosleep(&leader_exit_delay, NULL);
|
||||
if (write(demo->continue_pipe[1], &byte, 1) != 1)
|
||||
return -1;
|
||||
close_demo_pipe(&demo->ready_pipe[0]);
|
||||
close_demo_pipe(&demo->continue_pipe[1]);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int collect_demo_events(struct demo_process *demo,
|
||||
struct ring_buffer *ring)
|
||||
{
|
||||
for (int i = 0; i < 200; i++) {
|
||||
pid_t waited;
|
||||
|
||||
ring_buffer__poll(ring, 50);
|
||||
waited = waitpid(demo->child, &demo->status, WNOHANG);
|
||||
if (waited == demo->child) {
|
||||
demo->child = -1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
for (int i = 0; i < 10 && exit_events < 1; i++)
|
||||
ring_buffer__poll(ring, 50);
|
||||
return demo->child < 0 ? 0 : -1;
|
||||
}
|
||||
|
||||
static bool valid_demo_observation(const struct demo_process *demo,
|
||||
unsigned long long cgroup_id,
|
||||
const struct oom_watch_bpf *skel)
|
||||
{
|
||||
return WIFSIGNALED(demo->status) &&
|
||||
WTERMSIG(demo->status) == SIGKILL && victim_events == 1 &&
|
||||
exit_events == 1 && observed_cgroup_id == cgroup_id &&
|
||||
observed_victim_pid && observed_victim_tid &&
|
||||
observed_victim_pid != observed_victim_tid && observed_reclaims &&
|
||||
observed_cross_cgroup_reclaims && observed_stack_samples &&
|
||||
!skel->bss->dropped_victim_states &&
|
||||
!skel->bss->dropped_reclaim_states;
|
||||
}
|
||||
|
||||
static int run_demo(struct oom_runtime *runtime,
|
||||
const struct selected_cgroup *selected)
|
||||
{
|
||||
struct demo_process demo;
|
||||
int result = -1;
|
||||
|
||||
init_demo_process(&demo);
|
||||
if (start_demo_process(&demo, selected->path) ||
|
||||
collect_demo_events(&demo, runtime->ring))
|
||||
goto cleanup;
|
||||
printf("demo workload signaled=%d signal=%d\n",
|
||||
WIFSIGNALED(demo.status),
|
||||
WIFSIGNALED(demo.status) ? WTERMSIG(demo.status) : 0);
|
||||
if (!valid_demo_observation(&demo, selected->metadata.st_ino,
|
||||
runtime->skel))
|
||||
goto cleanup;
|
||||
printf("demo result=matched-profile-to-victim\n");
|
||||
result = 0;
|
||||
|
||||
cleanup:
|
||||
cleanup_demo_process(&demo);
|
||||
return result;
|
||||
}
|
||||
|
||||
static int watch_profiles(struct oom_runtime *runtime,
|
||||
unsigned int duration_seconds)
|
||||
{
|
||||
unsigned long long deadline = 0;
|
||||
|
||||
signal(SIGINT, handle_signal);
|
||||
signal(SIGTERM, handle_signal);
|
||||
if (duration_seconds)
|
||||
deadline = monotonic_ns() +
|
||||
(unsigned long long)duration_seconds * 1000000000ULL;
|
||||
while (!stop && (!deadline || monotonic_ns() < deadline)) {
|
||||
int result = ring_buffer__poll(runtime->ring, 100);
|
||||
|
||||
if (result < 0 && result != -EINTR) {
|
||||
fprintf(stderr, "ring buffer poll failed: %d\n", result);
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
print_live_profiles(&runtime->context);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
struct options options = { .sample_every = 1 };
|
||||
struct selected_cgroup selected = {};
|
||||
struct oom_runtime runtime = {};
|
||||
int err = 1;
|
||||
|
||||
setvbuf(stdout, NULL, _IONBF, 0);
|
||||
if (parse_options(argc, argv, &options)) {
|
||||
usage(argv[0]);
|
||||
return 2;
|
||||
}
|
||||
if (select_cgroup(&options, &selected) ||
|
||||
prepare_runtime(&runtime, &options, &selected))
|
||||
goto cleanup;
|
||||
|
||||
if (selected.path)
|
||||
printf("oom-watch tracing cgroup=%s cgroup_id=%llu\n",
|
||||
selected.path,
|
||||
(unsigned long long)selected.metadata.st_ino);
|
||||
else
|
||||
printf("oom-watch tracing all cgroups\n");
|
||||
|
||||
if ((options.demo && run_demo(&runtime, &selected)) ||
|
||||
(!options.demo && watch_profiles(&runtime,
|
||||
options.duration_seconds)))
|
||||
goto cleanup;
|
||||
printf("dropped_victim_states=%llu dropped_reclaim_states=%llu\n",
|
||||
(unsigned long long)runtime.skel->bss->dropped_victim_states,
|
||||
(unsigned long long)runtime.skel->bss->dropped_reclaim_states);
|
||||
err = 0;
|
||||
|
||||
cleanup:
|
||||
destroy_runtime(&runtime);
|
||||
cleanup_selected_cgroup(&selected, &err);
|
||||
return err;
|
||||
}
|
||||
```
|
||||
|
||||
启动时，loader 读取 `/proc/kallsyms`，按地址排序，再用二分搜索解析 BPF stack-trace map 中的每个地址。symbol address 被隐藏时，同一份 profile 仍然可以输出 raw address。stack group 按累计 reclaim time 排序，因此调用频繁的路径和一次特别慢的 interval 都能反映在排名中。
|
||||
|
||||
普通模式可以观察一个 cgroup，也可以覆盖全部 cgroup。跟踪结束后，程序遍历 profile map，即使没有发生 OOM，也会打印仍然存在的 histogram。OOM event 到来时会立即打印 snapshot，并附上累计时间最高的 5 组调用栈和 frame。
|
||||
|
||||
demo 模式创建 `memory.max=32 MiB`、无 swap、启用 grouped OOM behavior 的 cgroup。worker 先 fault 24 MiB 并暂停，让 parent 通过 `memory.reclaim` 请求回收 8 MiB；这次有意制造的 cross-cgroup request 会验证 target-memcg attribution。随后 worker 继续 fault 一段 128 MiB mapping，直到 cgroup OOM killer 选中它。进程 leader 会在第二阶段之前退出，因此测试也会验证 TGID 和 victim TID 的独立处理。清理阶段会恢复启动时观察到的 memory controller 状态。
|
||||
demo 模式创建一个 `memory.max=32 MiB`、无 swap、启用分组 OOM 行为的 cgroup。一个 worker 进程首先 fault 24 MiB 然后暂停，让父进程从 cgroup 外部通过 `memory.reclaim` 请求 8 MiB。这会测试跨 cgroup 归因。然后 worker 继续 fault 一个 128 MiB 的映射直到 OOM 杀死它。进程 leader 在第二阶段之前退出，测试我们是否正确地独立处理 TGID 和 victim TID。
|
||||
|
||||
## 编译和运行
|
||||
|
||||
构建 profiler：
|
||||
构建工具：
|
||||
|
||||
```bash
|
||||
cd src/57-oom-watch
|
||||
make
|
||||
```
|
||||
|
||||
分析一个 service cgroup 60 秒，每 10 次 reclaim interval 采样一次 kernel stack：
|
||||
分析一个特定的 cgroup 60 秒，每十个回收间隔采样一次内核调用栈：
|
||||
|
||||
```bash
|
||||
sudo ./oom_watch \
|
||||
@@ -1176,13 +462,13 @@ sudo ./oom_watch \
|
||||
--sample-every 10
|
||||
```
|
||||
|
||||
省略 `--cgroup` 会观察全部 cgroup，省略 `--duration` 则持续运行到收到中断。内置 demo 会捕获每次 reclaim stack：
|
||||
省略 `--cgroup` 可以观察所有 cgroup。省略 `--duration` 可以持续运行直到中断。内置 demo 会捕获每次回收调用栈：
|
||||
|
||||
```bash
|
||||
sudo ./oom_watch --demo
|
||||
```
|
||||
|
||||
PID、cgroup ID、地址和延迟会随运行变化。下面是一次真实运行的节选，保留完整 profile 总计，并缩短 stack frame 便于阅读：
|
||||
下面是一次真实 demo 运行的输出。PID、cgroup ID、地址和时间在不同运行之间会有变化：
|
||||
|
||||
```text
|
||||
oom-watch tracing cgroup=/sys/fs/cgroup/ebpf-oom-watch-1262 cgroup_id=151
|
||||
@@ -1206,25 +492,27 @@ demo result=matched-profile-to-victim
|
||||
dropped_victim_states=0 dropped_reclaim_states=0
|
||||
```
|
||||
|
||||
44 个 completed interval 与 44 个 begin event 一一对应，所有 histogram count 相加也是 44，其中一半由目标 cgroup 之外触发。两条排名路径分别显示 allocation charge reclaim 和显式 `memory.reclaim`，exit event 则确认被标记的 TID 最终收到 `SIGKILL`。
|
||||
看看这告诉我们什么：在 kill 之前完成了 44 个回收周期，其中一半是从目标 cgroup 外部触发的。直方图显示大多数周期很快（不到 8 微秒），但有几个花了更长时间。两个排名的调用栈将分配触发的回收与显式的 `memory.reclaim` 请求区分开来。exit 事件确认 victim 收到了 `SIGKILL`。
|
||||
|
||||
## 环境要求
|
||||
|
||||
| 要求 | 说明 |
|
||||
|---|---|
|
||||
| 内核 | Linux 7.1 或更高版本，需要携带 target memcg 的 vmscan tracepoint |
|
||||
| 内核配置 | `CONFIG_BPF`、`CONFIG_BPF_SYSCALL`、`CONFIG_BPF_JIT`、`CONFIG_BPF_EVENTS`、`CONFIG_DEBUG_INFO_BTF`、`CONFIG_MEMCG`，`CONFIG_KALLSYMS` 可以改善符号输出 |
|
||||
| cgroup | cgroup v2 与 memory controller，demo 模式还需要 cgroup 管理写权限 |
|
||||
| 权限 | root，或者等价的 BPF、tracing 与 cgroup 管理 capability |
|
||||
| 架构与硬件 | 当前声明并完成测试的目标是 x86-64，不需要特殊硬件 |
|
||||
| 内核 | Linux 7.1 或更高版本（需要 target-memcg vmscan tracepoint） |
|
||||
| 内核配置 | `CONFIG_BPF`、`CONFIG_BPF_SYSCALL`、`CONFIG_BPF_JIT`、`CONFIG_BPF_EVENTS`、`CONFIG_DEBUG_INFO_BTF`、`CONFIG_MEMCG`；`CONFIG_KALLSYMS` 可以改善符号输出 |
|
||||
| cgroup | 带 memory controller 的 cgroup v2；demo 模式需要 cgroup 管理权限 |
|
||||
| 权限 | root 或等效的 BPF 和 tracing capability |
|
||||
| 架构 | 已在 x86-64 上测试；不需要特殊硬件 |
|
||||
|
||||
## 实现范围
|
||||
|
||||
profile 从程序挂载开始累计到退出，并使用有界 LRU map：4096 份 cgroup profile、4096 条 active interval、8192 组 stack aggregate 和 1024 个 unique stack。每个 `pid_tgid` 保存一个 active interval，与这里观察的 begin/end 路径一致。工具捕获 kernel stack，symbolization 只负责展示，因此受限的 `kallsyms` 会把名称换成地址，并不会改变测量结果。
|
||||
Profile 从程序附加开始累积直到退出，使用有界的 LRU map：4096 个 cgroup profile、4096 个活动间隔、8192 个调用栈聚合和 1024 个唯一调用栈。每个 `pid_tgid` 保留一个活动间隔，与 begin/end 跟踪模式相匹配。工具捕获内核调用栈而不是用户调用栈，并将符号化视为展示——受限的 `kallsyms` 会将名称变为地址，但不影响测量结果。
|
||||
|
||||
## 总结
|
||||
|
||||
`oom-watch` 把 OOM kill 之前的阶段变成可以检查的证据。它测量每个匹配的 memcg reclaim interval，采样并排序内核路径，把工作归因到目标 cgroup，再将 profile 与 victim selection 和 exit 关联起来。
|
||||
`oom-watch` 将 OOM kill 之前的混乱变成可以检查的证据。它测量每个 memcg 回收间隔，采样并排名消耗时间的内核路径，正确归因工作到目标 cgroup（即使对于跨 cgroup 回收），并将此 profile 与 victim 选择和退出关联起来。
|
||||
|
||||
下次容器死掉有人问"发生了什么？"时，你将不仅仅有一行内核日志可以展示。
|
||||
|
||||
> 如果你想深入了解 eBPF，请查看我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或访问我们的网站 <https://eunomia.dev/tutorials/>。
|
||||
|
||||
|
||||
@@ -1,24 +1,40 @@
|
||||
# eBPF Tutorial by Example: Receive UDP Packets with AF_XDP
|
||||
# eBPF Tutorial: High-Performance UDP Packet Capture with AF_XDP
|
||||
|
||||
Sometimes a packet needs to reach a user-space program before the normal socket stack processes it. A packet recorder, protocol prototype, or specialized forwarder may want a small, explicit receive path: select traffic at the driver boundary, place it in shared memory, inspect it, then return the buffer for another packet.
|
||||
Have you ever wondered how packet capture tools like Suricata or high-frequency trading systems receive millions of packets per second without drowning in kernel overhead? The secret is bypassing most of the network stack entirely. AF_XDP lets you intercept packets at the driver boundary, copy them directly into your application's memory, and process them without system calls for every packet.
|
||||
|
||||
This tutorial builds that path from the AF_XDP ABI rather than hiding it behind a library. The resulting `afxdp-dump` tool captures IPv4 UDP packets for one port and queue, prints a payload preview, and recycles every frame. It uses one queue, single-buffer packets, and copy mode, which makes it a practical first AF_XDP program before zero-copy and multi-buffer extensions.
|
||||
This tutorial builds a complete packet capture tool from scratch using the raw AF_XDP interface. No helper libraries, no magic abstractions. You'll see exactly how UMEM registration, ring buffers, and XDP redirection work together. The result is `afxdp-dump`, a tool that captures IPv4 UDP packets for a specific port, prints a payload preview, and properly recycles every frame to keep receiving indefinitely.
|
||||
|
||||
> Complete source code: <https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/60-afxdp-dump>
|
||||
|
||||
## XDP Selects; AF_XDP Delivers
|
||||
## Why AF_XDP?
|
||||
|
||||
eBPF runs verifier-checked programs at kernel hooks, and XDP places one of those hooks at the earliest receive point in the Linux network path. AF_XDP, added in Linux 4.18, connects an XDP redirect to a socket backed by user-registered memory. An XSKMAP supplies the missing association between an RX queue number and the AF_XDP socket that serves it.
|
||||
Traditional packet capture with `libpcap` or raw sockets has a fundamental problem: every packet crosses the kernel-userspace boundary through expensive system calls. When you're capturing 10 Gbps of traffic, this overhead becomes the bottleneck, not your processing code.
|
||||
|
||||
The roles are deliberately separate. The XDP program parses just enough of the packet to decide whether it belongs to the tool. The AF_XDP socket owns the shared-memory rings and carries selected bytes to user space. This example checks XSKMAP with `bpf_map_lookup_elem()` before redirecting, support added in Linux 5.3. Its compare-and-detach cleanup uses the expected-program FD added in Linux 5.7, which sets the complete tool's minimum kernel version.
|
||||
AF_XDP solves this by establishing shared memory between kernel and userspace. The kernel writes packets directly into memory your application can read. You communicate through lock-free ring buffers instead of system calls. A single `poll()` can wake you for hundreds of packets. This architecture enables packet rates of millions per second on commodity hardware.
|
||||
|
||||
Follow one packet and one frame. User space allocates 64 frames of 4096 bytes in UMEM and posts their addresses to the fill ring. A UDP packet for the configured port reaches the XDP hook. The program validates Ethernet, IPv4, and UDP lengths, finds the socket registered for `ctx->rx_queue_index`, and returns `XDP_REDIRECT`. In copy mode the kernel copies the packet into one posted frame and publishes an `xdp_desc` on the RX ring. User space reads the descriptor, prints the packet, advances the consumer index, and puts the same address back on the fill ring.
|
||||
The technology has real production use. Meta runs AF_XDP in their load balancers. Cilium uses it for Kubernetes networking. High-frequency trading firms use it to shave microseconds off their latency. Even if you never build a trading system, understanding AF_XDP teaches you patterns that appear throughout high-performance systems: shared memory, lock-free data structures, and explicit ownership transfer.
|
||||
|
||||
That last step is what keeps the receiver alive. Without recycling, the initial 64 addresses would be exhausted after 64 packets. A 65-packet run proves that at least one frame has completed the entire ownership cycle.
|
||||
## The AF_XDP Architecture
|
||||
|
||||
## Shared Queue Limit
|
||||
AF_XDP works through four components that must coordinate precisely:
|
||||
|
||||
The shared header sets the XSKMAP capacity. Queue IDs are the map keys, so this example can address queues 0 through 63 while binding one queue per process.
|
||||
**UMEM (User Memory)** is a region of memory you allocate that both kernel and userspace can access. You divide it into fixed-size frames, typically 4096 bytes each. Every packet the kernel delivers arrives in one of these frames.
|
||||
|
||||
**The Fill Ring** is how you tell the kernel which frames are available for incoming packets. You post frame addresses here. The kernel consumes these addresses when it needs somewhere to put a packet.
|
||||
|
||||
**The RX Ring** is where the kernel tells you about received packets. Each entry contains a frame address and packet length. When you see an entry here, you own that frame until you return it.
|
||||
|
||||
**XSKMAP** is a BPF map that connects XDP programs to AF_XDP sockets. The XDP program decides which packets to redirect, looks up the socket for the current RX queue in this map, and calls `bpf_redirect_map()` to deliver the packet.
|
||||
|
||||
The flow works like this: you post 64 frame addresses to the Fill Ring. A UDP packet arrives. Your XDP program checks the destination port, looks up the socket in XSKMAP, and redirects. The kernel copies the packet into one of your frames and publishes a descriptor on the RX Ring. You read the descriptor, process the packet, and post the frame address back to the Fill Ring. The cycle continues indefinitely.
|
||||
|
||||
This ownership model is critical. A frame starts with you. You lend it to the kernel via the Fill Ring. The kernel borrows it to receive a packet. You reclaim it from the RX Ring. You must return it to the Fill Ring or you'll run out of frames after 64 packets. Our tool proves this works by successfully capturing 65 packets, which requires at least one frame to complete the full ownership cycle.
|
||||
|
||||
## The XDP Program: Filtering and Redirecting
|
||||
|
||||
The kernel-side BPF program is compact because it only handles filtering and redirection. All the complexity of buffer management lives in userspace.
|
||||
|
||||
First, we define a shared header that sets the XSKMAP capacity. Queue IDs are map keys, so this example can address queues 0 through 63:
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -30,9 +46,7 @@ The shared header sets the XSKMAP capacity. Queue IDs are the map keys, so this
|
||||
#endif /* __AFXDP_DUMP_H */
|
||||
```
|
||||
|
||||
## Filtering and Redirecting in XDP
|
||||
|
||||
The complete BPF program is small because buffer management belongs to the AF_XDP side.
|
||||
Now the XDP program itself:
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: GPL-2.0
|
||||
@@ -99,13 +113,19 @@ int redirect_udp(struct xdp_md *ctx)
|
||||
}
|
||||
```
|
||||
|
||||
Every header access is preceded by a bounds or length check. The program accepts Ethernet IPv4 packets, excludes fragments, follows the IPv4 header length to UDP, and verifies that the UDP length fits inside the IP payload. Only the configured destination port reaches the redirect path.
|
||||
The program parses packets layer by layer, validating boundaries at each step. This careful bounds checking is required by the BPF verifier and also ensures we don't misinterpret truncated or malformed packets.
|
||||
|
||||
The XSKMAP lookup is also part of correctness. A queue with no socket entry returns `XDP_PASS`, as do unrelated or malformed packets. `bpf_redirect_map()` uses the same pass action as its fallback. Once a redirect succeeds, the packet is consumed by AF_XDP rather than mirrored; a regular UDP socket will not receive that selected packet.
|
||||
The parsing starts with Ethernet, checking that there's room for the header and that the EtherType indicates IPv4. Then it validates the IPv4 header: correct version, UDP protocol, minimum header length, and no fragmentation (fragmented packets would need reassembly, which is beyond our scope). The program computes the actual IP header length from the IHL field and uses it to locate the UDP header.
|
||||
|
||||
## Building an AF_XDP Socket from the ABI
|
||||
The UDP validation ensures the length field is sane and that the destination port matches our target. Only then does the program check if there's actually a socket registered for this queue. This ordering is deliberate: most packets will fail earlier checks, so we avoid the map lookup cost for packets we won't capture anyway.
|
||||
|
||||
The user-space side below performs the UMEM registration, maps the rings, binds the socket, loads XDP, and runs the receive loop.
|
||||
The XSKMAP lookup is also a safety check. If userspace hasn't registered a socket for this queue, the lookup returns NULL and we pass the packet to the normal stack. When everything checks out, `bpf_redirect_map()` sends the packet to AF_XDP. The second argument is the queue index, which becomes the map key. The third argument is the fallback action if something goes wrong.
|
||||
|
||||
One important detail: once a packet is redirected, it's consumed by AF_XDP. The regular socket stack will never see it. This is exactly what we want for a capture tool, but it means you need to be careful about what you redirect.
|
||||
|
||||
## The Userspace Application
|
||||
|
||||
The userspace code handles everything AF_XDP needs: memory allocation, ring setup, socket binding, XDP loading, and the receive loop. Here's the complete implementation:
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
|
||||
@@ -599,15 +619,43 @@ cleanup:
|
||||
}
|
||||
```
|
||||
|
||||
`open_xsk()` creates a 256 KiB UMEM area, registers it through `XDP_UMEM_REG`, and asks the kernel for 64 fill, completion, and RX entries. `XDP_MMAP_OFFSETS` describes where each producer, consumer, flags, and descriptor array lives. The program maps those pages and posts all frame addresses before traffic can be redirected.
|
||||
### Understanding UMEM and Ring Setup
|
||||
|
||||
The ring indices are shared between kernel and user space, so their memory ordering matters. A producer writes a descriptor before publishing its new index with release semantics. A consumer acquires the producer index before reading descriptors. The code applies that pattern both while consuming RX entries and while returning addresses to the fill ring.
|
||||
The `open_xsk()` function creates the AF_XDP socket and all its supporting infrastructure. It starts by mapping 256 KiB of anonymous memory for UMEM, divided into 64 frames of 4096 bytes each. This memory will be shared with the kernel after registration.
|
||||
|
||||
`XDP_COPY` on the socket bind and XDP attach mode solve different problems. Copy mode tells AF_XDP how packet data enters UMEM and works without driver zero-copy support. The XDP program itself first tries native driver mode, then falls back to generic SKB mode when the interface rejects native XDP. `--skb-mode` selects the generic path directly.
|
||||
After creating the socket with `AF_XDP`, it registers the UMEM with `XDP_UMEM_REG`, telling the kernel where our packet buffers live. Then it requests fill, completion, and RX rings of 64 entries each with `setsockopt`. The `XDP_MMAP_OFFSETS` getsockopt reveals where each ring's producer index, consumer index, flags, and descriptor array are located within pages that can be mmap'd.
|
||||
|
||||
The receive loop expects one descriptor per packet. `XDP_PKT_CONTD` therefore produces `EMSGSIZE` instead of silently printing only the first fragment of a multi-buffer packet. Address and packet-length checks also keep every descriptor within the registered UMEM. On shutdown, compare-and-detach supplies the program FD as `old_prog_fd`, so the tool removes only the XDP program it attached.
|
||||
The ring mapping is tricky because the kernel lays out each ring at a fixed page offset. The Fill Ring lives at `XDP_UMEM_PGOFF_FILL_RING`, the Completion Ring at `XDP_UMEM_PGOFF_COMPLETION_RING`, and the RX Ring at `XDP_PGOFF_RX_RING`. Each ring contains pointers to the producer and consumer indices, which are the synchronization points between kernel and userspace.
|
||||
|
||||
## Build and Run
|
||||
The final step before binding is posting all 64 frame addresses to the Fill Ring. We write each address into the descriptor array and then publish the producer index with release semantics. This tells the kernel it has 64 frames available for receiving packets.
|
||||
|
||||
### Memory Ordering in Ring Operations
|
||||
|
||||
Ring indices are shared between kernel and userspace, which makes memory ordering critical. The pattern is consistent throughout: a producer writes descriptors before publishing its index with release semantics, and a consumer acquires the producer index before reading descriptors.
|
||||
|
||||
In `recycle_frame()`, we read our own producer index (which only we modify) with relaxed ordering, but we acquire the consumer index because the kernel writes it. If there's room in the ring, we write the address and publish with release. The kernel will eventually acquire our producer update and see the address we wrote.
|
||||
|
||||
The same pattern appears in the receive loop. We acquire the producer index written by the kernel, read descriptors, and publish our consumer update with release. This ensures the kernel knows we're done with those frames before it reuses them.
|
||||
|
||||
### Copy Mode vs Zero-Copy
|
||||
|
||||
The `XDP_COPY` flag in the socket address tells AF_XDP to copy packets into UMEM rather than doing true zero-copy. Copy mode works on any interface without driver support, making it the right choice for a learning example. The kernel allocates its own memory for incoming packets and copies them into our UMEM frames.
|
||||
|
||||
The XDP program attach mode is different. We first try native driver mode (`XDP_FLAGS_DRV_MODE`), which runs the XDP program in the driver before SKB allocation. If that fails because the driver doesn't support XDP, we fall back to generic SKB mode (`XDP_FLAGS_SKB_MODE`), which runs after the SKB is created but still lets us redirect to AF_XDP. The `--skb-mode` flag forces generic mode directly.
|
||||
|
||||
### The Receive Loop
|
||||
|
||||
The receive loop polls for packets, processes them in batches, and recycles frames immediately. The 250ms timeout ensures we can respond to signals even when no packets arrive.
|
||||
|
||||
When poll indicates data, we check the RX ring for new descriptors. For each one, we compute the actual data address (the descriptor address might be encoded with offset information), verify it's within UMEM bounds, and check that this isn't a multi-buffer packet (which would have `XDP_PKT_CONTD` set). Multi-buffer support would require accumulating fragments, which is beyond this example's scope.
|
||||
|
||||
After printing the packet, we immediately return the frame to the Fill Ring via `recycle_frame()`. This is critical: without recycling, we'd run out of frames after 64 packets. The tool proves recycling works by successfully capturing 65 packets.
|
||||
|
||||
### Cleanup and Safe Detach
|
||||
|
||||
On exit, we use compare-and-detach to remove only our XDP program. The `old_prog_fd` option tells `bpf_xdp_detach` to only detach if the currently attached program matches ours. This prevents accidentally detaching someone else's XDP program if they attached one while we were running.
|
||||
|
||||
## Compilation and Execution
|
||||
|
||||
Build the executable:
|
||||
|
||||
@@ -616,48 +664,65 @@ cd src/60-afxdp-dump
|
||||
make
|
||||
```
|
||||
|
||||
Capture five UDP packets arriving on queue 0 and destination port 8080:
|
||||
Capture five UDP packets arriving on queue 0 with destination port 8080:
|
||||
|
||||
```bash
|
||||
sudo ./afxdp_dump --interface eth0 --queue 0 --port 8080 --count 5
|
||||
```
|
||||
|
||||
`--count 0` runs until a signal. Add `--skb-mode` for generic XDP. The selected queue must receive the traffic; on a multi-queue interface this depends on the NIC's receive-side steering configuration.
|
||||
Use `--count 0` to run until interrupted with Ctrl+C. Add `--skb-mode` to force generic XDP mode. The selected queue must actually receive the traffic; on multi-queue NICs, this depends on RSS (receive-side scaling) configuration.
|
||||
|
||||
A longer run with `--count 65` makes frame recycling visible. One real run produced:
|
||||
Send test traffic from another machine or terminal:
|
||||
|
||||
```bash
|
||||
echo "hello-afxdp" | nc -u target-ip 8080
|
||||
```
|
||||
|
||||
A longer run with `--count 65` demonstrates that frame recycling works:
|
||||
|
||||
```text
|
||||
afxdp-dump ready interface=axdp1267r queue=0 port=8080 mode=driver count=65
|
||||
afxdp-dump ready interface=eth0 queue=0 port=8080 mode=driver count=65
|
||||
packet=1 10.77.0.1:60414 -> 10.77.0.2:8080 bytes=53 payload="hello-afxdp"
|
||||
packet=65 10.77.0.1:60414 -> 10.77.0.2:8080 bytes=53 payload="hello-afxdp"
|
||||
redirected=65
|
||||
```
|
||||
|
||||
`packet=65` demonstrates frame reuse beyond the 64 initially posted frames. `redirected=65` comes from the BPF-side counter. The XDP path returns `XDP_PASS` for other destination ports, so they remain on the normal network path.
|
||||
Packet 65 proves that at least one frame completed the full ownership cycle: posted to Fill, used for receive, consumed from RX, and posted back to Fill. The `redirected=65` counter comes from the BPF program and matches our receive count.
|
||||
|
||||
## Requirements
|
||||
|
||||
| Requirement | Details |
|
||||
|---|---|
|
||||
| Kernel | Linux 5.7 or newer; AF_XDP arrived in 4.18, XSKMAP lookup from XDP in 5.3, and safe expected-FD detach in 5.7 |
|
||||
| Kernel | Linux 5.7 or newer. AF_XDP arrived in 4.18, XSKMAP lookup from XDP in 5.3, and safe expected-FD detach in 5.7 |
|
||||
| Kernel config | `CONFIG_BPF`, `CONFIG_BPF_SYSCALL`, `CONFIG_BPF_JIT`, `CONFIG_XDP_SOCKETS`, `CONFIG_DEBUG_INFO_BTF` |
|
||||
| Privileges | Root, or equivalent BPF and network-administration capabilities |
|
||||
| Interface | An interface with the selected RX queue; native XDP is optional because generic XDP is available |
|
||||
| Architecture and hardware | x86-64 is the declared and tested target; copy mode needs no AF_XDP zero-copy driver support |
|
||||
| Privileges | Root or equivalent BPF and network-admin capabilities |
|
||||
| Interface | Any interface with the selected RX queue. Native XDP is optional since generic mode works everywhere |
|
||||
| Architecture | x86-64 is tested. Copy mode works without driver zero-copy support |
|
||||
|
||||
## Scope
|
||||
## What's Next
|
||||
|
||||
`afxdp-dump` is a receive-only, single-queue, single-buffer IPv4 UDP tool. It uses a fixed 64-frame UMEM and prints at most 32 payload characters. It provides the buffer lifecycle needed for a useful packet receiver while keeping TX rings, shared UMEM, zero-copy setup, multi-buffer reconstruction, and RX metadata for later examples.
|
||||
This example is deliberately minimal: receive-only, single-queue, single-buffer packets, copy mode. Real production AF_XDP applications extend this foundation in several directions:
|
||||
|
||||
**TX rings** let you send packets with the same zero-overhead model. You'd add a TX ring, post frame addresses with packet data, and poll for completion notifications.
|
||||
|
||||
**Zero-copy mode** eliminates the copy into UMEM. The driver uses your UMEM directly, but this requires driver support and careful buffer alignment.
|
||||
|
||||
**Multi-buffer packets** handle jumbo frames or when UMEM frames are smaller than the MTU. You'd accumulate fragments marked with `XDP_PKT_CONTD` until the final fragment.
|
||||
|
||||
**Shared UMEM** lets multiple sockets share the same memory, useful for load-balancing across queues or between RX and TX paths.
|
||||
|
||||
## Summary
|
||||
|
||||
This example exposes the complete AF_XDP receive contract. XDP selects one UDP flow, XSKMAP resolves the RX queue to a socket, the kernel publishes a UMEM descriptor, and user space returns the frame after inspection. The 65-packet run closes the loop by proving that ownership really comes back to the fill ring.
|
||||
AF_XDP gives you kernel-bypass packet reception with eBPF's safety guarantees. The XDP program selects traffic at the driver boundary, the XSKMAP routes packets to your socket, and lock-free ring buffers transfer data without system calls. This example showed the complete receive contract: post frames to Fill, receive descriptors on RX, process packets, recycle frames back to Fill.
|
||||
|
||||
Understanding this flow is valuable beyond packet capture. The patterns here, shared memory between kernel and userspace, explicit ownership transfer, lock-free synchronization, appear throughout high-performance systems from databases to GPU drivers.
|
||||
|
||||
> If you'd like to dive deeper into eBPF, check out our tutorial repository at <https://github.com/eunomia-bpf/bpf-developer-tutorial> or visit our website at <https://eunomia.dev/tutorials/>.
|
||||
|
||||
## References
|
||||
|
||||
- [Linux AF_XDP documentation](https://docs.kernel.org/networking/af_xdp.html)
|
||||
- [AF_XDP introduction commit](https://github.com/torvalds/linux/commit/c0c77d8fb787cfe0c3fca689c2a30d1dad4eaba7)
|
||||
- [XSKMAP lookup from XDP commit](https://github.com/torvalds/linux/commit/fada7fdc83c0)
|
||||
- [Expected-program FD for XDP replacement and detach](https://github.com/torvalds/linux/commit/92234c8f15c8d96ad7e52afdc5994cba6be68eb9)
|
||||
- [Linux AF_XDP Documentation](https://docs.kernel.org/networking/af_xdp.html)
|
||||
- [AF_XDP Introduction Commit](https://github.com/torvalds/linux/commit/c0c77d8fb787cfe0c3fca689c2a30d1dad4eaba7)
|
||||
- [XSKMAP Lookup from XDP](https://github.com/torvalds/linux/commit/fada7fdc83c0)
|
||||
- [Expected-Program FD for XDP Detach](https://github.com/torvalds/linux/commit/92234c8f15c8d96ad7e52afdc5994cba6be68eb9)
|
||||
- [libxdp Library](https://github.com/xdp-project/xdp-tools) - Higher-level AF_XDP helpers if you want to skip the raw ABI
|
||||
|
||||
@@ -1,24 +1,40 @@
|
||||
# eBPF 实战教程：用 AF_XDP 在用户态接收 UDP 报文
|
||||
# eBPF 实战教程：用 AF_XDP 实现高性能 UDP 抓包
|
||||
|
||||
有些报文需要在进入普通 socket stack 之前交给用户态程序。抓包工具、协议原型或专用转发器通常需要一条清晰的接收路径：在驱动入口筛选流量，把报文放入共享内存，完成检查，再把 buffer 交还给下一次接收。
|
||||
你有没有好奇过 Suricata 这样的抓包工具或者高频交易系统是如何每秒接收数百万个报文而不被内核开销拖垮的？秘密在于绕过大部分网络协议栈。AF_XDP 让你在网卡驱动入口拦截报文，直接复制到应用程序的内存空间，处理时几乎不需要为每个报文做系统调用。
|
||||
|
||||
本课直接使用 AF_XDP ABI 构建这条路径，不用库隐藏 ring 和 UMEM 的细节。最终得到的 `afxdp-dump` 会捕获一个队列上发往指定端口的 IPv4 UDP 报文，打印 payload 预览并循环使用每个 frame。它采用单队列、single-buffer packet 和 copy mode，适合作为 zero-copy 与 multi-buffer 之前的第一个 AF_XDP 例子。
|
||||
本教程从零开始，使用原始的 AF_XDP 接口构建一个完整的抓包工具。没有辅助库，没有魔法抽象。你会清楚看到 UMEM 注册、ring buffer 和 XDP 重定向是如何协同工作的。最终成果是 `afxdp-dump`，一个捕获指定端口 IPv4 UDP 报文、打印 payload 预览、并正确回收每个 frame 以持续接收的工具。
|
||||
|
||||
> 完整源代码：<https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/60-afxdp-dump>
|
||||
|
||||
## XDP 选择报文，AF_XDP 完成交付
|
||||
## 为什么选择 AF_XDP？
|
||||
|
||||
eBPF 可以让经过验证器检查的程序运行在内核 hook 上，XDP 则把其中一个 hook 放在 Linux 网络接收路径的最前端。Linux 4.18 引入的 AF_XDP 把 XDP redirect 连接到由用户态注册内存支撑的 socket，XSKMAP 再建立 RX queue number 与对应 AF_XDP socket 之间的关系。
|
||||
传统的 `libpcap` 或 raw socket 抓包有一个根本问题：每个报文都要通过昂贵的系统调用跨越内核与用户态的边界。当你抓取 10 Gbps 流量时，这些开销会成为瓶颈，而不是你的处理代码。
|
||||
|
||||
两边的职责很明确。XDP 程序只解析足够的信息，判断报文是否属于这个工具；AF_XDP socket 管理共享内存 ring，并把选中的字节交给用户态。本例还会在 redirect 之前用 `bpf_map_lookup_elem()` 检查 XSKMAP，这项能力从 Linux 5.3 开始可用。清理阶段的 compare-and-detach 使用 Linux 5.7 加入的 expected-program FD，因此完整工具的最低内核版本是 5.7。
|
||||
AF_XDP 通过在内核和用户态之间建立共享内存来解决这个问题。内核把报文直接写入你的应用程序可以读取的内存，双方通过无锁的 ring buffer 通信，而不是系统调用。一次 `poll()` 就能唤醒你处理数百个报文。这种架构在普通硬件上就能达到每秒数百万报文的接收速率。
|
||||
|
||||
跟着一个报文和一个 frame 走一遍。用户态在 UMEM 中分配 64 个 4096 字节 frame，把它们的地址发布到 fill ring。发往指定端口的 UDP 报文到达 XDP hook 后，程序检查 Ethernet、IPv4 与 UDP 长度，通过 `ctx->rx_queue_index` 找到注册 socket，再返回 `XDP_REDIRECT`。copy mode 下，内核把报文复制到一个已发布的 frame，并把 `xdp_desc` 放入 RX ring。用户态读取 descriptor、打印报文、推进 consumer index，最后把同一个地址放回 fill ring。
|
||||
这项技术已经在生产环境中使用。Meta 的负载均衡器运行 AF_XDP，Cilium 用它做 Kubernetes 网络，高频交易公司用它来缩短微秒级延迟。即使你永远不会构建交易系统，理解 AF_XDP 也能教会你高性能系统中反复出现的模式：共享内存、无锁数据结构、显式的所有权转移。
|
||||
|
||||
最后一步决定了接收循环能否持续。如果 frame 没有回收，最初的 64 个地址会在 64 个报文之后耗尽。一次接收 65 个报文，就可以用第 65 个报文证明至少有一个 frame 已经走完完整的所有权循环。
|
||||
## AF_XDP 架构
|
||||
|
||||
## 共享的队列上限
|
||||
AF_XDP 通过四个必须精确协调的组件工作：
|
||||
|
||||
共享头文件定义 XSKMAP 容量。queue ID 是 map key，因此例子可以表示 0 到 63 号队列，每个进程绑定其中一个。
|
||||
**UMEM（User Memory）** 是你分配的、内核和用户态都能访问的内存区域。你把它划分为固定大小的 frame，通常每个 4096 字节。内核送达的每个报文都会放进这些 frame 之一。
|
||||
|
||||
**Fill Ring** 是你告诉内核哪些 frame 可用于接收报文的方式。你把 frame 地址发布到这里，内核需要存放报文时就会消费这些地址。
|
||||
|
||||
**RX Ring** 是内核通知你已接收报文的地方。每个 entry 包含 frame 地址和报文长度。当你在这里看到 entry 时，你就拥有那个 frame 的所有权，直到你归还它。
|
||||
|
||||
**XSKMAP** 是一个 BPF map，用于连接 XDP 程序和 AF_XDP socket。XDP 程序决定重定向哪些报文，在这个 map 中查找当前 RX queue 对应的 socket，然后调用 `bpf_redirect_map()` 来投递报文。
|
||||
|
||||
流程是这样的：你向 Fill Ring 发布 64 个 frame 地址。一个 UDP 报文到达。你的 XDP 程序检查目的端口，在 XSKMAP 中查找 socket，然后重定向。内核把报文复制到你的某个 frame 中，并在 RX Ring 上发布一个 descriptor。你读取 descriptor、处理报文，再把这个 frame 地址放回 Fill Ring。循环继续。
|
||||
|
||||
这个所有权模型至关重要。一个 frame 开始时属于你。你通过 Fill Ring 借给内核。内核借用它来接收报文。你从 RX Ring 回收它。你必须把它归还到 Fill Ring，否则 64 个报文之后就会耗尽 frame。我们的工具通过成功捕获 65 个报文来证明这个机制有效——这需要至少一个 frame 完成完整的所有权周期。
|
||||
|
||||
## XDP 程序：过滤和重定向
|
||||
|
||||
内核侧的 BPF 程序很精简，因为它只负责过滤和重定向。buffer 管理的所有复杂性都在用户态。
|
||||
|
||||
首先，我们定义一个共享头文件来设置 XSKMAP 容量。queue ID 是 map 的 key，所以这个例子可以寻址 0 到 63 号队列：
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -30,9 +46,7 @@ eBPF 可以让经过验证器检查的程序运行在内核 hook 上，XDP 则
|
||||
#endif /* __AFXDP_DUMP_H */
|
||||
```
|
||||
|
||||
## 在 XDP 中筛选并重定向
|
||||
|
||||
完整 BPF 程序很短，因为 buffer 管理属于 AF_XDP 一侧。
|
||||
现在是 XDP 程序本身：
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: GPL-2.0
|
||||
@@ -99,13 +113,19 @@ int redirect_udp(struct xdp_md *ctx)
|
||||
}
|
||||
```
|
||||
|
||||
程序在每次 header 访问之前都检查边界或协议长度。它接收 Ethernet IPv4 报文，排除分片，根据 IPv4 header length 找到 UDP，再确认 UDP length 位于 IP payload 内。只有指定目的端口会进入 redirect 路径。
|
||||
程序逐层解析报文，在每一步都验证边界。这种仔细的边界检查是 BPF 验证器的要求，同时也确保我们不会误解截断或畸形的报文。
|
||||
|
||||
XSKMAP lookup 也是正确性的一部分。队列没有 socket entry 时返回 `XDP_PASS`，无关报文和格式异常的报文同样沿普通网络路径继续；`bpf_redirect_map()` 的 fallback action 也是 pass。一旦 redirect 成功，报文会由 AF_XDP 消费，而不是复制一份镜像，因此普通 UDP socket 不会再收到这条被选中的报文。
|
||||
解析从 Ethernet 开始，检查是否有足够空间容纳头部，以及 EtherType 是否表示 IPv4。然后验证 IPv4 头部：正确的版本、UDP 协议、最小头部长度、没有分片（分片报文需要重组，超出了我们的范围）。程序从 IHL 字段计算实际的 IP 头部长度，并用它来定位 UDP 头部。
|
||||
|
||||
## 直接使用 ABI 构建 AF_XDP socket
|
||||
UDP 验证确保长度字段合理，目的端口与目标匹配。只有这时程序才检查这个 queue 是否真的注册了 socket。这个顺序是刻意的：大多数报文会在更早的检查中失败，所以对不会捕获的报文我们避免了 map lookup 的开销。
|
||||
|
||||
下面的用户态程序完成 UMEM 注册、ring 映射、socket bind、XDP 加载和接收循环。
|
||||
XSKMAP lookup 也是一个安全检查。如果用户态没有为这个 queue 注册 socket，lookup 返回 NULL，我们就把报文传递给普通协议栈。当所有检查都通过时，`bpf_redirect_map()` 把报文发送到 AF_XDP。第二个参数是 queue index，会成为 map 的 key。第三个参数是出错时的回退 action。
|
||||
|
||||
一个重要细节：一旦报文被重定向，它就被 AF_XDP 消费了。普通的 socket 协议栈永远不会看到它。这正是抓包工具需要的，但这意味着你需要小心选择重定向什么。
|
||||
|
||||
## 用户态应用程序
|
||||
|
||||
用户态代码处理 AF_XDP 需要的一切：内存分配、ring 设置、socket 绑定、XDP 加载和接收循环。以下是完整实现：
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
|
||||
@@ -599,13 +619,41 @@ cleanup:
|
||||
}
|
||||
```
|
||||
|
||||
`open_xsk()` 创建 256 KiB UMEM，通过 `XDP_UMEM_REG` 注册内存，再向内核申请各 64 个 fill、completion 和 RX entry。`XDP_MMAP_OFFSETS` 描述每个 producer、consumer、flags 和 descriptor array 的位置，程序据此映射页面，并在 redirect 开始前发布全部 frame 地址。
|
||||
### 理解 UMEM 和 Ring 设置
|
||||
|
||||
ring index 由内核和用户态共享，因此内存顺序很关键。producer 需要先写 descriptor，再用 release 语义发布新 index；consumer 则先 acquire producer index，再读取 descriptor。代码在消费 RX entry 和向 fill ring 归还地址时都遵循这组规则。
|
||||
`open_xsk()` 函数创建 AF_XDP socket 及其所有支撑基础设施。它首先映射 256 KiB 的匿名内存作为 UMEM，划分为 64 个 4096 字节的 frame。这块内存在注册后会与内核共享。
|
||||
|
||||
socket bind 使用的 `XDP_COPY` 与 XDP attach mode 解决的是两个问题。copy mode 决定报文如何进入 UMEM，不依赖驱动的 zero-copy 支持；XDP 程序本身会先尝试 native driver mode，如果网卡不支持，再回退到 generic SKB mode。`--skb-mode` 可以直接选择 generic 路径。
|
||||
用 `AF_XDP` 创建 socket 后，它通过 `XDP_UMEM_REG` 注册 UMEM，告诉内核我们的报文 buffer 在哪里。然后用 `setsockopt` 请求各 64 个 entry 的 fill、completion 和 RX ring。`XDP_MMAP_OFFSETS` getsockopt 揭示了每个 ring 的 producer index、consumer index、flags 和 descriptor 数组在可 mmap 的页面中的位置。
|
||||
|
||||
接收循环要求一个 descriptor 对应一个完整报文。遇到 `XDP_PKT_CONTD` 时会返回 `EMSGSIZE`，避免把 multi-buffer packet 的第一个 fragment 当成完整报文打印。地址和长度检查还会保证 descriptor 始终落在已注册 UMEM 内。退出时 compare-and-detach 把当前 program FD 作为 `old_prog_fd`，工具只会移除自己挂载的 XDP 程序。
|
||||
Ring 映射有些技巧，因为内核把每个 ring 放在固定的页面偏移。Fill Ring 在 `XDP_UMEM_PGOFF_FILL_RING`，Completion Ring 在 `XDP_UMEM_PGOFF_COMPLETION_RING`，RX Ring 在 `XDP_PGOFF_RX_RING`。每个 ring 包含指向 producer 和 consumer index 的指针，它们是内核和用户态之间的同步点。
|
||||
|
||||
bind 之前的最后一步是把所有 64 个 frame 地址发布到 Fill Ring。我们把每个地址写入 descriptor 数组，然后用 release 语义发布 producer index。这告诉内核有 64 个 frame 可用于接收报文。
|
||||
|
||||
### Ring 操作中的内存顺序
|
||||
|
||||
Ring index 由内核和用户态共享，这使得内存顺序至关重要。模式是一致的：producer 在用 release 语义发布 index 之前先写 descriptor，consumer 在读取 descriptor 之前先 acquire producer index。
|
||||
|
||||
在 `recycle_frame()` 中，我们用 relaxed 顺序读取自己的 producer index（只有我们修改它），但要 acquire consumer index，因为内核会写它。如果 ring 有空间，我们写入地址并用 release 发布。内核最终会 acquire 我们的 producer 更新并看到我们写的地址。
|
||||
|
||||
同样的模式出现在接收循环中。我们 acquire 内核写的 producer index，读取 descriptor，用 release 发布我们的 consumer 更新。这确保内核知道我们已经完成这些 frame，然后才能复用它们。
|
||||
|
||||
### Copy Mode 与 Zero-Copy
|
||||
|
||||
Socket 地址中的 `XDP_COPY` 标志告诉 AF_XDP 把报文复制到 UMEM，而不是真正的 zero-copy。Copy mode 在任何接口上都能工作，不需要驱动支持，是学习示例的正确选择。内核为传入报文分配自己的内存，然后复制到我们的 UMEM frame。
|
||||
|
||||
XDP 程序的 attach mode 是另一回事。我们首先尝试 native driver mode（`XDP_FLAGS_DRV_MODE`），它在 SKB 分配之前在驱动中运行 XDP 程序。如果因为驱动不支持 XDP 而失败，我们回退到 generic SKB mode（`XDP_FLAGS_SKB_MODE`），它在 SKB 创建之后运行，但仍然让我们重定向到 AF_XDP。`--skb-mode` 标志直接强制使用 generic mode。
|
||||
|
||||
### 接收循环
|
||||
|
||||
接收循环轮询报文，批量处理它们，并立即回收 frame。250ms 超时确保即使没有报文到达，我们也能响应信号。
|
||||
|
||||
当 poll 指示有数据时，我们检查 RX ring 是否有新 descriptor。对于每一个，我们计算实际的数据地址（descriptor 地址可能编码了 offset 信息），验证它在 UMEM 边界内，并检查这不是一个 multi-buffer 报文（那会设置 `XDP_PKT_CONTD`）。Multi-buffer 支持需要累积分片，超出了这个例子的范围。
|
||||
|
||||
打印报文后，我们立即通过 `recycle_frame()` 把 frame 归还到 Fill Ring。这至关重要：不回收的话，64 个报文之后就会耗尽 frame。工具通过成功捕获 65 个报文来证明回收有效。
|
||||
|
||||
### 清理和安全卸载
|
||||
|
||||
退出时，我们使用 compare-and-detach 只移除我们自己的 XDP 程序。`old_prog_fd` 选项告诉 `bpf_xdp_detach` 只在当前挂载的程序与我们的匹配时才卸载。这防止了在我们运行期间如果有人挂载了其他 XDP 程序时，意外卸载别人的程序。
|
||||
|
||||
## 编译和运行
|
||||
|
||||
@@ -622,36 +670,52 @@ make
|
||||
sudo ./afxdp_dump --interface eth0 --queue 0 --port 8080 --count 5
|
||||
```
|
||||
|
||||
`--count 0` 会持续运行到收到信号，`--skb-mode` 用于 generic XDP。选中的 queue 必须实际接收这条流量，在多队列网卡上，这取决于 NIC 的 receive-side steering 配置。
|
||||
使用 `--count 0` 持续运行直到 Ctrl+C 中断。添加 `--skb-mode` 强制使用 generic XDP mode。选中的 queue 必须实际接收到流量；在多队列网卡上，这取决于 RSS（receive-side scaling）配置。
|
||||
|
||||
使用 `--count 65` 运行更长时间，可以直接看到 frame 已经完成复用。一次真实运行的输出如下：
|
||||
从另一台机器或另一个终端发送测试流量：
|
||||
|
||||
```bash
|
||||
echo "hello-afxdp" | nc -u target-ip 8080
|
||||
```
|
||||
|
||||
使用 `--count 65` 运行更长时间可以证明 frame 回收有效：
|
||||
|
||||
```text
|
||||
afxdp-dump ready interface=axdp1267r queue=0 port=8080 mode=driver count=65
|
||||
afxdp-dump ready interface=eth0 queue=0 port=8080 mode=driver count=65
|
||||
packet=1 10.77.0.1:60414 -> 10.77.0.2:8080 bytes=53 payload="hello-afxdp"
|
||||
packet=65 10.77.0.1:60414 -> 10.77.0.2:8080 bytes=53 payload="hello-afxdp"
|
||||
redirected=65
|
||||
```
|
||||
|
||||
`packet=65` 证明 frame 已经在最初的 64 个地址之外完成复用，`redirected=65` 来自 BPF 侧计数器。XDP 路径会对其他目的端口返回 `XDP_PASS`，让这些流量继续走普通网络路径。
|
||||
Packet 65 证明至少有一个 frame 完成了完整的所有权周期：发布到 Fill，用于接收，从 RX 消费，再发布回 Fill。`redirected=65` 计数器来自 BPF 程序，与我们的接收计数一致。
|
||||
|
||||
## 环境要求
|
||||
|
||||
| 要求 | 说明 |
|
||||
|---|---|
|
||||
| 内核 | Linux 5.7 或更高版本，AF_XDP 在 4.18 引入，XDP 对 XSKMAP 的 lookup 在 5.3 引入，安全的 expected-FD detach 在 5.7 引入 |
|
||||
| 内核 | Linux 5.7 或更高版本。AF_XDP 在 4.18 引入，XDP 对 XSKMAP 的 lookup 在 5.3 引入，安全的 expected-FD detach 在 5.7 引入 |
|
||||
| 内核配置 | `CONFIG_BPF`、`CONFIG_BPF_SYSCALL`、`CONFIG_BPF_JIT`、`CONFIG_XDP_SOCKETS`、`CONFIG_DEBUG_INFO_BTF` |
|
||||
| 权限 | root，或者等价的 BPF 与网络管理 capability |
|
||||
| 网络接口 | 接口包含所选 RX queue，native XDP 可选，工具也支持 generic XDP |
|
||||
| 架构与硬件 | 当前声明并完成测试的目标是 x86-64，copy mode 不要求驱动支持 AF_XDP zero-copy |
|
||||
| 权限 | root 或等价的 BPF 与网络管理 capability |
|
||||
| 网络接口 | 任何包含所选 RX queue 的接口。native XDP 可选，generic mode 处处可用 |
|
||||
| 架构 | x86-64 已测试。copy mode 不需要驱动 zero-copy 支持 |
|
||||
|
||||
## 实现范围
|
||||
## 后续扩展
|
||||
|
||||
`afxdp-dump` 是一个 receive-only、单队列、single-buffer 的 IPv4 UDP 工具，使用固定的 64-frame UMEM，最多打印 32 个 payload 字符。它保留了一个实用接收器需要的 buffer 生命周期，TX ring、shared UMEM、zero-copy、multi-buffer 重组和 RX metadata 可以在后续例子中继续展开。
|
||||
这个例子有意做得最小化：只接收、单队列、single-buffer 报文、copy mode。真正的生产 AF_XDP 应用会在这个基础上向多个方向扩展：
|
||||
|
||||
**TX ring** 让你用同样的零开销模型发送报文。你需要添加 TX ring，发布带有报文数据的 frame 地址，然后轮询 completion 通知。
|
||||
|
||||
**Zero-copy mode** 消除了复制到 UMEM 的过程。驱动直接使用你的 UMEM，但这需要驱动支持和仔细的 buffer 对齐。
|
||||
|
||||
**Multi-buffer 报文** 处理巨型帧或 UMEM frame 小于 MTU 的情况。你需要累积标记了 `XDP_PKT_CONTD` 的分片，直到最后一个分片。
|
||||
|
||||
**Shared UMEM** 让多个 socket 共享同一块内存，对于跨 queue 负载均衡或 RX 和 TX 路径之间共享很有用。
|
||||
|
||||
## 总结
|
||||
|
||||
这个例子展示了完整的 AF_XDP 接收契约：XDP 选择一条 UDP 流，XSKMAP 把 RX queue 解析到 socket，内核发布 UMEM descriptor，用户态检查完成后归还 frame。一次接收 65 个报文，进一步证明所有权确实回到了 fill ring。
|
||||
AF_XDP 给你提供了具有 eBPF 安全保证的内核旁路报文接收。XDP 程序在驱动边界选择流量，XSKMAP 把报文路由到你的 socket，无锁的 ring buffer 不用系统调用就能传输数据。这个例子展示了完整的接收契约：向 Fill 发布 frame，在 RX 上接收 descriptor，处理报文，把 frame 回收到 Fill。
|
||||
|
||||
理解这个流程的价值超越了抓包本身。这里的模式——内核与用户态之间的共享内存、显式的所有权转移、无锁同步——在从数据库到 GPU 驱动的各种高性能系统中反复出现。
|
||||
|
||||
> 如果你想深入了解 eBPF，请查看我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或访问我们的网站 <https://eunomia.dev/tutorials/>。
|
||||
|
||||
@@ -660,4 +724,5 @@ redirected=65
|
||||
- [Linux AF_XDP 文档](https://docs.kernel.org/networking/af_xdp.html)
|
||||
- [AF_XDP 引入 commit](https://github.com/torvalds/linux/commit/c0c77d8fb787cfe0c3fca689c2a30d1dad4eaba7)
|
||||
- [XDP 支持 XSKMAP lookup 的 commit](https://github.com/torvalds/linux/commit/fada7fdc83c0)
|
||||
- [XDP replace 与 detach 的 expected-program FD](https://github.com/torvalds/linux/commit/92234c8f15c8d96ad7e52afdc5994cba6be68eb9)
|
||||
- [XDP detach 的 expected-program FD](https://github.com/torvalds/linux/commit/92234c8f15c8d96ad7e52afdc5994cba6be68eb9)
|
||||
- [libxdp 库](https://github.com/xdp-project/xdp-tools) - 如果想跳过原始 ABI，这里有更高层的 AF_XDP 辅助函数
|
||||
|
||||
Reference in New Issue
Block a user