diff --git a/README.md b/README.md index 14479f1..28b5bf7 100644 --- a/README.md +++ b/README.md @@ -77,6 +77,7 @@ Networking: - [lesson 42-xdp-loadbalancer](src/42-xdp-loadbalancer/README.md) XDP Load Balancer - [lesson 46-xdp-test](src/46-xdp-test/README.md) Building a High-Performance XDP Packet Generator - [lesson 50-tcx](src/50-tcx/README.md) Composable Traffic Control with TCX Links +- [lesson 53-egress-pacer](src/53-egress-pacer/README.md) Pace Egress Traffic with a BPF Qdisc Tracing: diff --git a/README.zh.md b/README.zh.md index 1b1e580..4dbd3f2 100644 --- a/README.zh.md +++ b/README.zh.md @@ -76,6 +76,7 @@ GPU: - [lesson 42-xdp-loadbalancer](src/42-xdp-loadbalancer/README.zh.md) eBPF 开发者教程： 简单的 XDP 负载均衡器 - [lesson 46-xdp-test](src/46-xdp-test/README.zh.md) eBPF 实例教程：构建高性能 XDP 数据包生成器 - [lesson 50-tcx](src/50-tcx/README.zh.md) eBPF 入门实践教程第五十篇：使用 TCX Link 实现可组合的流量控制 +- [lesson 53-egress-pacer](src/53-egress-pacer/README.zh.md) 使用 BPF Qdisc 对出口流量进行限速 追踪: diff --git a/src/53-egress-pacer/README.md b/src/53-egress-pacer/README.md new file mode 100644 index 0000000..7f986b5 --- /dev/null +++ b/src/53-egress-pacer/README.md @@ -0,0 +1,272 @@ +# eBPF Tutorial by Example: Pace Egress Traffic with a BPF Qdisc + +You have a lab, a test bench, or a service running behind a dedicated veth or TAP interface. Something sends a burst, the queue grows unbounded, latency spikes, and you have no visibility into the drop. You want a short-lived egress pacer that installs instantly, reports queue and byte counters, and removes itself on exit or signal. + +This tutorial builds a single-purpose BPF qdisc that does exactly that. It uses the Linux 6.16 `struct_ops`-based BPF qdisc infrastructure to register a bounded FIFO, pace packets at an aggregate rate you choose, and expose enqueue/dequeue/drop statistics in user space. + +This is not production-ready for a general-purpose NIC. It is intentionally designed for a controlled interface (veth, TAP, IFB) where you make an explicit lifecycle decision. The loader refuses to replace any existing root qdisc. + +## Quick start + +Build without root: + +```bash +cd src/53-egress-pacer +make clean +make -j2 +``` + +Run on a dedicated interface: + +```bash +sudo ./egress_pacer --interface veth-service --rate-kbps 64000 --queue-limit 256 --duration 30 +``` + +The tool prints a `READY` line when the qdisc is active, then a `SUMMARY` line with final counters when it exits: + +```text +READY interface=veth-service rate_kbps=64000 queue_limit=256 duration=30 +SUMMARY enqueued=... dequeued=... policy_dropped=... cleanup_dropped=... bytes_dequeued=... max_qlen=... +``` + +Usage: + +```text +Usage: ./egress_pacer --interface IFACE [--rate-kbps KBPS] [--queue-limit PACKETS] [--duration SEC] [--verbose] +``` + +| Option | Range | Default | +|--------|-------|---------| +| `--interface` | required | none | +| `--rate-kbps` | 8 to 100000000 | 1024 | +| `--queue-limit` | 1 to 65535 packets | 256 | +| `--duration` | 1 to 86400 seconds | 10 | +| `--verbose` | flag | off | + +If the interface already has a root qdisc, the tool exits with: + +```text +refusing to replace the existing root qdisc on IFACE +``` + +## Verified test output + +The repository includes a deterministic integration fixture. This output was captured in a KVM guest running kernel `7.0.0-rc2+` after a clean host build: + +```console +READY interface=epac_tx rate_kbps=64 queue_limit=8 duration=2 +SUMMARY enqueued=9 dequeued=9 policy_dropped=31 cleanup_dropped=0 bytes_dequeued=9216 max_qlen=8 +TEST-SUMMARY attempts=40 received=9 send_errors=31 span_ms=1024 observed_kbps=64.0 +PASS: conflict refusal, bounded drops, pacing, accounting, and normal/signal cleanup succeeded +``` + +This is a functional smoke test, not benchmark-quality performance evidence. 40 raw 1024-byte EtherType `0x88B5` frames are sent in a burst through a 64 Kbit/s pacer with an 8-packet queue. 31 are dropped at the queue limit; 9 are paced and delivered. Do not generalize these numbers beyond this fixture. + +Run the fixture yourself on a suitable disposable Linux 6.16+ system: + +```bash +sudo python3 tests/test_egress_pacer.py ./egress_pacer +``` + +The fixture creates and deletes `epac_tx`/`epac_rx` veth interfaces, installs a conflicting `pfifo` first to prove refusal, then sends the burst. It asserts bounded drops, byte/packet accounting, pacing, normal cleanup, SIGTERM cleanup, and deletes the veth pair in a `finally` block. + +## How it works + +The tool has two parts: a BPF object that implements `struct Qdisc_ops`, and a user-space loader that manages the qdisc lifecycle. + +### BPF qdisc: enqueue + +When a packet arrives at the qdisc, the enqueue callback checks the queue length against `sch->limit`. If the queue is full, the packet is dropped and `policy_dropped` increments. Otherwise: + +1. Allocate a `packet_node` with `bpf_obj_new`. +2. Transfer the skb into the node via `bpf_kptr_xchg`. +3. Compute the packet's eligible departure timestamp: `eligible_ns = max(next_departure_ns, now)`. Then advance `next_departure_ns` by the packet's serialization interval `(packet_len * 8 * 1,000,000 / rate_kbps)` nanoseconds. +4. Push the node onto a BPF linked list and update qlen/backlog. + +From the BPF source (`egress_pacer.bpf.c`): + +```c +SEC("struct_ops/egress_pacer_enqueue") +int BPF_PROG(egress_pacer_enqueue, struct sk_buff *skb, struct Qdisc *sch, + struct bpf_sk_buff_ptr *to_free) +{ + struct packet_node *packet; + __u64 eligible_ns, interval_ns, now; + __u32 packet_len; + + packet_len = qdisc_packet_len(skb); + if (sch->q.qlen >= sch->limit) + goto drop; + + packet = bpf_obj_new(typeof(*packet)); + if (!packet) + goto drop; + + now = bpf_ktime_get_ns(); + interval_ns = (__u64)packet_len * 8 * 1000000ULL / rate_kbps; + if (!interval_ns) + interval_ns = 1; + packet->packet_len = packet_len; + skb = bpf_kptr_xchg(&packet->skb, skb); + if (skb) + bpf_qdisc_skb_drop(skb, to_free); + + bpf_spin_lock(&queue_lock); + eligible_ns = next_departure_ns > now ? next_departure_ns : now; + next_departure_ns = eligible_ns + interval_ns; + packet->eligible_ns = eligible_ns; + bpf_list_push_back(&packet_queue, &packet->node); + sch->q.qlen++; + sch->qstats.backlog += packet_len; + __sync_fetch_and_add(&stats.enqueued, 1); + if (sch->q.qlen > stats.max_qlen) + stats.max_qlen = sch->q.qlen; + bpf_spin_unlock(&queue_lock); + return NET_XMIT_SUCCESS; + +drop: + bpf_qdisc_skb_drop(skb, to_free); + __sync_fetch_and_add(&stats.policy_dropped, 1); + return NET_XMIT_DROP; +} +``` + +The interval formula means the first packet after an idle period departs immediately (because `next_departure_ns` will be in the past, so `eligible_ns = now`). Subsequent packets are spaced by serialized packet length at the configured rate. + +### BPF qdisc: dequeue + +The dequeue callback pops the head of the list. If the current time is before the packet's `eligible_ns`, the packet goes back to the front of the list and the qdisc watchdog is scheduled: + +```c +SEC("struct_ops/egress_pacer_dequeue") +struct sk_buff *BPF_PROG(egress_pacer_dequeue, struct Qdisc *sch) +{ + struct bpf_list_node *node; + struct packet_node *packet; + struct sk_buff *skb = NULL; + __u64 expire, now; + __u32 packet_len; + + bpf_spin_lock(&queue_lock); + node = bpf_list_pop_front(&packet_queue); + bpf_spin_unlock(&queue_lock); + if (!node) + return NULL; + + packet = container_of(node, struct packet_node, node); + now = bpf_ktime_get_ns(); + if (now < packet->eligible_ns) { + expire = packet->eligible_ns; + bpf_spin_lock(&queue_lock); + bpf_list_push_front(&packet_queue, &packet->node); + bpf_spin_unlock(&queue_lock); + bpf_qdisc_watchdog_schedule(sch, expire, 0); + return NULL; + } + + packet_len = packet->packet_len; + skb = bpf_kptr_xchg(&packet->skb, skb); + bpf_spin_lock(&queue_lock); + sch->q.qlen--; + sch->qstats.backlog -= packet_len; + bpf_spin_unlock(&queue_lock); + bpf_obj_drop(packet); + + if (!skb) + return NULL; + + bpf_qdisc_bstats_update(sch, skb); + __sync_fetch_and_add(&stats.dequeued, 1); + __sync_fetch_and_add(&stats.bytes_dequeued, packet_len); + return skb; +} +``` + +The watchdog (`bpf_qdisc_watchdog_schedule`) tells the kernel to call dequeue again at the eligible time. This is the pacing mechanism: the kernel does not poll; it sleeps until the next packet is due. + +### BPF qdisc: reset + +On qdisc removal, the `reset` callback drains all retained skbs, frees the `packet_node` objects, and zeroes qlen/backlog. Packets freed here are counted in `cleanup_dropped`, distinct from the policy drops during normal operation. + +### User-space lifecycle + +The loader (`egress_pacer.c`) follows this sequence: + +1. Parse arguments and resolve the interface name to an ifindex. +2. Open the BPF skeleton and set `rodata->rate_kbps` and `rodata->queue_limit`. +3. Load the BPF object (verifier runs here). +4. Attach the struct_ops map, registering `bpf_pacer` as an available qdisc type. +5. Create the qdisc at the interface root via `bpf_tc_hook_create` with `BPF_TC_QDISC` and `TC_H_ROOT`. If the interface already has a root qdisc, the call returns `-EEXIST` and the tool prints the refusal and exits. +6. Print `READY`, then sleep in 100 ms increments until the duration expires or a signal arrives. +7. Destroy the qdisc via `bpf_tc_hook_destroy`, read final BSS counters, print `SUMMARY`, and destroy the skeleton. + +Signal handlers set a flag that breaks the sleep loop. SIGINT and SIGTERM are tested cleanup paths. + +### Statistics + +The `pacer_stats` struct in BSS holds: + +| Field | Meaning | +|-------|---------| +| `enqueued` | packets successfully queued | +| `dequeued` | packets transmitted | +| `policy_dropped` | packets dropped at the queue limit | +| `cleanup_dropped` | packets drained during reset | +| `bytes_dequeued` | total bytes of dequeued packets | +| `max_qlen` | peak queue occupancy | + +The loader reads these from `skel->bss->stats` after destroying the qdisc. + +## Requirements + +- Linux kernel 6.16 or later. BPF qdisc support entered in [this commit](https://github.com/torvalds/linux/commit/c8240344956e3f0b4e8f1d40ec3435e47040cacb). +- libbpf 1.6.0 or later for BPF qdisc TC hook support ([libbpf commit](https://github.com/libbpf/libbpf/commit/f580871b429c550edf910a1b0d700510245351df)). This repository vendors bpftool v7.7.0 and libbpf v1.7.0. +- Architecture: x86_64 (tested). +- Root privileges for BPF load, qdisc creation, and raw test sockets. +- BTF and BPF JIT enabled. +- Kernel configs: `CONFIG_BPF=y`, `CONFIG_BPF_SYSCALL=y`, `CONFIG_BPF_JIT=y`, `CONFIG_DEBUG_INFO_BTF=y`, `CONFIG_NET_SCHED=y`, `CONFIG_NET_SCH_BPF=y`. +- A controlled network interface. A veth pair is sufficient. + +## Limitations + +- One aggregate FIFO. No fairness, per-flow isolation, per-cgroup policy, classes, priorities, ECN, burst budget, or congestion-control integration. +- Queue limit is in packets; rate accounting uses `qdisc_skb_cb(skb)->pkt_len`. +- The first packet after an idle period departs immediately. +- Rate and queue limit are fixed for one invocation. +- One root qdisc instance per loader process due to global queue and BSS state. +- Refuses every existing root qdisc without inspecting, preserving, stacking, or restoring it. +- Duration-bounded CLI. Not a daemon, controller, metrics exporter, or persistent policy manager. +- Normal exit, SIGINT, and SIGTERM are tested cleanup paths. SIGKILL, host crash, or abnormal process failure will not clean up the qdisc. +- KVM test validates behavior but is not a throughput or precision benchmark. It does not account for driver/hardware offload, GSO/TSO, or real NIC timing. + +## Recovery + +After an abnormal exit, verify with: + +```bash +tc qdisc show dev IFACE +``` + +If `bpf_pacer` is still present on a controlled interface, remove it with: + +```bash +sudo tc qdisc del dev IFACE root +``` + +This changes traffic scheduling on the interface. Only do this on an interface you control. + +## References + +- [Linux BPF qdisc merge commit](https://github.com/torvalds/linux/commit/c8240344956e3f0b4e8f1d40ec3435e47040cacb) +- [Qdisc watchdog follow-up commit](https://github.com/torvalds/linux/commit/7a2dafd33c4506e4cb4e63a422530c1cd5a35481) +- [libbpf qdisc TC hook support](https://github.com/libbpf/libbpf/commit/f580871b429c550edf910a1b0d700510245351df) +- [Upstream BPF FIFO qdisc selftest](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/progs/bpf_qdisc_fifo.c) +- [Upstream BPF qdisc test runner](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c) +- [libbpf 1.7.0 release](https://github.com/libbpf/libbpf/releases/tag/v1.7.0) + +## More + +This example is part of the eBPF Developer Tutorial. For more examples and the full tutorial, visit: + +- +- diff --git a/src/53-egress-pacer/README.zh.md b/src/53-egress-pacer/README.zh.md new file mode 100644 index 0000000..2e4abbb --- /dev/null +++ b/src/53-egress-pacer/README.zh.md @@ -0,0 +1,272 @@ +# eBPF 实践教程: 使用 BPF Qdisc 对出口流量进行限速 + +你有一个实验环境、测试台或通过专用 veth/TAP 接口运行的服务。某个组件突然发出流量突发，队列无限制地增长，延迟飙升，而你对丢包情况一无所知。你需要一个即装即用的短期出口限速器，它能报告队列和字节统计，并在退出或收到信号时自动移除。 + +本教程构建的工具正是做这件事。它利用 Linux 6.16 的 `struct_ops` BPF qdisc 基础设施注册一个有界 FIFO，以你指定的总速率对报文进行整形，并将入队/出队/丢弃统计暴露给用户态。 + +这不是生产级工具，不适用于通用物理网卡。它的设计目标是受控接口（veth、TAP、IFB），由操作者做出明确的生命周期决定。加载器拒绝替换任何已存在的 root qdisc。 + +## 快速开始 + +无需 root 即可编译: + +```bash +cd src/53-egress-pacer +make clean +make -j2 +``` + +在专用接口上运行: + +```bash +sudo ./egress_pacer --interface veth-service --rate-kbps 64000 --queue-limit 256 --duration 30 +``` + +qdisc 激活后输出 `READY`，退出时输出 `SUMMARY` 统计: + +```text +READY interface=veth-service rate_kbps=64000 queue_limit=256 duration=30 +SUMMARY enqueued=... dequeued=... policy_dropped=... cleanup_dropped=... bytes_dequeued=... max_qlen=... +``` + +完整用法: + +```text +Usage: ./egress_pacer --interface IFACE [--rate-kbps KBPS] [--queue-limit PACKETS] [--duration SEC] [--verbose] +``` + +| 选项 | 取值范围 | 默认值 | +|------|----------|--------| +| `--interface` | 必选 | 无 | +| `--rate-kbps` | 8 到 100000000 | 1024 | +| `--queue-limit` | 1 到 65535 个报文 | 256 | +| `--duration` | 1 到 86400 秒 | 10 | +| `--verbose` | 开关 | 关闭 | + +如果接口已有 root qdisc，工具会拒绝并输出: + +```text +refusing to replace the existing root qdisc on IFACE +``` + +## 验证过的测试输出 + +仓库包含一个确定性集成测试。以下输出在 KVM 客户机（内核 `7.0.0-rc2+`）中于宿主机编译后采集: + +```console +READY interface=epac_tx rate_kbps=64 queue_limit=8 duration=2 +SUMMARY enqueued=9 dequeued=9 policy_dropped=31 cleanup_dropped=0 bytes_dequeued=9216 max_qlen=8 +TEST-SUMMARY attempts=40 received=9 send_errors=31 span_ms=1024 observed_kbps=64.0 +PASS: conflict refusal, bounded drops, pacing, accounting, and normal/signal cleanup succeeded +``` + +这是功能性冒烟测试，不是基准性能数据。测试向 64 Kbit/s、队列容量 8 的 pacer 突发 40 个 1024 字节的 EtherType `0x88B5` 原始帧。31 个在队列满时被丢弃，9 个被整形后交付。不要将这些数字推广到其他场景。 + +在一个可随意使用的 Linux 6.16+ 系统上自行运行: + +```bash +sudo python3 tests/test_egress_pacer.py ./egress_pacer +``` + +该测试会创建并删除 `epac_tx`/`epac_rx` veth 对，先安装一个 `pfifo` 以验证冲突拒绝行为，然后发送突发流量。它断言有界丢包、字节和报文计数正确、整形有效、正常退出清理正常、SIGTERM 清理正常，最后在 `finally` 块中删除 veth 对。 + +## 工作原理 + +本工具由两部分组成: 实现 `struct Qdisc_ops` 的 BPF 对象，以及管理 qdisc 生命周期的用户态加载器。 + +### BPF qdisc: 入队 + +当报文到达 qdisc 时，入队回调检查队列长度是否超过 `sch->limit`。如果队列已满，报文被丢弃并累加 `policy_dropped`。否则: + +1. 通过 `bpf_obj_new` 分配一个 `packet_node`。 +2. 通过 `bpf_kptr_xchg` 将 skb 转移到节点中。 +3. 计算报文的允许发送时间戳: `next_departure_ns + (packet_len * 8 * 1,000,000 / rate_kbps)` 纳秒。 +4. 将节点推入 BPF 链表，更新 qlen 和 backlog。 + +BPF 源码（`egress_pacer.bpf.c`）: + +```c +SEC("struct_ops/egress_pacer_enqueue") +int BPF_PROG(egress_pacer_enqueue, struct sk_buff *skb, struct Qdisc *sch, + struct bpf_sk_buff_ptr *to_free) +{ + struct packet_node *packet; + __u64 eligible_ns, interval_ns, now; + __u32 packet_len; + + packet_len = qdisc_packet_len(skb); + if (sch->q.qlen >= sch->limit) + goto drop; + + packet = bpf_obj_new(typeof(*packet)); + if (!packet) + goto drop; + + now = bpf_ktime_get_ns(); + interval_ns = (__u64)packet_len * 8 * 1000000ULL / rate_kbps; + if (!interval_ns) + interval_ns = 1; + packet->packet_len = packet_len; + skb = bpf_kptr_xchg(&packet->skb, skb); + if (skb) + bpf_qdisc_skb_drop(skb, to_free); + + bpf_spin_lock(&queue_lock); + eligible_ns = next_departure_ns > now ? next_departure_ns : now; + next_departure_ns = eligible_ns + interval_ns; + packet->eligible_ns = eligible_ns; + bpf_list_push_back(&packet_queue, &packet->node); + sch->q.qlen++; + sch->qstats.backlog += packet_len; + __sync_fetch_and_add(&stats.enqueued, 1); + if (sch->q.qlen > stats.max_qlen) + stats.max_qlen = sch->q.qlen; + bpf_spin_unlock(&queue_lock); + return NET_XMIT_SUCCESS; + +drop: + bpf_qdisc_skb_drop(skb, to_free); + __sync_fetch_and_add(&stats.policy_dropped, 1); + return NET_XMIT_DROP; +} +``` + +间隔公式意味着空闲后的第一个报文可以立即发出（因为 `next_departure_ns` 已经过去，`eligible_ns = now`）。后续报文按配置速率和序列化报文长度间隔发送。 + +### BPF qdisc: 出队 + +出队回调弹出链表头部。如果当前时间早于报文的 `eligible_ns`，报文被放回链表头部，并调度 qdisc watchdog: + +```c +SEC("struct_ops/egress_pacer_dequeue") +struct sk_buff *BPF_PROG(egress_pacer_dequeue, struct Qdisc *sch) +{ + struct bpf_list_node *node; + struct packet_node *packet; + struct sk_buff *skb = NULL; + __u64 expire, now; + __u32 packet_len; + + bpf_spin_lock(&queue_lock); + node = bpf_list_pop_front(&packet_queue); + bpf_spin_unlock(&queue_lock); + if (!node) + return NULL; + + packet = container_of(node, struct packet_node, node); + now = bpf_ktime_get_ns(); + if (now < packet->eligible_ns) { + expire = packet->eligible_ns; + bpf_spin_lock(&queue_lock); + bpf_list_push_front(&packet_queue, &packet->node); + bpf_spin_unlock(&queue_lock); + bpf_qdisc_watchdog_schedule(sch, expire, 0); + return NULL; + } + + packet_len = packet->packet_len; + skb = bpf_kptr_xchg(&packet->skb, skb); + bpf_spin_lock(&queue_lock); + sch->q.qlen--; + sch->qstats.backlog -= packet_len; + bpf_spin_unlock(&queue_lock); + bpf_obj_drop(packet); + + if (!skb) + return NULL; + + bpf_qdisc_bstats_update(sch, skb); + __sync_fetch_and_add(&stats.dequeued, 1); + __sync_fetch_and_add(&stats.bytes_dequeued, packet_len); + return skb; +} +``` + +watchdog（`bpf_qdisc_watchdog_schedule`）告诉内核在 eligible 时间再次调用 dequeue。这就是整形机制: 内核不轮询，而是休眠到下一个报文到期。 + +### BPF qdisc: reset + +qdisc 被移除时，`reset` 回调排空所有残留的 skb，释放 `packet_node` 对象，将 qlen/backlog 归零。在此阶段释放的报文计入 `cleanup_dropped`，区别于正常运行期间的策略丢弃。 + +### 用户态生命周期 + +加载器（`egress_pacer.c`）按以下流程执行: + +1. 解析参数，将接口名转换为 ifindex。 +2. 打开 BPF skeleton，设置 `rodata->rate_kbps` 和 `rodata->queue_limit`。 +3. 加载 BPF 对象（验证器在此运行）。 +4. attach struct_ops map，将 `bpf_pacer` 注册为可用的 qdisc 类型。 +5. 通过 `bpf_tc_hook_create`（带 `BPF_TC_QDISC` 和 `TC_H_ROOT`）在接口 root 创建 qdisc。若接口已有 root qdisc，返回 `-EEXIST`，工具输出拒绝信息并退出。 +6. 打印 `READY`，以 100 ms 间隔休眠直到持续时间到期或信号到达。 +7. 通过 `bpf_tc_hook_destroy` 销毁 qdisc，读取最终 BSS 计数器，打印 `SUMMARY`，销毁 skeleton。 + +信号处理器设置标志位来中断休眠循环。SIGINT 和 SIGTERM 都是经过测试的清理路径。 + +### 统计字段 + +BSS 中的 `pacer_stats` 结构体包含: + +| 字段 | 含义 | +|------|------| +| `enqueued` | 成功入队的报文数 | +| `dequeued` | 已发送的报文数 | +| `policy_dropped` | 队列满时丢弃的报文数 | +| `cleanup_dropped` | reset 阶段排空的报文数 | +| `bytes_dequeued` | 已发送报文的总字节数 | +| `max_qlen` | 队列峰值占用 | + +加载器在销毁 qdisc 后从 `skel->bss->stats` 读取这些字段。 + +## 运行要求 + +- Linux 内核 6.16 及以上。BPF qdisc 支持通过[此提交](https://github.com/torvalds/linux/commit/c8240344956e3f0b4e8f1d40ec3435e47040cacb)合入。 +- libbpf 1.6.0 及以上用于 BPF qdisc TC hook 支持（[libbpf 提交](https://github.com/libbpf/libbpf/commit/f580871b429c550edf910a1b0d700510245351df)）。本仓库集成了 bpftool v7.7.0 和 libbpf v1.7.0。 +- 架构: x86_64（已测试）。 +- 需要 root 权限进行 BPF 加载、qdisc 创建和原始测试套接字操作。 +- 需要开启 BTF 和 BPF JIT。 +- 内核配置: `CONFIG_BPF=y`、`CONFIG_BPF_SYSCALL=y`、`CONFIG_BPF_JIT=y`、`CONFIG_DEBUG_INFO_BTF=y`、`CONFIG_NET_SCHED=y`、`CONFIG_NET_SCH_BPF=y`。 +- 需要受控网络接口，veth 对即可满足。 + +## 限制 + +- 单一聚合 FIFO。没有公平调度、每流隔离、每 cgroup 策略、分类、优先级、ECN、突发预算或拥塞控制集成。 +- 队列限制以报文数计量，速率计算使用 `qdisc_skb_cb(skb)->pkt_len`。 +- 空闲后的第一个报文可以立即发出。 +- 速率和队列限制在一次调用中固定不变。 +- 由于全局队列和 BSS 状态，每个加载器进程只能运行一个 root qdisc 实例。 +- 拒绝所有已存在的 root qdisc，不检查、不保留、不堆叠、不恢复。 +- 有限时间的 CLI 工具，不是守护进程、控制器、指标导出器或持久策略管理器。 +- 正常退出、SIGINT 和 SIGTERM 是经过测试的清理路径。SIGKILL、宿主机崩溃或进程异常终止不保证清理 qdisc。 +- KVM 测试验证行为正确性，不是吞吐量或精度基准测试，不考虑驱动/硬件卸载、GSO/TSO 或真实网卡时序。 + +## 恢复 + +异常退出后，验证 qdisc 状态: + +```bash +tc qdisc show dev IFACE +``` + +如果 `bpf_pacer` 仍存在于受控接口上，移除它: + +```bash +sudo tc qdisc del dev IFACE root +``` + +此操作会改变接口的流量调度。只在你控制的接口上执行。 + +## 参考资料 + +- [Linux BPF qdisc 合入提交](https://github.com/torvalds/linux/commit/c8240344956e3f0b4e8f1d40ec3435e47040cacb) +- [Qdisc watchdog 后续提交](https://github.com/torvalds/linux/commit/7a2dafd33c4506e4cb4e63a422530c1cd5a35481) +- [libbpf qdisc TC hook 支持](https://github.com/libbpf/libbpf/commit/f580871b429c550edf910a1b0d700510245351df) +- [上游 BPF FIFO qdisc 自测程序](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/progs/bpf_qdisc_fifo.c) +- [上游 BPF qdisc 测试运行器](https://github.com/torvalds/linux/blob/v7.1/tools/testing/selftests/bpf/prog_tests/bpf_qdisc.c) +- [libbpf 1.7.0 发布](https://github.com/libbpf/libbpf/releases/tag/v1.7.0) + +## 更多内容 + +本示例是 eBPF 开发者教程的一部分。更多示例和完整教程请访问: + +- +- diff --git a/src/SUMMARY.md b/src/SUMMARY.md index ca62e48..523a4a2 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -68,6 +68,7 @@ Networking: - [lesson 42-xdp-loadbalancer](42-xdp-loadbalancer/README.md) XDP Load Balancer - [lesson 46-xdp-test](46-xdp-test/README.md) Building a High-Performance XDP Packet Generator - [lesson 50-tcx](50-tcx/README.md) Composable Traffic Control with TCX Links +- [lesson 53-egress-pacer](53-egress-pacer/README.md) Pace Egress Traffic with a BPF Qdisc Tracing: diff --git a/src/SUMMARY.zh.md b/src/SUMMARY.zh.md index b0ed1c4..646689f 100644 --- a/src/SUMMARY.zh.md +++ b/src/SUMMARY.zh.md @@ -68,6 +68,7 @@ GPU: - [lesson 42-xdp-loadbalancer](42-xdp-loadbalancer/README.zh.md) eBPF 开发者教程： 简单的 XDP 负载均衡器 - [lesson 46-xdp-test](46-xdp-test/README.zh.md) eBPF 实例教程：构建高性能 XDP 数据包生成器 - [lesson 50-tcx](50-tcx/README.zh.md) eBPF 入门实践教程第五十篇：使用 TCX Link 实现可组合的流量控制 +- [lesson 53-egress-pacer](53-egress-pacer/README.zh.md) 使用 BPF Qdisc 对出口流量进行限速 追踪: diff --git a/src/compatibility.md b/src/compatibility.md index e3d8161..349d498 100644 --- a/src/compatibility.md +++ b/src/compatibility.md @@ -57,6 +57,7 @@ This table is generated from each tutorial's `.config` metadata. `Minimum kernel | [eBPF Tutorial: Energy Monitoring for Process-Level Power Analysis](48-energy/README.md) | 5.8 | Required feature | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_TRACEPOINTS=y | None | Required | CI build | | [eBPF Tutorial: Fixing Broken HID Devices Without Kernel Patches](49-hid/README.md) | 6.3 | Tutorial docs | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_HID_BPF=y, CONFIG_UHID=y\|m | None | Required | CI build | | [eBPF Tutorial by Example 50: Composable Traffic Control with TCX Links](50-tcx/README.md) | 6.6 | Tutorial docs | x86_64, arm64 | Optional | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_NET_XGRESS=y | Network interface | Required | CI build | +| [eBPF Tutorial by Example: Pace Egress Traffic with a BPF Qdisc](53-egress-pacer/README.md) | 6.16 | Tutorial docs | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_NET_SCHED=y, CONFIG_NET_SCH_BPF=y | Network interface | Required | CI build | | [eBPF Tutorial: cgroup-based Policy Control](cgroup/README.md) | 5.8 | Required feature | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_CGROUPS=y, CONFIG_CGROUP_BPF=y, CONFIG_CGROUP_DEVICE=y, CONFIG_DEBUG_INFO_BTF=y | None | Required | CI build | | [eBPF Tutorial by Example: BPF Arena for Zero-Copy Shared Memory](features/bpf_arena/README.md) | 6.9 | Tutorial docs | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y | None | Required | CI build | | [eBPF Tutorial: BPF Iterators for Kernel Data Export](features/bpf_iters/README.md) | 5.8 | Tutorial docs | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_DEBUG_INFO_BTF=y | None | Required | CI build | diff --git a/src/compatibility.zh.md b/src/compatibility.zh.md index 8a5b44a..3070435 100644 --- a/src/compatibility.zh.md +++ b/src/compatibility.zh.md @@ -57,6 +57,7 @@ | [eBPF 教程：进程级能源监控与功耗分析](48-energy/README.zh.md) | 5.8 | 必需特性 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_TRACEPOINTS=y | 无 | 需要 | CI 构建测试 | | [eBPF 教程：无需内核补丁修复故障的 HID 设备](49-hid/README.zh.md) | 6.3 | 教程文档 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_HID_BPF=y, CONFIG_UHID=y\|m | 无 | 需要 | CI 构建测试 | | [eBPF 入门实践教程第五十篇：使用 TCX Link 实现可组合的流量控制](50-tcx/README.zh.md) | 6.6 | 教程文档 | x86_64, arm64 | 可选 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_NET_XGRESS=y | 网络接口 | 需要 | CI 构建测试 | +| [eBPF 实践教程: 使用 BPF Qdisc 对出口流量进行限速](53-egress-pacer/README.zh.md) | 6.16 | 教程文档 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_NET_SCHED=y, CONFIG_NET_SCH_BPF=y | 网络接口 | 需要 | CI 构建测试 | | [eBPF 实例教程：基于 cgroup 的策略控制](cgroup/README.zh.md) | 5.8 | 必需特性 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_CGROUPS=y, CONFIG_CGROUP_BPF=y, CONFIG_CGROUP_DEVICE=y, CONFIG_DEBUG_INFO_BTF=y | 无 | 需要 | CI 构建测试 | | [eBPF 实例教程：BPF Arena 零拷贝共享内存](features/bpf_arena/README.zh.md) | 6.9 | 教程文档 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y | 无 | 需要 | CI 构建测试 | | [eBPF 教程：BPF 迭代器用于内核数据导出](features/bpf_iters/README.zh.md) | 5.8 | 教程文档 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_DEBUG_INFO_BTF=y | 无 | 需要 | CI 构建测试 |