From e08f6047693e3fded2595be64e28bc2ef5f617e6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=BA=91=E5=BE=AE?= <1067852565@qq.com> Date: Tue, 14 Jul 2026 04:10:51 -0700 Subject: [PATCH] Fix bilingual tutorial indexes and internal links (#209) * Fix bilingual tutorial index generation * Check relative Markdown links in CI * Harden documentation regression checks * Reduce link parser complexity * Run link checks for every repository change --------- Co-authored-by: LinuxDev9002 --- .github/workflows/check-docs.yml | 23 +++ README.md | 8 +- README.zh.md | 39 +++-- scripts/README.md.template | 2 +- scripts/README.zh.md.template | 6 +- scripts/check_internal_links.py | 205 +++++++++++++++++++++++++++ scripts/generate_toc.py | 28 ++-- scripts/test_check_internal_links.py | 94 ++++++++++++ scripts/test_generate_toc.py | 49 +++++++ src/SUMMARY.md | 6 +- src/SUMMARY.zh.md | 33 ++++- 11 files changed, 456 insertions(+), 37 deletions(-) create mode 100644 .github/workflows/check-docs.yml create mode 100644 scripts/check_internal_links.py create mode 100644 scripts/test_check_internal_links.py create mode 100644 scripts/test_generate_toc.py diff --git a/.github/workflows/check-docs.yml b/.github/workflows/check-docs.yml new file mode 100644 index 0000000..ba06c84 --- /dev/null +++ b/.github/workflows/check-docs.yml @@ -0,0 +1,23 @@ +name: Check documentation + +on: + push: + branches: ["main"] + pull_request: + +permissions: + contents: read + +jobs: + docs: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + - name: Test documentation scripts + run: python3 -m unittest scripts.test_check_internal_links scripts.test_generate_toc + - name: Regenerate documentation indexes + run: python3 scripts/generate_toc.py + - name: Verify generated documentation is current + run: git diff --exit-code -- README.md README.zh.md src/SUMMARY.md src/SUMMARY.zh.md + - name: Check relative Markdown links + run: python3 scripts/check_internal_links.py diff --git a/README.md b/README.md index 045358d..698bbff 100644 --- a/README.md +++ b/README.md @@ -56,9 +56,9 @@ This section covers advanced topics related to eBPF, including using eBPF progra GPU: - [lesson 47-cuda-events](src/47-cuda-events/README.md) Tracing CUDA GPU Operations -- [lesson xpu/npu-kernel-driver](src/xpu/npu-kernel-driver/README.md) Tracing Intel NPU Kernel Driver Operations - [xpu flamegraph](src/xpu/flamegraph/README.md) Building a GPU Flamegraph Profiler with CUPTI - [lesson xpu/gpu-kernel-driver](src/xpu/gpu-kernel-driver/README.md) Monitoring GPU Driver Activity with Kernel Tracepoints +- [lesson xpu/npu-kernel-driver](src/xpu/npu-kernel-driver/README.md) Tracing Intel NPU Kernel Driver Operations Scheduler: @@ -104,11 +104,11 @@ Features: - [lesson 36-userspace-ebpf](src/36-userspace-ebpf/README.md) Userspace eBPF Runtimes: Overview and Applications - [lesson 38-btf-uprobe](src/38-btf-uprobe/README.md) Expanding eBPF Compile Once, Run Everywhere(CO-RE) to Userspace Compatibility - [lesson 43-kfuncs](src/43-kfuncs/README.md) Extending eBPF Beyond Its Limits: Custom kfuncs in Kernel Modules +- [features bpf_arena](src/features/bpf_arena/README.md) BPF Arena for Zero-Copy Shared Memory - [features bpf_iters](src/features/bpf_iters/README.md) BPF Iterators for Kernel Data Export - [features bpf_token](src/features/bpf_token/README.md) BPF Token for Delegated Privilege and Secure Program Loading -- [features dynptr](src/features/dynptr/README.md) BPF Dynamic Pointers for Variable-Length Data - [features bpf_wq](src/features/bpf_wq/README.md) BPF Workqueues for Asynchronous Sleepable Tasks -- [features bpf_arena](src/features/bpf_arena/README.md) BPF Arena for Zero-Copy Shared Memory +- [features dynptr](src/features/dynptr/README.md) BPF Dynamic Pointers for Variable-Length Data - [features struct_ops](src/features/struct_ops/README.md) Extending Kernel Subsystems with BPF struct_ops Other: @@ -125,7 +125,7 @@ Continuously updating... ## Why write this tutorial? -In the process of learning eBPF, we have been inspired and helped by the [bcc python developer tutorial](src/bcc-documents/tutorial_bcc_python_developer.md). However, from the current perspective, using `libbpf` to develop eBPF applications is a relatively better choice. +In the process of learning eBPF, we have been inspired and helped by the [bcc Python developer tutorial](https://github.com/iovisor/bcc/blob/master/docs/tutorial_bcc_python_developer.md). However, from the current perspective, using `libbpf` to develop eBPF applications is a relatively better choice. This project is mainly based on [libbpf](https://github.com/libbpf/libbpf) frameworks. diff --git a/README.zh.md b/README.zh.md index e5e2ce7..9dd066a 100644 --- a/README.zh.md +++ b/README.zh.md @@ -4,7 +4,7 @@ [GitHub](https://github.com/eunomia-bpf/bpf-developer-tutorial) [Gitee 镜像](https://gitee.com/yunwei37/bpf-developer-tutorial) -[English Version](README_en.md) +[English Version](README.md) #### [**Check out the English version here**](README.md) @@ -50,16 +50,22 @@ 这一部分涵盖了与 eBPF 相关的高级主题，包括在 Android 上使用 eBPF 程序、利用 eBPF 程序进行的潜在攻击和防御以及复杂的追踪。结合用户模式和内核模式的 eBPF 可以带来强大的能力（也可能带来安全风险）。 + + GPU: - [lesson 47-cuda-events](src/47-cuda-events/README.zh.md) eBPF 教程：追踪 CUDA GPU 操作 -- [lesson xpu/npu-kernel-driver](src/xpu/npu-kernel-driver/README.zh.md) eBPF 实例教程：跟踪 Intel NPU 内核驱动操作 - [xpu flamegraph](src/xpu/flamegraph/README.zh.md) eBPF 示例：使用 CUPTI 构建 GPU 火焰图分析器 - [lesson xpu/gpu-kernel-driver](src/xpu/gpu-kernel-driver/README.zh.md) eBPF 实例教程：使用内核跟踪点监控 GPU 驱动活动 +- [lesson xpu/npu-kernel-driver](src/xpu/npu-kernel-driver/README.zh.md) eBPF 实例教程：跟踪 Intel NPU 内核驱动操作 + + 调度器: - [lesson 44-scx-simple](src/44-scx-simple/README.zh.md) eBPF 教程：BPF 调度器入门 - [lesson 45-scx-nest](src/45-scx-nest/README.zh.md) eBPF 示例教程：实现 `scx_nest` 调度器 + + 网络: - [lesson 23-http](src/23-http/README.zh.md) 通过 eBPF socket filter 或 syscall trace 追踪 HTTP 请求等七层协议 - eBPF 实践教程 @@ -68,6 +74,19 @@ GPU: - [lesson 42-xdp-loadbalancer](src/42-xdp-loadbalancer/README.zh.md) eBPF 开发者教程： 简单的 XDP 负载均衡器 - [lesson 46-xdp-test](src/46-xdp-test/README.zh.md) eBPF 实例教程：构建高性能 XDP 数据包生成器 - [lesson 50-tcx](src/50-tcx/README.zh.md) eBPF 入门实践教程第五十篇：使用 TCX Link 实现可组合的流量控制 + + +追踪: + +- [lesson 30-sslsniff](src/30-sslsniff/README.zh.md) eBPF 实践教程：使用 uprobe 捕获多种库的 SSL/TLS 明文数据 +- [lesson 31-goroutine](src/31-goroutine/README.zh.md) eBPF 实践教程：使用 eBPF 跟踪 Go 协程状态 +- [lesson 33-funclatency](src/33-funclatency/README.zh.md) 使用 eBPF 测量函数延迟 +- [lesson 37-uprobe-rust](src/37-uprobe-rust/README.zh.md) eBPF 实践：使用 Uprobe 追踪用户态 Rust 应用 +- [lesson 39-nginx](src/39-nginx/README.zh.md) 使用 eBPF 跟踪 Nginx 请求 +- [lesson 40-mysql](src/40-mysql/README.zh.md) 使用 eBPF 跟踪 MySQL 查询 +- [lesson 48-energy](src/48-energy/README.zh.md) eBPF 教程：进程级能源监控与功耗分析 + + 安全: - [lesson 24-hide](src/24-hide/README.zh.md) eBPF 开发实践：使用 eBPF 隐藏进程或文件信息 @@ -76,22 +95,28 @@ GPU: - [lesson 27-replace](src/27-replace/README.zh.md) 替换任意程序读取或者写入的文本 - [lesson 28-detach](src/28-detach/README.zh.md) 在应用程序退出后运行 eBPF 程序：eBPF 程序的生命周期 - [lesson 34-syscall](src/34-syscall/README.zh.md) eBPF 开发实践：使用 eBPF 修改系统调用参数 + + 特性: - [lesson 35-user-ringbuf](src/35-user-ringbuf/README.zh.md) eBPF开发实践：使用 user ring buffer 向内核异步发送信息 - [lesson 36-userspace-ebpf](src/36-userspace-ebpf/README.zh.md) 用户空间 eBPF 运行时：深度解析与应用实践 - [lesson 38-btf-uprobe](src/38-btf-uprobe/README.zh.md) 借助 eBPF 和 BTF，让用户态也能一次编译、到处运行 - [lesson 43-kfuncs](src/43-kfuncs/README.zh.md) 超越 eBPF 的极限：在内核模块中定义自定义 kfunc +- [features bpf_arena](src/features/bpf_arena/README.zh.md) eBPF 实例教程：BPF Arena 零拷贝共享内存 - [features bpf_iters](src/features/bpf_iters/README.zh.md) eBPF 教程：BPF 迭代器用于内核数据导出 - [features bpf_token](src/features/bpf_token/README.zh.md) eBPF 入门实践教程：BPF Token，安全的委托式权限与程序加载 -- [features dynptr](src/features/dynptr/README.zh.md) BPF Dynamic Pointers for Variable-Length Data - [features bpf_wq](src/features/bpf_wq/README.zh.md) eBPF 教程：BPF 工作队列用于异步可睡眠任务 -- [features bpf_arena](src/features/bpf_arena/README.zh.md) eBPF 实例教程：BPF Arena 零拷贝共享内存 +- [features dynptr](src/features/dynptr/README.zh.md) BPF Dynamic Pointers for Variable-Length Data - [features struct_ops](src/features/struct_ops/README.zh.md) eBPF 教程：使用 BPF struct_ops 扩展内核子系统 -特性: + + +其他: - [lesson 49-hid](src/49-hid/README.zh.md) eBPF 教程：无需内核补丁修复故障的 HID 设备 - [cgroup](src/cgroup/README.zh.md) eBPF 实例教程：基于 cgroup 的策略控制 + + Android: - [lesson 22-android](src/22-android/README.zh.md) 在 Android 上使用 eBPF 程序 @@ -100,7 +125,7 @@ Android: ## 为什么要写这个教程？ -在学习 eBPF 的过程中，我们受到了 [bcc python developer tutorial](src/bcc-documents/tutorial_bcc_python_developer.md) 的许多启发和帮助，但从当下的角度出发，使用 libbpf 开发 eBPF 的应用是目前相对更好的选择。但目前似乎很少有基于 libbpf 和 BPF CO-RE 出发的、通过案例和工具介绍 eBPF 开发的教程，因此我们发起了这个项目，采用类似 bcc python developer tutorial 的组织方式，但使用 CO-RE 的 libbpf 进行开发。 +在学习 eBPF 的过程中，我们受到了 [bcc Python developer tutorial](https://github.com/iovisor/bcc/blob/master/docs/tutorial_bcc_python_developer.md) 的许多启发和帮助，但从当下的角度出发，使用 libbpf 开发 eBPF 的应用是目前相对更好的选择。但目前似乎很少有基于 libbpf 和 BPF CO-RE 出发的、通过案例和工具介绍 eBPF 开发的教程，因此我们发起了这个项目，采用类似 bcc Python developer tutorial 的组织方式，但使用 CO-RE 的 libbpf 进行开发。 本项目主要基于 [libbpf-boostrap](https://github.com/libbpf/libbpf-bootstrap) 和 [eunomia-bpf](https://github.com/eunomia-bpf/eunomia-bpf) 两个框架完成，并使用 eunomia-bpf 帮助简化一部分 libbpf eBPF 用户态代码的编写，让开发者专注于内核态的 eBPF 代码的开发。 @@ -228,8 +253,6 @@ eunomia-bpf 由一个编译工具链和一个运行时库组成, 对比传统的 ![ebpf-chatgpt-signal](imgs/ebpf-chatgpt-signal.png) -完整的对话记录可以在这里找到: [ChatGPT.md](src/ChatGPT.md) - 我们也构建了一个命令行工具的 demo ，通过本教程的训练， 让它通过自然语言描述即可自动编写 eBPF 程序，追踪 Linux 系统： ![ebpf-chatgpt-signal](https://github.com/eunomia-bpf/GPTtrace/blob/main/doc/result.gif) diff --git a/scripts/README.md.template b/scripts/README.md.template index f0718ae..206027a 100644 --- a/scripts/README.md.template +++ b/scripts/README.md.template @@ -19,7 +19,7 @@ The tutorial focuses on eBPF examples in observability, networking, security, an ## Why write this tutorial? -In the process of learning eBPF, we have been inspired and helped by the [bcc python developer tutorial](src/bcc-documents/tutorial_bcc_python_developer.md). However, from the current perspective, using `libbpf` to develop eBPF applications is a relatively better choice. +In the process of learning eBPF, we have been inspired and helped by the [bcc Python developer tutorial](https://github.com/iovisor/bcc/blob/master/docs/tutorial_bcc_python_developer.md). However, from the current perspective, using `libbpf` to develop eBPF applications is a relatively better choice. This project is mainly based on [libbpf](https://github.com/libbpf/libbpf) frameworks. diff --git a/scripts/README.zh.md.template b/scripts/README.zh.md.template index 17ecd3f..c277d81 100644 --- a/scripts/README.zh.md.template +++ b/scripts/README.zh.md.template @@ -4,7 +4,7 @@ [GitHub](https://github.com/eunomia-bpf/bpf-developer-tutorial) [Gitee 镜像](https://gitee.com/yunwei37/bpf-developer-tutorial) -[English Version](README_en.md) +[English Version](README.md) #### [**Check out the English version here**](README.md) @@ -18,7 +18,7 @@ ## 为什么要写这个教程？ -在学习 eBPF 的过程中，我们受到了 [bcc python developer tutorial](src/bcc-documents/tutorial_bcc_python_developer.md) 的许多启发和帮助，但从当下的角度出发，使用 libbpf 开发 eBPF 的应用是目前相对更好的选择。但目前似乎很少有基于 libbpf 和 BPF CO-RE 出发的、通过案例和工具介绍 eBPF 开发的教程，因此我们发起了这个项目，采用类似 bcc python developer tutorial 的组织方式，但使用 CO-RE 的 libbpf 进行开发。 +在学习 eBPF 的过程中，我们受到了 [bcc Python developer tutorial](https://github.com/iovisor/bcc/blob/master/docs/tutorial_bcc_python_developer.md) 的许多启发和帮助，但从当下的角度出发，使用 libbpf 开发 eBPF 的应用是目前相对更好的选择。但目前似乎很少有基于 libbpf 和 BPF CO-RE 出发的、通过案例和工具介绍 eBPF 开发的教程，因此我们发起了这个项目，采用类似 bcc Python developer tutorial 的组织方式，但使用 CO-RE 的 libbpf 进行开发。 本项目主要基于 [libbpf-boostrap](https://github.com/libbpf/libbpf-bootstrap) 和 [eunomia-bpf](https://github.com/eunomia-bpf/eunomia-bpf) 两个框架完成，并使用 eunomia-bpf 帮助简化一部分 libbpf eBPF 用户态代码的编写，让开发者专注于内核态的 eBPF 代码的开发。 @@ -146,8 +146,6 @@ eunomia-bpf 由一个编译工具链和一个运行时库组成, 对比传统的 ![ebpf-chatgpt-signal](imgs/ebpf-chatgpt-signal.png) -完整的对话记录可以在这里找到: [ChatGPT.md](src/ChatGPT.md) - 我们也构建了一个命令行工具的 demo ，通过本教程的训练， 让它通过自然语言描述即可自动编写 eBPF 程序，追踪 Linux 系统： ![ebpf-chatgpt-signal](https://github.com/eunomia-bpf/GPTtrace/blob/main/doc/result.gif) diff --git a/scripts/check_internal_links.py b/scripts/check_internal_links.py new file mode 100644 index 0000000..287077c --- /dev/null +++ b/scripts/check_internal_links.py @@ -0,0 +1,205 @@ +#!/usr/bin/env python3 +"""Check relative Markdown links and images without making network requests.""" + +from __future__ import annotations + +import argparse +import re +import sys +from dataclasses import dataclass +from pathlib import Path +from urllib.parse import unquote, urlsplit + + +INLINE_LINK_PATTERN = re.compile(r"(??@\[\\\]^_`{|}~])") +IGNORED_SCHEMES = {"data", "http", "https", "javascript", "mailto"} + + +@dataclass(frozen=True) +class BrokenLink: + source: Path + line: int + destination: str + + +def markdown_files(root: Path) -> list[Path]: + return sorted( + path + for path in root.rglob("*.md") + if not {".git", "build", "node_modules", "site"}.intersection(path.parts) + ) + + +def strip_code_spans(line: str) -> str: + """Blank inline code while preserving character offsets.""" + result = list(line) + position = 0 + while position < len(line): + if line[position] != "`": + position += 1 + continue + + delimiter_end = position + while delimiter_end < len(line) and line[delimiter_end] == "`": + delimiter_end += 1 + delimiter_length = delimiter_end - position + closing = delimiter_end + while closing < len(line): + if line[closing] != "`": + closing += 1 + continue + closing_end = closing + while closing_end < len(line) and line[closing_end] == "`": + closing_end += 1 + if closing_end - closing == delimiter_length: + break + closing = closing_end + if closing >= len(line): + position = delimiter_end + continue + + result[position : closing + delimiter_length] = " " * ( + closing + delimiter_length - position + ) + position = closing + delimiter_length + return "".join(result) + + +def parse_angle_destination(text: str, position: int) -> str: + destination: list[str] = [] + while position < len(text): + if text[position] == "\\" and position + 1 < len(text): + destination.extend(text[position : position + 2]) + position += 2 + elif text[position] == ">": + return "".join(destination) + else: + destination.append(text[position]) + position += 1 + return "" + + +def parse_bare_destination(text: str, position: int) -> str: + destination = [] + parentheses = 0 + while position < len(text): + character = text[position] + if character == "\\" and position + 1 < len(text): + destination.extend(text[position : position + 2]) + position += 2 + continue + if character == "(": + parentheses += 1 + elif character == ")": + if parentheses == 0: + break + parentheses -= 1 + elif character.isspace() and parentheses == 0: + break + destination.append(character) + position += 1 + return "".join(destination) if parentheses == 0 else "" + + +def parse_destination(text: str, start: int = 0) -> str: + """Parse one CommonMark-style link destination from text.""" + position = start + while position < len(text) and text[position] in " \t": + position += 1 + if position >= len(text): + return "" + if text[position] == "<": + return parse_angle_destination(text, position + 1) + return parse_bare_destination(text, position) + + +def destinations_in_line(line: str) -> list[str]: + without_code = strip_code_spans(line) + destinations = [ + parse_destination(without_code, match.end()) + for match in INLINE_LINK_PATTERN.finditer(without_code) + ] + reference = REFERENCE_DEFINITION_PATTERN.match(without_code) + if reference: + destinations.append(parse_destination(reference.group(1))) + return destinations + + +def is_ignored(destination: str) -> bool: + parsed = urlsplit(destination) + return ( + not destination + or destination.startswith(("#", "/")) + or parsed.scheme.lower() in IGNORED_SCHEMES + or any(marker in destination for marker in ("${", "{{", "}}")) + ) + + +def filesystem_path(destination: str) -> str: + decoded = unquote(urlsplit(destination).path) + return MARKDOWN_ESCAPE_PATTERN.sub(r"\1", decoded) + + +def find_broken_links(root: Path) -> list[BrokenLink]: + root = root.resolve() + broken: list[BrokenLink] = [] + + for source in markdown_files(root): + fence: tuple[str, int] | None = None + for line_number, line in enumerate(source.read_text(encoding="utf-8").splitlines(), 1): + match = FENCE_PATTERN.match(line) + if fence is not None: + marker, minimum_length = fence + if re.fullmatch( + rf" {{0,3}}{re.escape(marker)}{{{minimum_length},}}[ \t]*", line + ): + fence = None + continue + if match: + marker = match.group(1) + if not (marker[0] == "`" and "`" in match.group(2)): + fence = (marker[0], len(marker)) + continue + + for destination in destinations_in_line(line): + if is_ignored(destination): + continue + relative_path = filesystem_path(destination) + if relative_path and not (source.parent / relative_path).exists(): + broken.append( + BrokenLink(source.relative_to(root), line_number, destination) + ) + + return broken + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "root", + nargs="?", + type=Path, + default=Path(__file__).resolve().parent.parent, + help="repository root (defaults to the parent of scripts/)", + ) + args = parser.parse_args() + + broken = find_broken_links(args.root) + if not broken: + print("All relative Markdown links resolve.") + return 0 + + print(f"Found {len(broken)} broken relative Markdown link(s):", file=sys.stderr) + for item in broken: + print( + f"{item.source}:{item.line}: {item.destination}", + file=sys.stderr, + ) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/generate_toc.py b/scripts/generate_toc.py index bee1277..eb928cf 100644 --- a/scripts/generate_toc.py +++ b/scripts/generate_toc.py @@ -25,7 +25,8 @@ def generate_toc(base_dir, project_root, output_file_dir): # To ensure numeric sorting of directories def sort_key(directory_name): - return list(map(int, re.findall(r'\d+', directory_name))) + numbers = tuple(map(int, re.findall(r'\d+', directory_name))) + return (0, numbers, directory_name) if numbers else (1, (), directory_name) sections = {} # {section_level: {subsection_type: [lessons]}} @@ -48,7 +49,7 @@ def generate_toc(base_dir, project_root, output_file_dir): all_dirs.append(os.path.join(item, subitem)) # Sort directories properly by numeric order (non-numeric dirs go to end) - all_dirs = sorted(all_dirs, key=lambda d: sort_key(d) if re.search(r'\d+', d) else [999999]) + all_dirs = sorted(all_dirs, key=sort_key) # Loop over the sorted directories for directory in all_dirs: @@ -142,21 +143,22 @@ def generate_toc_cn(base_dir, project_root, output_file_dir): } subsection_titles = { - "Android": "Android:\n\n", - "GPU": "GPU:\n\n", - "Scheduler": "调度器:\n\n", - "Networking": "网络:\n\n", - "tracing": "Tracing:\n\n", - "Security": "安全:\n\n", - "Features": "特性:\n\n", - "Other": "特性:\n\n" + "Android": "\n\nAndroid:\n\n", + "GPU": "\n\nGPU:\n\n", + "Scheduler": "\n\n调度器:\n\n", + "Networking": "\n\n网络:\n\n", + "Tracing": "\n\n追踪:\n\n", + "Security": "\n\n安全:\n\n", + "Features": "\n\n特性:\n\n", + "Other": "\n\n其他:\n\n" } - subsection_order = ['GPU', 'Scheduler', 'Networking', 'tracing', 'Security', 'Features', 'Other', 'Android'] + subsection_order = ['GPU', 'Scheduler', 'Networking', 'Tracing', 'Security', 'Features', 'Other', 'Android'] # To ensure numeric sorting of directories def sort_key(directory_name): - return list(map(int, re.findall(r'\d+', directory_name))) + numbers = tuple(map(int, re.findall(r'\d+', directory_name))) + return (0, numbers, directory_name) if numbers else (1, (), directory_name) sections = {} # {section_level: {subsection_type: [lessons]}} @@ -179,7 +181,7 @@ def generate_toc_cn(base_dir, project_root, output_file_dir): all_dirs.append(os.path.join(item, subitem)) # Sort directories properly by numeric order (non-numeric dirs go to end) - all_dirs = sorted(all_dirs, key=lambda d: sort_key(d) if re.search(r'\d+', d) else [999999]) + all_dirs = sorted(all_dirs, key=sort_key) # Loop over the sorted directories for directory in all_dirs: diff --git a/scripts/test_check_internal_links.py b/scripts/test_check_internal_links.py new file mode 100644 index 0000000..f0b0a6f --- /dev/null +++ b/scripts/test_check_internal_links.py @@ -0,0 +1,94 @@ +import tempfile +import unittest +from pathlib import Path + +from scripts.check_internal_links import find_broken_links + + +class InternalLinkCheckerTest(unittest.TestCase): + def test_accepts_existing_paths_and_ignores_non_file_links(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + (root / "guide").mkdir() + (root / "guide" / "target file.md").write_text("# Target\n", encoding="utf-8") + (root / "guide_(v1).md").write_text("# Target\n", encoding="utf-8") + (root / "README.md").write_text( + "\n".join( + [ + "[guide](guide/)", + "[encoded](guide/target%20file.md)", + "[with title](guide/target%20file.md \"Target\")", + "[parentheses](guide_(v1).md)", + "[reference][target]", + "[target]: guide/target%20file.md \"Target\"", + "[anchor](#section)", + "[site route](/tutorials/)", + "[external](https://example.com/missing)", + "```markdown", + "[example](missing-example.md)", + "```", + "`[inline example](missing-inline.md)`", + "````markdown", + "```", + "[nested example](missing-nested.md)", + "```", + "````", + ] + ), + encoding="utf-8", + ) + + self.assertEqual(find_broken_links(root), []) + + def test_reports_missing_relative_link_with_source_line(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + (root / "README.md").write_text( + "# Guide\n\n[missing](docs/missing.md)\n", encoding="utf-8" + ) + + broken = find_broken_links(root) + + self.assertEqual(len(broken), 1) + self.assertEqual(broken[0].source, Path("README.md")) + self.assertEqual(broken[0].line, 3) + self.assertEqual(broken[0].destination, "docs/missing.md") + + def test_reports_missing_reference_destination(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + (root / "README.md").write_text( + "[guide][docs]\n[docs]: missing_(v1).md\n", encoding="utf-8" + ) + + broken = find_broken_links(root) + + self.assertEqual(len(broken), 1) + self.assertEqual(broken[0].line, 2) + self.assertEqual(broken[0].destination, "missing_(v1).md") + + def test_closes_fence_only_with_a_long_enough_matching_delimiter(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + (root / "README.md").write_text( + "\n".join( + [ + "````markdown", + "```", + "[example](ignored.md)", + "````", + "[missing](reported.md)", + ] + ), + encoding="utf-8", + ) + + broken = find_broken_links(root) + + self.assertEqual(len(broken), 1) + self.assertEqual(broken[0].line, 5) + self.assertEqual(broken[0].destination, "reported.md") + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test_generate_toc.py b/scripts/test_generate_toc.py new file mode 100644 index 0000000..33d768c --- /dev/null +++ b/scripts/test_generate_toc.py @@ -0,0 +1,49 @@ +import re +import unittest +from pathlib import Path + +from scripts.generate_toc import generate_toc, generate_toc_cn + + +ROOT = Path(__file__).resolve().parent.parent +LINK_PATTERN = re.compile(r"\]\((src/.+?)/README(?:\.zh)?\.md\)") +TRACING_LESSONS = { + "src/30-sslsniff", + "src/31-goroutine", + "src/33-funclatency", + "src/37-uprobe-rust", + "src/39-nginx", + "src/40-mysql", + "src/48-energy", +} + + +def lesson_links(markdown: str) -> set[str]: + return set(LINK_PATTERN.findall(markdown)) + + +def subsection(markdown: str, heading: str, next_heading: str) -> str: + return markdown.split(f"\n\n{heading}:\n\n", 1)[1].split( + f"\n\n{next_heading}:\n\n", 1 + )[0] + + +class TocParityTest(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.english = generate_toc(ROOT / "src", ROOT, ROOT) + cls.chinese = generate_toc_cn(ROOT / "src", ROOT, ROOT) + + def test_english_and_chinese_indexes_contain_the_same_lessons(self): + self.assertSetEqual(lesson_links(self.english), lesson_links(self.chinese)) + + def test_chinese_tracing_subsection_contains_all_tracing_lessons(self): + english_tracing = lesson_links(subsection(self.english, "Tracing", "Security")) + chinese_tracing = lesson_links(subsection(self.chinese, "追踪", "安全")) + + self.assertSetEqual(english_tracing, TRACING_LESSONS) + self.assertSetEqual(chinese_tracing, TRACING_LESSONS) + + +if __name__ == "__main__": + unittest.main() diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 1cdd59e..2043d2b 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -47,9 +47,9 @@ This section covers advanced topics related to eBPF, including using eBPF progra GPU: - [lesson 47-cuda-events](47-cuda-events/README.md) Tracing CUDA GPU Operations -- [lesson xpu/npu-kernel-driver](xpu/npu-kernel-driver/README.md) Tracing Intel NPU Kernel Driver Operations - [xpu flamegraph](xpu/flamegraph/README.md) Building a GPU Flamegraph Profiler with CUPTI - [lesson xpu/gpu-kernel-driver](xpu/gpu-kernel-driver/README.md) Monitoring GPU Driver Activity with Kernel Tracepoints +- [lesson xpu/npu-kernel-driver](xpu/npu-kernel-driver/README.md) Tracing Intel NPU Kernel Driver Operations Scheduler: @@ -95,11 +95,11 @@ Features: - [lesson 36-userspace-ebpf](36-userspace-ebpf/README.md) Userspace eBPF Runtimes: Overview and Applications - [lesson 38-btf-uprobe](38-btf-uprobe/README.md) Expanding eBPF Compile Once, Run Everywhere(CO-RE) to Userspace Compatibility - [lesson 43-kfuncs](43-kfuncs/README.md) Extending eBPF Beyond Its Limits: Custom kfuncs in Kernel Modules +- [features bpf_arena](features/bpf_arena/README.md) BPF Arena for Zero-Copy Shared Memory - [features bpf_iters](features/bpf_iters/README.md) BPF Iterators for Kernel Data Export - [features bpf_token](features/bpf_token/README.md) BPF Token for Delegated Privilege and Secure Program Loading -- [features dynptr](features/dynptr/README.md) BPF Dynamic Pointers for Variable-Length Data - [features bpf_wq](features/bpf_wq/README.md) BPF Workqueues for Asynchronous Sleepable Tasks -- [features bpf_arena](features/bpf_arena/README.md) BPF Arena for Zero-Copy Shared Memory +- [features dynptr](features/dynptr/README.md) BPF Dynamic Pointers for Variable-Length Data - [features struct_ops](features/struct_ops/README.md) Extending Kernel Subsystems with BPF struct_ops Other: diff --git a/src/SUMMARY.zh.md b/src/SUMMARY.zh.md index 64b8bf4..da0d98a 100644 --- a/src/SUMMARY.zh.md +++ b/src/SUMMARY.zh.md @@ -42,16 +42,22 @@ 这一部分涵盖了与 eBPF 相关的高级主题，包括在 Android 上使用 eBPF 程序、利用 eBPF 程序进行的潜在攻击和防御以及复杂的追踪。结合用户模式和内核模式的 eBPF 可以带来强大的能力（也可能带来安全风险）。 + + GPU: - [lesson 47-cuda-events](47-cuda-events/README.zh.md) eBPF 教程：追踪 CUDA GPU 操作 -- [lesson xpu/npu-kernel-driver](xpu/npu-kernel-driver/README.zh.md) eBPF 实例教程：跟踪 Intel NPU 内核驱动操作 - [xpu flamegraph](xpu/flamegraph/README.zh.md) eBPF 示例：使用 CUPTI 构建 GPU 火焰图分析器 - [lesson xpu/gpu-kernel-driver](xpu/gpu-kernel-driver/README.zh.md) eBPF 实例教程：使用内核跟踪点监控 GPU 驱动活动 +- [lesson xpu/npu-kernel-driver](xpu/npu-kernel-driver/README.zh.md) eBPF 实例教程：跟踪 Intel NPU 内核驱动操作 + + 调度器: - [lesson 44-scx-simple](44-scx-simple/README.zh.md) eBPF 教程：BPF 调度器入门 - [lesson 45-scx-nest](45-scx-nest/README.zh.md) eBPF 示例教程：实现 `scx_nest` 调度器 + + 网络: - [lesson 23-http](23-http/README.zh.md) 通过 eBPF socket filter 或 syscall trace 追踪 HTTP 请求等七层协议 - eBPF 实践教程 @@ -60,6 +66,19 @@ GPU: - [lesson 42-xdp-loadbalancer](42-xdp-loadbalancer/README.zh.md) eBPF 开发者教程： 简单的 XDP 负载均衡器 - [lesson 46-xdp-test](46-xdp-test/README.zh.md) eBPF 实例教程：构建高性能 XDP 数据包生成器 - [lesson 50-tcx](50-tcx/README.zh.md) eBPF 入门实践教程第五十篇：使用 TCX Link 实现可组合的流量控制 + + +追踪: + +- [lesson 30-sslsniff](30-sslsniff/README.zh.md) eBPF 实践教程：使用 uprobe 捕获多种库的 SSL/TLS 明文数据 +- [lesson 31-goroutine](31-goroutine/README.zh.md) eBPF 实践教程：使用 eBPF 跟踪 Go 协程状态 +- [lesson 33-funclatency](33-funclatency/README.zh.md) 使用 eBPF 测量函数延迟 +- [lesson 37-uprobe-rust](37-uprobe-rust/README.zh.md) eBPF 实践：使用 Uprobe 追踪用户态 Rust 应用 +- [lesson 39-nginx](39-nginx/README.zh.md) 使用 eBPF 跟踪 Nginx 请求 +- [lesson 40-mysql](40-mysql/README.zh.md) 使用 eBPF 跟踪 MySQL 查询 +- [lesson 48-energy](48-energy/README.zh.md) eBPF 教程：进程级能源监控与功耗分析 + + 安全: - [lesson 24-hide](24-hide/README.zh.md) eBPF 开发实践：使用 eBPF 隐藏进程或文件信息 @@ -68,22 +87,28 @@ GPU: - [lesson 27-replace](27-replace/README.zh.md) 替换任意程序读取或者写入的文本 - [lesson 28-detach](28-detach/README.zh.md) 在应用程序退出后运行 eBPF 程序：eBPF 程序的生命周期 - [lesson 34-syscall](34-syscall/README.zh.md) eBPF 开发实践：使用 eBPF 修改系统调用参数 + + 特性: - [lesson 35-user-ringbuf](35-user-ringbuf/README.zh.md) eBPF开发实践：使用 user ring buffer 向内核异步发送信息 - [lesson 36-userspace-ebpf](36-userspace-ebpf/README.zh.md) 用户空间 eBPF 运行时：深度解析与应用实践 - [lesson 38-btf-uprobe](38-btf-uprobe/README.zh.md) 借助 eBPF 和 BTF，让用户态也能一次编译、到处运行 - [lesson 43-kfuncs](43-kfuncs/README.zh.md) 超越 eBPF 的极限：在内核模块中定义自定义 kfunc +- [features bpf_arena](features/bpf_arena/README.zh.md) eBPF 实例教程：BPF Arena 零拷贝共享内存 - [features bpf_iters](features/bpf_iters/README.zh.md) eBPF 教程：BPF 迭代器用于内核数据导出 - [features bpf_token](features/bpf_token/README.zh.md) eBPF 入门实践教程：BPF Token，安全的委托式权限与程序加载 -- [features dynptr](features/dynptr/README.zh.md) BPF Dynamic Pointers for Variable-Length Data - [features bpf_wq](features/bpf_wq/README.zh.md) eBPF 教程：BPF 工作队列用于异步可睡眠任务 -- [features bpf_arena](features/bpf_arena/README.zh.md) eBPF 实例教程：BPF Arena 零拷贝共享内存 +- [features dynptr](features/dynptr/README.zh.md) BPF Dynamic Pointers for Variable-Length Data - [features struct_ops](features/struct_ops/README.zh.md) eBPF 教程：使用 BPF struct_ops 扩展内核子系统 -特性: + + +其他: - [lesson 49-hid](49-hid/README.zh.md) eBPF 教程：无需内核补丁修复故障的 HID 设备 - [cgroup](cgroup/README.zh.md) eBPF 实例教程：基于 cgroup 的策略控制 + + Android: - [lesson 22-android](22-android/README.zh.md) 在 Android 上使用 eBPF 程序