mirror of
https://github.com/eunomia-bpf/bpf-developer-tutorial.git
synced 2026-02-03 10:14:44 +08:00
update introduction
This commit is contained in:
云微
@@ -1,168 +1,163 @@
|
||||
# eBPF 入门开发实践教程零：介绍 eBPF 的基本概念、常见的开发工具
|
||||
# eBPF 示例教程 0：核心概念与工具简介
|
||||
|
||||
## 1. eBPF 简介：安全和有效地扩展内核
|
||||
这是一个全面的 eBPF 开发教程的第一部分，旨在通过实用的 eBPF 开发指导您从初学者到高级用户。它涵盖了基本概念、实际代码示例以及在现代系统中的应用。我们将不再专注于传统工具如 BCC，而是使用现代框架如 `libbpf`、`Cilium`、`libbpf-rs` 和 eunomia-bpf，并提供 `C`、`Go` 和 `Rust` 的示例。
|
||||
|
||||
eBPF 是一项革命性的技术，起源于 Linux 内核，可以在操作系统的内核中运行沙盒程序。它被用来安全和有效地扩展内核的功能，而不需要改变内核的源代码或加载内核模块。eBPF 通过允许在操作系统内运行沙盒程序，应用程序开发人员可以在运行时，可编程地向操作系统动态添加额外的功能。然后，操作系统保证安全和执行效率，就像在即时编译（JIT）编译器和验证引擎的帮助下进行本地编译一样。eBPF 程序在内核版本之间是可移植的，并且可以自动更新，从而避免了工作负载中断和节点重启。
|
||||
本教程的主要目标是提供清晰简洁的 eBPF 工具示例（起步只需 20 行代码！），帮助开发者快速掌握基本的 eBPF 开发技术。每个示例都是独立的，可以在目录结构中找到，每个目录代表一个独立的 eBPF 工具。您还可以访问我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或网站 <https://eunomia.dev/tutorials/> 获取更多示例和完整的教程源代码。
|
||||
|
||||
今天，eBPF 被广泛用于各类场景：在现代数据中心和云原生环境中，可以提供高性能的网络包处理和负载均衡；以非常低的资源开销，做到对多种细粒度指标的可观测性，帮助应用程序开发人员跟踪应用程序，为性能故障排除提供洞察力；保障应用程序和容器运行时的安全执行，等等。可能性是无穷的，而 eBPF 在操作系统内核中所释放的创新才刚刚开始[3]。
|
||||
## eBPF 简介：安全高效的内核扩展
|
||||
|
||||
### eBPF 的未来：内核的 JavaScript 可编程接口
|
||||
eBPF（扩展的 Berkeley Packet Filter）是一项突破性的技术，允许开发者在内核空间中安全高效地运行小型程序。与传统方法需要修改内核源代码或加载新模块不同，eBPF 使得动态定制和优化网络行为成为可能，且不会中断系统操作。这种灵活性和高效性使 eBPF 成为克服传统网络栈限制的关键技术。
|
||||
|
||||
对于浏览器而言，JavaScript 的引入带来的可编程性开启了一场巨大的革命，使浏览器发展成为几乎独立的操作系统。现在让我们回到 eBPF：为了理解 eBPF 对 Linux 内核的可编程性影响，对 Linux 内核的结构以及它如何与应用程序和硬件进行交互有一个高层次的理解是有帮助的[4]。
|
||||
### eBPF 的强大之处是什么？
|
||||
|
||||
|
||||
- **直接内核交互**：eBPF 程序在内核中执行，与系统级事件如网络包、系统调用或追踪点交互。
|
||||
- **安全执行**：eBPF 通过验证器在程序运行前检查其逻辑，防止潜在的内核崩溃或安全漏洞。
|
||||
- **最低开销**：eBPF 通过使用即时编译器（JIT），将 eBPF 字节码转换为针对特定架构的优化机器码，实现近原生执行速度。
|
||||
|
||||
Linux 内核的主要目的是抽象出硬件或虚拟硬件，并提供一个一致的 API（系统调用），允许应用程序运行和共享资源。为了实现这个目的，我们维护了一系列子系统和层，以分配这些责任[5]。每个子系统通常允许某种程度的配置，以考虑到用户的不同需求。如果不能配置所需的行为，就需要改变内核，从历史上看，改变内核的行为，或者让用户编写的程序能够在内核中运行，就有两种选择:
|
||||
## eBPF：过去、现在与未来
|
||||
|
||||
| 本地支持内核模块 | 写一个内核模块 |
|
||||
| ----------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
|
||||
| 改变内核源代码，并说服Linux内核社区相信这种改变是必要的。等待几年，让新的内核版本成为一种商品。 | 定期修复它，因为每个内核版本都可能破坏它。由于缺乏安全边界，冒着破坏你的Linux内核的风险 |
|
||||
### 过去：可编程网络的变革
|
||||
|
||||
实际上，两种方案都不常用，前者成本太高，后者则几乎没有可移植性。
|
||||
eBPF 于 2014 年推出，彻底改变了开发者处理网络的方式，允许小型可编程内核空间应用程序实时处理数据包。通过钩住关键内核点，eBPF 使得在网络包到达时应用自定义逻辑成为可能，从而提高了效率和灵活性。这使得组织能够在不需要自定义驱动程序或修改内核的情况下定制网络行为，为云原生和数据中心环境创造了理想的解决方案。
|
||||
|
||||
有了 eBPF，就有了一个新的选择，可以重新编程 Linux 内核的行为，而不需要改变内核的源代码或加载内核模块，同时保证在不同内核版本之间一定程度上的行为一致性和兼容性、以及安全性[6]。为了实现这个目的，eBPF 程序也需要有一套对应的 API，允许用户定义的应用程序运行和共享资源 --- 换句话说，某种意义上讲 eBPF 虚拟机也提供了一套类似于系统调用的机制，借助 eBPF 和用户态通信的机制，Wasm 虚拟机和用户态应用也可以获得这套“系统调用”的完整使用权，一方面能可编程地扩展传统的系统调用的能力，另一方面能在网络、文件系统等许多层次实现更高效的可编程 IO 处理。
|
||||
### 现在：满足现代计算需求的多功能框架
|
||||
|
||||
|
||||
eBPF 已发展为一个多功能框架，超越了其最初的网络用途，现在涵盖了可观测性、追踪、安全性，甚至系统资源管理。eBPF 程序可以动态钩住内核事件，赋予开发者精确控制系统行为和性能优化的能力，而无需修改内核或重启系统。这使得 eBPF 成为系统管理员和开发者监控、优化和保护环境的必备工具。
|
||||
|
||||
正如上图所示，当今的 Linux 内核正在向一个新的内核模型演化：用户定义的应用程序可以在内核态和用户态同时执行，用户态通过传统的系统调用访问系统资源，内核态则通过 BPF Helper Calls 和系统的各个部分完成交互。截止 2023 年初，内核中的 eBPF 虚拟机中已经有 220 多个 Helper 系统接口，涵盖了非常多的应用场景。
|
||||
以下是 eBPF 目前广泛应用的一些关键领域：
|
||||
|
||||
值得注意的是，BPF Helper Call 和系统调用二者并不是竞争关系，它们的编程模型和有性能优势的场景完全不同，也不会完全替代对方。对 Wasm 和 Wasi 相关生态来说，情况也类似，专门设计的 Wasi 接口需要经历一个漫长的标准化过程，但可能在特定场景能为用户态应用获取更佳的性能和可移植性保证，而 eBPF 在保证沙箱本质和可移植性的前提下，可以提供一个快速灵活的扩展系统接口的方案。
|
||||
- **网络**：eBPF 提供内核中实时、高速的数据包过滤和处理，允许创建自定义协议解析器和网络策略，无需新驱动程序或系统重启。这在云和数据中心环境中实现了高效的网络管理。
|
||||
|
||||
- **可观测性**：eBPF 使开发者能够通过收集自定义指标和执行内核级数据聚合来深入了解系统行为。通过利用内核追踪点和函数调用，eBPF 有助于识别性能问题和定位难以发现的错误。
|
||||
|
||||
- **追踪与分析**：eBPF 提供强大的追踪和分析能力，通过附加到内核函数、追踪点甚至用户空间探针，使开发者能够深入了解系统和应用程序的行为，从而优化性能和解决复杂的系统问题。
|
||||
|
||||
- **安全**：eBPF 在实时安全监控中发挥重要作用。它能够深入检查系统调用、网络流量和其他内核活动，帮助执行动态安全策略和检测异常行为，为基础设施提供高效的保护。
|
||||
|
||||
- **调度器优化**：eBPF 越来越多地用于增强 CPU 调度，能够监控 CPU 负载并优化任务在核心之间的分配。这可以更有效地利用 CPU 资源，提高系统响应能力。
|
||||
|
||||
- **HID（人机接口设备）驱动增强**：开发者使用 eBPF 优化键盘、鼠标和触摸屏等设备的 HID 驱动程序。通过为处理输入事件添加自定义逻辑，eBPF 提高了对延迟敏感应用的响应速度。
|
||||
|
||||
目前的 eBPF 仍然处于早期阶段，但是借助当前 eBPF 提供的内核接口和用户态交互的能力，经由 Wasm-bpf 的系统接口转换，Wasm 虚拟机中的应用已经几乎有能力获取内核以及用户态任意一个函数调用的数据和返回值（kprobe，uprobe...）；以很低的代价收集和理解所有系统调用，并获取所有网络操作的数据包和套接字级别的数据（tracepoint，socket...）；在网络包处理解决方案中添加额外的协议分析器，并轻松地编程任何转发逻辑（XDP，TC...），以满足不断变化的需求，而无需离开Linux内核的数据包处理环境。
|
||||
各行业组织已大规模采用 eBPF：
|
||||
|
||||
不仅如此，eBPF 还有能力往用户空间任意进程的任意地址写入数据（bpf_probe_write_user[7]），有限度地修改内核函数的返回值（bpf_override_return[8]），甚至在内核态直接执行某些系统调用[9]；所幸的是，eBPF 在加载进内核之前对字节码会进行严格的安全检查，确保没有内存越界等操作，同时，许多可能会扩大攻击面、带来安全风险的功能都是需要在编译内核时明确选择启用才能使用的；在 Wasm 虚拟机将字节码加载进内核之前，也可以明确选择启用或者禁用某些 eBPF 功能，以确保沙箱的安全性。
|
||||
- **Google**：使用 eBPF 进行安全审计、数据包处理、实时性能监控以及优化其庞大基础设施的 CPU 调度。
|
||||
- **Netflix**：利用 eBPF 进行网络流量分析，确保流媒体服务的高可用性和性能。
|
||||
- **Android**：应用 eBPF 优化网络使用、功耗和资源分配，提升数百万设备的性能和电池寿命。
|
||||
- **S&P Global**：通过 **Cilium** 使用 eBPF 管理跨多个云和本地系统的网络，确保可扩展性和安全性。
|
||||
- **Shopify**：与 **Falco** 一起实施 eBPF 进行入侵检测，增强其电子商务平台的安全性。
|
||||
- **Cloudflare**：使用 eBPF 进行网络可观测性、安全监控和性能优化，保护全球数百万网站。
|
||||
|
||||
除了内核态的 eBPF 运行时，eBPF 也可以拓展到用户空间，例如 [bpftime](https://github.com/eunomia-bpf/bpftime)，实现更高性能的用户态追踪、性能分析、插件等等。
|
||||
eBPF 能够动态调整系统行为并扩展到用户空间，使其成为现代计算不可或缺的技术。无论是优化网络流量、提升安全性，还是增强系统性能，eBPF 都能帮助开发者高效、安全地应对实时需求。
|
||||
|
||||
## 2. 关于如何学习 eBPF 相关的开发的一些建议
|
||||
除了其内核模式运行时，eBPF 还可以扩展到用户空间。例如，[bpftime](https://github.com/eunomia-bpf/bpftime) 是一个用户空间 eBPF 运行时，允许在用户空间应用中进行高性能追踪、性能分析和插件支持。这种 eBPF 向用户空间的扩展有助于在各种超越内核级任务的用例中提高灵活性和性能。
|
||||
|
||||
本文不会对 eBPF 的原理做更详细的介绍，不过这里有一个学习规划和参考资料，也许会有一些价值：
|
||||
### 未来：eBPF 的扩展潜力
|
||||
|
||||
### eBPF 入门（5-7h）
|
||||
展望未来，预计 eBPF 将成为操作系统更为重要的一部分。重点将放在提升其灵活性、模块化和易用性上，使其能够应用于更广泛的场景。内存管理、并发机制的创新以及与用户空间应用的更好集成已在路上。已经有项目在编译 Linux 内核的关键部分到 BPF 指令集，这可能彻底改变内核开发和分析的方式。
|
||||
|
||||
- Google 或者其他搜索引擎查找：eBPF
|
||||
- 询问 ChatGPT 之类的东西：eBPF 是什么？
|
||||
动态栈、更好的用户空间可观测性工具（例如快速 Uprobes 和特定语言的栈行走器）以及更安全的程序终止机制等进展将继续增强 eBPF 的可靠性并扩展其使用场景。此外，新工具和库将简化 eBPF 开发，降低内核和应用开发者的入门门槛。
|
||||
|
||||
## 开始学习教程
|
||||
|
||||
本教程提供实用的 eBPF 开发实践，涵盖从初级到高级的主题。我们专注于在可观测性、网络和安全等领域的动手示例，使用 `libbpf`、`libbpf-rs` 和 `eunomia-bpf` 等框架，并提供 C、Go 和 Rust 的示例。
|
||||
|
||||
### 本教程适合谁？
|
||||
|
||||
- **开发者** 希望实现自定义内核解决方案。
|
||||
- **系统管理员** 旨在提升性能和安全性。
|
||||
- **技术爱好者** 探索前沿的内核技术。
|
||||
|
||||
### 你将学到什么？
|
||||
|
||||
- **核心概念**：eBPF 基础知识及其与 Linux 内核的集成。
|
||||
- **实用技能**：编写和部署 eBPF 程序。
|
||||
- **高级主题**：探索 eBPF 在安全、追踪和未来创新方面的应用。
|
||||
|
||||
---
|
||||
|
||||
## 目录
|
||||
|
||||
1. **eBPF 简介**
|
||||
基本概念和入门所需的工具。
|
||||
|
||||
2. **初学者示例**
|
||||
简单的程序，如“Hello World”及使用 kprobe 和 uprobe 进行基础追踪。
|
||||
|
||||
3. **可观测性**
|
||||
侧重于使用 eBPF 监控网络流量、文件操作和进程行为的示例。
|
||||
|
||||
4. **网络**
|
||||
侧重于修改和优化网络流量的示例，如 XDP、TC 和 socket。
|
||||
|
||||
5. **安全**
|
||||
用于隐藏进程和文件、发送信号杀死进程以及跟踪进程事件以增强安全性的程序。
|
||||
|
||||
6. **高级用例**
|
||||
涉及性能分析、调度器优化和用户空间 eBPF（如 bpftime）的复杂示例。
|
||||
|
||||
7. **深入主题**
|
||||
探索 eBPF 在 Android 上的应用、使用 eBPF 进行网络加速以及通过系统调用修改来保护系统。
|
||||
|
||||
## 如何使用 eBPF 编程
|
||||
|
||||
从头编写 eBPF 程序可能较为复杂。为简化这一过程，LLVM 于 2015 年引入了将高级语言代码编译为 eBPF 字节码的能力。自那时起，eBPF 社区构建了像 `libbpf` 这样的库来管理这些程序。这些库帮助将 eBPF 字节码加载到内核中并执行基本任务。Linux 内核源代码中 `samples/bpf/` 目录包含了众多 eBPF 示例。
|
||||
|
||||
典型的 eBPF 程序包含两个部分：内核空间代码（`*_kern.c`）和用户空间代码（`*_user.c`）。内核空间代码定义逻辑，而用户空间代码负责加载和与内核交互。然而，像 `libbpf-bootstrap` 和 Go eBPF 库这样的工具简化了这一过程，允许一次性编译和更容易的开发。
|
||||
|
||||
### eBPF 开发工具
|
||||
|
||||
- **BCC**：一个基于 Python 的工具链，简化了 eBPF 程序的编写、编译和加载。它提供了许多预构建的追踪工具，但在依赖和兼容性方面存在一些限制。
|
||||
- **eBPF Go 库**：一个 Go 库，解耦了获取 eBPF 字节码的过程与加载和管理 eBPF 程序的过程。
|
||||
- **libbpf-bootstrap**：基于 `libbpf` 的现代脚手架，提供了高效的工作流用于编写 eBPF 程序，提供简单的一次性编译过程以生成可重用的字节码。
|
||||
- **eunomia-bpf**：一个用于编写仅包含内核空间代码的 eBPF 程序的工具链。它通过动态加载 eBPF 程序简化了 eBPF 程序的开发。
|
||||
|
||||
这些工具有助于减少开发 eBPF 程序的复杂性，使开发者更容易优化系统性能、安全性和可观测性。
|
||||
|
||||
## 学习 eBPF 开发的一些技巧
|
||||
|
||||
本文不会提供更详细的 eBPF 原理介绍，但以下是一个学习计划和参考资料，可能对您有帮助：
|
||||
|
||||
### eBPF 简介（5-7 小时）
|
||||
|
||||
- 使用 Google 或其他搜索引擎搜索：eBPF
|
||||
- 询问类似 ChatGPT 的工具：什么是 eBPF？
|
||||
|
||||
推荐：
|
||||
|
||||
- 阅读 ebpf 简介：<https://ebpf.io/> （30min）
|
||||
- 简要了解一下 ebpf 内核相关文档：<https://prototype-kernel.readthedocs.io/en/latest/bpf/> （知道有问题去哪里查询，30min）
|
||||
- 阅读 ebpf 中文入门指南：<https://www.ebpf.top/post/ebpf_intro> （1h）
|
||||
- 有大量的参考资料：<https://github.com/zoidbergwill/awesome-ebpf> （2-3h）
|
||||
- 可以选自己感兴趣的 PPT 翻一翻：<https://github.com/gojue/ebpf-slide> （1-2h）
|
||||
- 阅读 eBPF 介绍：<https://ebpf.io/>（30 分钟）
|
||||
- 简要了解 eBPF 内核相关文档：<https://docs.ebpf.io/>（了解技术细节的查询来源，30 分钟）
|
||||
|
||||
回答三个问题：
|
||||
|
||||
1. 了解 eBPF 是什么东西？为啥要有这个玩意，不能用内核模块？
|
||||
2. 它有什么功能？能在 Linux 内核里面完成哪些事情？有哪些 eBPF 程序的类型和 helper（不需要知道全部，但是需要知道去哪里找）？
|
||||
3. 能拿来做什么？比如说在哪些场景中进行运用？网络、安全、可观测性？
|
||||
1. 了解 eBPF 是什么？我们为什么需要它？难道不能使用内核模块吗？
|
||||
2. 它有哪些功能？它在 Linux 内核中能做什么？eBPF 程序和助手函数有哪些类型（不需要全部了解，但需要知道在哪里查找）？
|
||||
3. 它能用于哪些场景？例如，可以在哪些情况下使用？网络、安全、可观测性？
|
||||
|
||||
### 了解如何开发 eBPF 程序（10-15h）
|
||||
### 理解如何开发 eBPF 程序（10-15 小时）
|
||||
|
||||
了解并尝试一下 eBPF 开发框架：
|
||||
了解并尝试 eBPF 开发框架：
|
||||
|
||||
- bpftrace 教程，对于最简单的应用来说，bpftrace 可能是最方便的：<https://eunomia.dev/zh/tutorials/bpftrace-tutorial/> （试试，1h）
|
||||
- BCC 开发各类小工具的例子：<https://github.com/iovisor/bcc/blob/master/docs/tutorial_bcc_python_developer.md> （跑一遍，3-4h）
|
||||
- libbpf 的一些例子：<https://github.com/libbpf/libbpf-bootstrap> （选感兴趣的运行一下，并阅读一下源代码，2h）
|
||||
- 基于 C 语言 libbpf, Go 语言或者 Rust 语言和 eunomia-bpf 的教程：<https://github.com/eunomia-bpf/bpf-developer-tutorial> （阅读 1-20 的部分，3-8h）
|
||||
- bpftrace 教程：<https://eunomia.dev/tutorials/bpftrace-tutorial/>（尝试，1 小时）
|
||||
- 使用 BCC 开发各种工具的示例：<https://github.com/iovisor/bcc/blob/master/docs/tutorial_bcc_python_developer.md>（运行，3-4 小时）
|
||||
- libbpf 的一些示例：<https://github.com/libbpf/libbpf-bootstrap>（运行任何有趣的示例并阅读源代码，2 小时）
|
||||
- 教程：<https://github.com/eunomia-bpf/bpf-developer-tutorial>（阅读第 1-10 部分，3-4 小时）
|
||||
|
||||
有任何问题或者想了解的东西，不管是不是和本项目相关，都可以在本项目的 discussions 里面开始讨论。
|
||||
其他开发框架：Go 或 Rust 语言，请自行搜索和尝试（0-2 小时）
|
||||
|
||||
回答一些问题，并且进行一些尝试（2-5h）：
|
||||
如果有问题或想了解的内容，无论是否与本项目相关，都可以在该项目的讨论区开始讨论。
|
||||
|
||||
1. 如何开发一个最简单的 eBPF 程序？
|
||||
2. 如何用 eBPF 追踪一个内核功能或函数？有很多种方法，举出对应的代码；
|
||||
3. 有哪些方案能通过用户态和内核态通信？如何从用户态向内核态传送信息？如何从内核态向用户态传递信息？举出代码示例；
|
||||
4. 编写一个你自己的 eBPF 程序，实现一个功能；
|
||||
5. eBPF 程序的整个生命周期里面，分别在用户态和内核态做了哪些事情？
|
||||
回答一些问题并尝试一些实验（2-5 小时）：
|
||||
|
||||
## 3. 如何使用 eBPF 编程
|
||||
|
||||
原始的 eBPF 程序编写是非常繁琐和困难的。为了改变这一现状，llvm 于 2015 年推出了可以将由高级语言编写的代码编译为 eBPF 字节码的功能，同时，eBPF 社区将 `bpf()` 等原始的系统调用进行了初步地封装，给出了 `libbpf` 库。这些库会包含将字节码加载到内核中的函数以及一些其他的关键函数。在 Linux 的源码包的 `samples/bpf/` 目录下，有大量 Linux 提供的基于 `libbpf` 的 eBPF 样例代码。
|
||||
|
||||
一个典型的基于 `libbpf` 的 eBPF 程序具有 `*_kern.c` 和 `*_user.c` 两个文件，`*_kern.c` 中书写在内核中的挂载点以及处理函数，`*_user.c` 中书写用户态代码，完成内核态代码注入以及与用户交互的各种任务。 更为详细的教程可以参考[该视频](https://www.bilibili.com/video/BV1f54y1h74r?spm_id_from=333.999.0.0)。然而由于该方法仍然较难理解且入门存在一定的难度，因此现阶段的eBPF程序开发大多基于一些工具，比如：
|
||||
|
||||
- BCC
|
||||
- BPFtrace
|
||||
- libbpf-bootstrap
|
||||
- Go eBPF library
|
||||
|
||||
以及还有比较新的工具，例如 `eunomia-bpf`.
|
||||
|
||||
## 编写 eBPF 程序
|
||||
|
||||
eBPF 程序由内核态部分和用户态部分构成。内核态部分包含程序的实际逻辑，用户态部分负责加载和管理内核态部分。使用 eunomia-bpf 开发工具，只需编写内核态部分的代码。
|
||||
|
||||
内核态部分的代码需要符合 eBPF 的语法和指令集。eBPF 程序主要由若干个函数组成，每个函数都有其特定的作用。可以使用的函数类型包括：
|
||||
|
||||
- kprobe：插探函数，在指定的内核函数前或后执行。
|
||||
- tracepoint：跟踪点函数，在指定的内核跟踪点处执行。
|
||||
- raw_tracepoint：原始跟踪点函数，在指定的内核原始跟踪点处执行。
|
||||
- xdp：网络数据处理函数，拦截和处理网络数据包。
|
||||
- perf_event：性能事件函数，用于处理内核性能事件。
|
||||
- kretprobe：函数返回插探函数，在指定的内核函数返回时执行。
|
||||
- tracepoint_return：跟踪点函数返回，在指定的内核跟踪点返回时执行。
|
||||
- raw_tracepoint_return：原始跟踪点函数返回，在指定的内核原始跟踪
|
||||
|
||||
### BCC
|
||||
|
||||
BCC 全称为 BPF Compiler Collection，该项目是一个 python 库，
|
||||
包含了完整的编写、编译、和加载 BPF 程序的工具链，以及用于调试和诊断性能问题的工具。
|
||||
|
||||
自 2015 年发布以来，BCC 经过上百位贡献者地不断完善后，目前已经包含了大量随时可用的跟踪工具。[其官方项目库](https://github.com/iovisor/bcc/blob/master/docs/tutorial.md)
|
||||
提供了一个方便上手的教程，用户可以快速地根据教程完成 BCC 入门工作。
|
||||
|
||||
用户可以在 BCC 上使用 Python、Lua 等高级语言进行编程。
|
||||
相较于使用 C 语言直接编程，这些高级语言具有极大的便捷性，用户只需要使用 C 来设计内核中的
|
||||
BPF 程序，其余包括编译、解析、加载等工作在内，均可由 BCC 完成。
|
||||
|
||||
然而使用 BCC 存在一个缺点便是在于其兼容性并不好。基于 BCC 的
|
||||
eBPF 程序每次执行时候都需要进行编译，编译则需要用户配置相关的头文件和对应实现。在实际应用中，
|
||||
相信大家也会有体会，编译依赖问题是一个很棘手的问题。也正是因此，在本项目的开发中我们放弃了 BCC，
|
||||
选择了可以做到一次编译-多次运行的 libbpf-bootstrap 工具。
|
||||
|
||||
### eBPF Go library
|
||||
|
||||
eBPF Go 库提供了一个通用的 eBPF 库，它解耦了获取 eBPF 字节码的过程和 eBPF 程序的加载和管理，并实现了类似 libbpf 一样的 CO- 功能。eBPF 程序通常是通过编写高级语言创建的，然后使用 clang/LLVM 编译器编译为 eBPF 字节码。
|
||||
|
||||
### libbpf
|
||||
|
||||
`libbpf-bootstrap` 是一个基于 `libbpf` 库的 BPF 开发脚手架，从其
|
||||
[github](https://github.com/libbpf/libbpf-bootstrap) 上可以得到其源码。
|
||||
|
||||
`libbpf-bootstrap` 综合了 BPF 社区过去多年的实践，为开发者提了一个现代化的、便捷的工作流，实
|
||||
现了一次编译，重复使用的目的。
|
||||
|
||||
基于 `libbpf-bootstrap` 的 BPF 程序对于源文件有一定的命名规则，
|
||||
用于生成内核态字节码的 bpf 文件以 `.bpf.c` 结尾，用户态加载字节码的文件以 `.c` 结尾，且这两个文件的
|
||||
前缀必须相同。
|
||||
|
||||
基于 `libbpf-bootstrap` 的 BPF 程序在编译时会先将 `*.bpf.c` 文件编译为
|
||||
对应的 `.o` 文件，然后根据此文件生成 `skeleton` 文件，即 `*.skel.h`，这个文件会包含内核态中定义的一些
|
||||
数据结构，以及用于装载内核态代码的关键函数。在用户态代码 `include` 此文件之后调用对应的装载函数即可将
|
||||
字节码装载到内核中。同样的，`libbpf-bootstrap` 也有非常完备的入门教程，用户可以在[该处](https://nakryiko.com/posts/libbpf-bootstrap/)
|
||||
得到详细的入门操作介绍。
|
||||
|
||||
### eunomia-bpf
|
||||
|
||||
[eunomia-bpf](https://github.com/eunomia-bpf/eunomia-bpf) 是一个开源的 eBPF 动态加载运行时和开发工具链，是为了简化 eBPF 程序的开发、构建、分发、运行而设计的，基于 libbpf 的 CO-RE 轻量级开发框架。
|
||||
|
||||
使用 eunomia-bpf ，可以：
|
||||
|
||||
- 在编写 eBPF 程序或工具时只编写内核态代码，自动获取内核态导出信息，并作为模块动态加载；
|
||||
- eunomia-bpf 可以将预编译的 eBPF 程序打包为通用的 JSON 或 WASM 模块，跨架构和内核版本进行分发，无需重新编译即可动态加载运行。
|
||||
|
||||
eunomia-bpf 由一个编译工具链和一个运行时库组成, 对比传统的 BCC、原生 libbpf 等框架，简化了 eBPF 程序的开发流程，在大多数时候只需编写内核态代码，即可轻松构建、打包、发布完整的 eBPF 应用。
|
||||
|
||||
> - eunomia-bpf 项目 Github 地址: <https://github.com/eunomia-bpf/eunomia-bpf>
|
||||
> - gitee 镜像: <https://gitee.com/anolis/eunomia>
|
||||
1. 如何开发最简单的 eBPF 程序？
|
||||
2. 如何使用 eBPF 追踪内核功能或函数？有很多方法，提供相应的代码示例；
|
||||
3. 用户模式和内核模式之间的通信解决方案有哪些？如何将信息从用户模式发送到内核模式？如何将信息从内核模式传递到用户模式？提供代码示例；
|
||||
4. 编写您自己的 eBPF 程序以实现某个功能；
|
||||
5. 在 eBPF 程序的整个生命周期中，用户模式和内核模式分别做了什么？
|
||||
|
||||
## 参考资料
|
||||
|
||||
- eBPF 介绍：<https://ebpf.io/>
|
||||
- BPF Compiler Collection (BCC)：<https://github.com/iovisor/bcc>
|
||||
- BPF 编译器集合（BCC）：<https://github.com/iovisor/bcc>
|
||||
- eunomia-bpf：<https://github.com/eunomia-bpf/eunomia-bpf>
|
||||
|
||||
您还可以访问我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或网站 <https://eunomia.dev/zh/tutorials/> 以获取更多示例和完整的教程源代码。我们会继续分享更多有关 eBPF 开发实践的内容，帮助您更好地理解和掌握 eBPF 技术。
|
||||
|
||||
> 原文地址：<https://eunomia.dev/zh/tutorials/0-introduce/> 转载请注明
|
||||
您还可以访问我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或网站 <https://eunomia.dev/tutorials/> 获取更多示例和完整的教程源代码。所有内容均为开源。我们将继续分享更多关于 eBPF 开发实践的内容，帮助您更好地理解和掌握 eBPF 技术。
|
||||
@@ -1,33 +1,118 @@
|
||||
# eBPF Tutorial by Example 0: Introduction to Core Concepts and Tools
|
||||
|
||||
This is the first part of a comprehensive development tutorial for eBPF, designed to guide you through practical eBPF development, from beginner to advanced. It covers fundamental concepts, real-world code examples, and applications in modern systems. Rather than focusing on traditional tools like BCC, we will use modern frameworks such as `libbpf`, `Cilium`, `libbpf-rs`, and eunomia-bpf, with examples provided in `C`, `Go`, and `Rust`.
|
||||
|
||||
The primary goal of this tutorial is to provide clear and concise examples of eBPF tools (starting with as little as 20 lines of code!) to help developers quickly grasp essential eBPF development techniques. Each example is self-contained and can be found in the directory structure, with every directory representing an independent eBPF tool. You can also visit our tutorial code repository <https://github.com/eunomia-bpf/bpf-developer-tutorial> or website <https://eunomia.dev/tutorials/> for more examples and complete tutorial source code.
|
||||
|
||||
## Introduction to eBPF: Secure and Efficient Kernel Extension
|
||||
|
||||
eBPF is a revolutionary technology that originated in the Linux kernel and allows sandbox programs to run in the kernel of an operating system. It is used to securely and efficiently extend the functionality of the kernel without the need to modify the kernel's source code or load kernel modules. By allowing the execution of sandbox programs in the operating system, eBPF enables application developers to dynamically add additional functionality to the operating system at runtime. The operating system then ensures security and execution efficiency, similar to performing native compilation with the help of a Just-In-Time (JIT) compiler and verification engine. eBPF programs are portable between kernel versions and can be automatically updated, avoiding workload interruptions and node restarts.
|
||||
eBPF (extended Berkeley Packet Filter) is a groundbreaking technology that allows developers to run small programs directly in kernel space, safely and efficiently. Unlike traditional approaches that required modifying kernel source code or loading new modules, eBPF made it possible to customize and optimize network behavior dynamically, all without disrupting system operations. This flexibility and efficiency made eBPF a pivotal technology for overcoming the limitations of traditional networking stacks.
|
||||
|
||||
Today, eBPF is widely used in various scenarios: in modern data centers and cloud-native environments, it provides high-performance network packet processing and load balancing; it achieves observability of various fine-grained metrics with very low resource overhead, helping application developers trace applications and provide insights for performance troubleshooting; it ensures secure execution of application and container runtimes, and more. The possibilities are endless, and the innovation unleashed by eBPF in the operating system kernel has only just begun [3].
|
||||
### What Makes eBPF So Powerful?
|
||||
|
||||
### The Future of eBPF: Kernel's JavaScript Programmable Interface
|
||||
- **Direct Kernel Interaction:** eBPF programs execute within the kernel, interacting with system-level events such as network packets, system calls, or tracepoints.
|
||||
- **Safe Execution:** eBPF ensures safety through a verifier that checks the logic of the program before it runs, preventing potential kernel crashes or security breaches.
|
||||
- **Minimal Overhead:** eBPF achieves near-native execution speed by employing a Just-In-Time (JIT) compiler, which translates eBPF bytecode into optimized machine code for the specific architecture.
|
||||
|
||||
For browsers, the introduction of JavaScript brought programmability and initiated a tremendous revolution, transforming browsers into almost independent operating systems. Now let's return to eBPF: in order to understand the programmability impact of eBPF on the Linux kernel, it is helpful to have a high-level understanding of the structure of the Linux kernel and how it interacts with applications and hardware [4].
|
||||
## eBPF: Past, Present, and Future
|
||||
|
||||
|
||||
### Past: Programmable Networking Transformed
|
||||
|
||||
The main purpose of the Linux kernel is to abstract hardware or virtual hardware and provide a consistent API (system calls) that allows applications to run and share resources. To achieve this goal, we maintain a series of subsystems and layers to distribute these responsibilities [5]. Each subsystem typically allows some level of configuration to cater to different user requirements. If the desired behavior cannot be achieved through configuration, there are two options: either modify the kernel source code and convince the Linux kernel community that the change is necessary (waiting for several years for the new kernel version to become a commodity), or write a kernel module and regularly fix it because each kernel version may break it. In practice, neither of the two solutions is commonly used: the former is too costly, and the latter lacks portability.
|
||||
When eBPF was introduced in 2014, it revolutionized how developers approached networking by allowing small, programmable kernel-space applications to handle packet processing in real time. By hooking into key kernel points, eBPF enabled custom logic to be applied whenever a network packet arrived, leading to higher efficiency and flexibility. This allowed organizations to tailor networking behavior without the overhead of custom drivers or kernel modifications, creating an ideal solution for cloud-native and data-center environments.
|
||||
### Present: A Versatile Framework for Modern Computing Needs
|
||||
|
||||
With eBPF, there is a new option to reprogram the behavior of the Linux kernel without modifying the kernel's source code or loading kernel modules, while ensuring a certain degree of behavioral consistency, compatibility, and security across different kernel versions [6]. To achieve this, eBPF programs also need a corresponding API that allows the execution and sharing of resources for user-defined applications. In other words, in a sense, the eBPF virtual machine also provides a mechanism similar to system calls. With the help of the communication mechanism between eBPF and user space, both the Wasm virtual machine and user space applications can have full access to this set of "system calls." On the one hand, it can programmatically extend the capabilities of traditional system calls, and on the other hand, it can achieve more efficient programmable IO processing in many layers such as networking and file systems.
|
||||
eBPF has evolved into a versatile framework that extends beyond its original purpose of networking, now encompassing observability, tracing, security, and even system resource management. eBPF programs can dynamically hook into kernel events, giving developers precise control over system behavior and performance optimization without requiring kernel modifications or reboots. This makes eBPF an essential tool for system administrators and developers who aim to monitor, optimize, and secure their environments.
|
||||
|
||||
|
||||
Here are some key areas where eBPF is widely used today:
|
||||
|
||||
As shown in the above figure, the Linux kernel of today is evolving towards a new kernel model: user-defined applications can run in both kernel space and user space, with user space accessing system resources through traditional system calls and kernel space interacting with various parts of the system through BPF Helper Calls. As of early 2023, there are already more than 220 helper system interfaces in the eBPF virtual machine in the kernel, covering a wide range of application scenarios.## Note
|
||||
It is worth noting that BPF Helper Call and system calls are not competitive. Their programming models and performance advantages are completely different and they do not completely replace each other. For the Wasm and Wasi related ecosystems, the situation is similar. The specially designed wasi interface needs to go through a long standardization process. However, it may provide better performance and portability guarantees for user-mode applications in specific scenarios. On the other hand, eBPF can provide a fast and flexible solution for extending system interfaces, while ensuring sandbox nature and portability.
|
||||
- **Networking:** eBPF offers real-time, high-speed packet filtering and processing within the kernel, allowing for the creation of custom protocol parsers and network policies without needing new drivers or system restarts. This enables highly efficient network management in cloud and data center environments.
|
||||
|
||||
Currently, eBPF is still in the early stages. However, with the help of the kernel interfaces provided by eBPF and the ability to interact with user space, applications in the Wasm virtual machine can almost access the data and return values of any kernel or user mode function call (kprobe, uprobe...). It can collect and understand all system calls at a low cost and obtain packet-level data and socket-level data for all network operations (tracepoint, socket...). It can also add additional protocol analyzers and easily program any forwarding logic in the network packet processing solution (XDP, TC...), without leaving the packet processing environment of the Linux kernel.
|
||||
- **Observability:** eBPF enables developers to gather detailed insights into system behavior by collecting custom metrics and performing in-kernel data aggregation. By tapping into kernel tracepoints and function calls, eBPF helps identify performance issues and track down elusive bugs.
|
||||
|
||||
Moreover, eBPF has the ability to write data to any address of a user space process (bpf_probe_write_user[7]), partially modify the return value of a kernel function (bpf_override_return[8]), and even directly execute certain system calls in kernel mode[9]. Fortunately, eBPF performs strict security checks on the bytecode before loading it into the kernel to ensure that there are no operations such as memory out-of-bounds. Moreover, many features that may expand the attack surface and pose security risks need to be explicitly enabled during kernel compilation. Before loading the bytecode into the kernel, the Wasm virtual machine can also choose to enable or disable certain eBPF features to ensure the security of the sandbox.
|
||||
- **Tracing & Profiling:** eBPF provides powerful tracing and profiling capabilities by attaching to kernel functions, tracepoints, and even user-space probes. This allows developers to gain deep insights into system and application behavior, enabling them to optimize performance and resolve complex system issues.
|
||||
|
||||
In addition to the kernel mode eBPF runtime, eBPF can also be extended to user space, for example, [bpftime](https://github.com/eunomia-bpf/bpftime) to achieve higher performance user space tracing, performance analysis, plugins, etc.
|
||||
- **Security:** eBPF plays a vital role in real-time security monitoring. It enables deep inspection of system calls, network traffic, and other kernel activities, helping to enforce dynamic security policies and detect anomalous behavior, providing an efficient way to safeguard infrastructure.
|
||||
|
||||
## 2. Some Tips on Learning eBPF Development
|
||||
- **Scheduler Optimization:** eBPF is increasingly used to enhance CPU scheduling, offering the ability to monitor CPU load and optimize how tasks are distributed across cores. This can lead to more efficient use of CPU resources and improved system responsiveness.
|
||||
|
||||
- **HID (Human Interface Device) Driver Enhancements:** Developers use eBPF to optimize HID drivers for devices like keyboards, mice, and touchscreens. By adding custom logic for handling input events, eBPF improves responsiveness in latency-sensitive applications.
|
||||
|
||||
Organizations across industries have adopted eBPF at scale:
|
||||
|
||||
- **Google:** Uses eBPF for security auditing, packet processing, real-time performance monitoring, and optimizing CPU scheduling across its vast infrastructure.
|
||||
- **Netflix:** Leverages eBPF for network traffic analysis, ensuring high availability and performance for streaming services.
|
||||
- **Android:** Applies eBPF to optimize network usage, power consumption, and resource allocation, improving performance and battery life on millions of devices.
|
||||
- **S&P Global:** Utilizes eBPF through **Cilium** for managing networking across multiple clouds and on-premises systems, ensuring scalability and security.
|
||||
- **Shopify:** Implements eBPF with **Falco** for intrusion detection, bolstering security on its e-commerce platform.
|
||||
- **Cloudflare:** Uses eBPF for network observability, security monitoring, and performance optimization, protecting millions of websites globally.
|
||||
|
||||
eBPF's ability to dynamically adjust system behavior and extend into user space makes it an essential technology for modern computing. Whether it's optimizing network traffic, improving security, or enhancing system performance, eBPF enables developers to address real-time requirements efficiently and safely.
|
||||
|
||||
In addition to its kernel-mode runtime, eBPF can also be extended to user space. For example, [bpftime](https://github.com/eunomia-bpf/bpftime), a user-space eBPF runtime, allows for higher-performance tracing, performance analysis, and plugin support in user-space applications. This extension of eBPF into user space helps improve flexibility and performance in various use cases that go beyond kernel-level tasks.
|
||||
|
||||
### Future: The Expanding Potential of eBPF
|
||||
|
||||
Looking forward, eBPF is expected to become an even more integral part of operating systems. The focus is on improving its flexibility, modularity, and ease of use, making it accessible for an even broader range of applications. Innovations in memory management, concurrency mechanisms, and better integration with user-space applications are on the horizon. Projects are already underway to compile significant parts of the Linux kernel to the BPF instruction set, potentially revolutionizing how kernel development and analysis are performed.
|
||||
|
||||
Advancements such as dynamic stacks, better observability tools for user space (e.g., Fast Uprobes and language-specific stack walkers), and safer program termination mechanisms will continue to strengthen eBPF’s reliability and expand its use cases. Additionally, new tools and libraries will simplify eBPF development, lowering the barrier to entry for both kernel and application developers.
|
||||
|
||||
## Getting Started with the Tutorial
|
||||
|
||||
This tutorial provides practical eBPF development practices, covering topics from beginner to advanced levels. We focus on hands-on examples in areas like observability, networking, and security, using frameworks like `libbpf`, `libbpf-rs`, and `eunomia-bpf`, with examples in C, Go, and Rust.
|
||||
|
||||
### Who Is This Tutorial For?
|
||||
|
||||
- **Developers** looking to implement custom kernel solutions.
|
||||
- **System Administrators** aiming to enhance performance and security.
|
||||
- **Tech Enthusiasts** exploring cutting-edge kernel technologies.
|
||||
|
||||
### What Will You Learn?
|
||||
|
||||
- **Core Concepts:** eBPF fundamentals and integration with the Linux kernel.
|
||||
- **Practical Skills:** Writing and deploying eBPF programs.
|
||||
- **Advanced Topics:** Exploring security, tracing, and future innovations in eBPF.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. **Introduction to eBPF**
|
||||
Basic concepts and the tools you need to get started.
|
||||
|
||||
2. **Beginner Examples**
|
||||
Simple programs such as "Hello World" and basic tracing using kprobe and uprobe.
|
||||
|
||||
3. **Observability**
|
||||
Examples focused on monitoring network traffic, file operations, and process behavior using eBPF.
|
||||
|
||||
4. **Networking**
|
||||
Examples focused on modifying and optimizing network traffic, such as XDP, TC, and socket.
|
||||
|
||||
5. **Security**
|
||||
Programs for hiding process and files, sending signals to kill process, and tracking process events for security.
|
||||
|
||||
6. **Advanced Use Cases**
|
||||
Complex examples involving performance profiling, scheduler optimization, and eBPF in user space (e.g., bpftime).
|
||||
|
||||
7. **In-Depth Topics**
|
||||
Exploring eBPF for Android, using eBPF for network acceleration, and securing systems through syscall modifications.
|
||||
|
||||
## How to Use eBPF Programming
|
||||
|
||||
Writing eBPF programs from scratch can be complex. To simplify this, LLVM introduced the ability to compile high-level language code into eBPF bytecode in 2015. The eBPF community has since built libraries like `libbpf` to manage these programs. These libraries help load eBPF bytecode into the kernel and perform essential tasks. The Linux kernel source contains numerous eBPF examples in the `samples/bpf/` directory.
|
||||
|
||||
A typical eBPF program involves two parts: kernel space code (`*_kern.c`) and user space code (`*_user.c`). The kernel space code defines the logic, while the user space code manages loading and interacting with the kernel. However, tools like `libbpf-bootstrap` and the Go eBPF library help simplify this process, allowing for one-time compilation and easier development.
|
||||
|
||||
### Tools for eBPF Development
|
||||
|
||||
- **BCC**: A Python-based toolchain that simplifies writing, compiling, and loading eBPF programs. It offers many pre-built tracing tools but has limitations with dependencies and compatibility.
|
||||
- **eBPF Go Library**: A Go library that decouples the process of obtaining eBPF bytecode from the loading and management of eBPF programs.
|
||||
- **libbpf-bootstrap**: A modern scaffold based on `libbpf` that provides an efficient workflow for writing eBPF programs, offering a simple one-time compilation process for reusable bytecode.
|
||||
- **eunomia-bpf**: A toolchain for writing eBPF programs with only kernel space code. It simplifies the development of eBPF programs by dynamically loading them.
|
||||
|
||||
These tools help reduce the complexity of developing eBPF programs, making the process more accessible to developers aiming to optimize system performance, security, and observability.
|
||||
|
||||
## Some Tips on Learning eBPF Development
|
||||
|
||||
This article will not provide a more detailed introduction to the principles of eBPF, but here is a learning plan and reference materials that may be of value:
|
||||
|
||||
@@ -39,10 +124,7 @@ This article will not provide a more detailed introduction to the principles of
|
||||
Recommended:
|
||||
|
||||
- Read the introduction to ebpf: <https://ebpf.io/> (30min)
|
||||
- Briefly understand the ebpf kernel-related documentation: <https://prototype-kernel.readthedocs.io/en/latest/bpf/> (Know where to queries, 30min)
|
||||
- Read the Chinese ebpf beginner's guide: <https://www.modb.pro/db/391570> (1h)
|
||||
- There are a lot of reference materials: <https://github.com/zoidbergwill/awesome-ebpf> (2-3h)
|
||||
- You can choose to flip through PPTs that interest you: <https://github.com/gojue/ebpf-slide> (1-2h)
|
||||
- Briefly understand the ebpf kernel-related documentation: <https://docs.ebpf.io/> (Know where to queries for tech details, 30min)
|
||||
|
||||
Answer three questions:
|
||||
|
||||
@@ -57,7 +139,7 @@ Understand and try eBPF development frameworks:
|
||||
- bpftrace tutorial：<https://eunomia.dev/tutorials/bpftrace-tutorial/> （Try it，1h）
|
||||
- Examples of developing various tools with BCC: <https://github.com/iovisor/bcc/blob/master/docs/tutorial_bcc_python_developer.md> (Run through, 3-4h)
|
||||
- Some examples of libbpf: <https://github.com/libbpf/libbpf-bootstrap> (Run any interesting one and read the source code, 2h)
|
||||
- Tutorial based on libbpf and eunomia-bpf: <https://github.com/eunomia-bpf/bpf-developer-tutorial> (Read part 1-10, 3-4h)
|
||||
- Tutorials: <https://github.com/eunomia-bpf/bpf-developer-tutorial> (Read part 1-10, 3-4h)
|
||||
|
||||
Other development frameworks: Go or Rust language, please search and try on your own (0-2h)
|
||||
|
||||
@@ -69,73 +151,7 @@ Answer some questions and try some experiments (2-5h):
|
||||
2. How to trace a kernel feature or function with eBPF? There are many ways, provide corresponding code examples;
|
||||
3. What are the solutions for communication between user mode and kernel mode? How to send information from user mode to kernel mode? How to pass information from kernel mode to user mode? Provide code examples;
|
||||
4. Write your own eBPF program to implement a feature;
|
||||
5. In the entire lifecycle of an eBPF program, what does it do in user mode and kernel mode?## 3. How to use eBPF programming
|
||||
|
||||
Writing original eBPF programs is very tedious and difficult. To change this situation, llvm introduced in 2015 the ability to compile code written in high-level languages into eBPF bytecode, and the eBPF community wrapped primitive system calls such as `bpf()` and provided the `libbpf` library. These libraries include functions for loading bytecode into the kernel and other key functions. In the Linux source code package, there are numerous eBPF sample codes provided by Linux based on `libbpf`, located in the `samples/bpf/` directory.
|
||||
|
||||
A typical `libbpf`-based eBPF program consists of two files: `*_kern.c` and `*_user.c`. The mounting points and processing functions in the kernel are written in `*_kern.c`, while the user code for injecting kernel code and performing various tasks in user space is written in `*_user.c`. For more detailed tutorials, refer to [this video](https://www.bilibili.com/video/BV1f54y1h74r?spm_id_from=333.999.0.0). However, due to the difficulty in understanding and the entry barrier, most eBPF program development at the current stage is based on tools such as:
|
||||
|
||||
- BCC
|
||||
- BPFtrace
|
||||
- libbpf-bootstrap
|
||||
- Go eBPF library
|
||||
|
||||
And there are newer tools such as `eunomia-bpf`.
|
||||
|
||||
## Writing eBPF programs
|
||||
|
||||
eBPF programs consist of a kernel space part and a user space part. The kernel space part contains the actual logic of the program, while the user space part is responsible for loading and managing the kernel space part. With the eunomia-bpf development tool, only the kernel space part needs to be written.
|
||||
|
||||
The code in the kernel space part needs to conform to the syntax and instruction set of eBPF. eBPF programs mainly consist of several functions, each with its own specific purpose. The available function types include:
|
||||
|
||||
- kprobe: probe function, executed before or after a specified kernel function.
|
||||
- tracepoint: tracepoint function, executed at a specified kernel tracepoint.
|
||||
- raw_tracepoint: raw tracepoint function, executed at a specified kernel raw tracepoint.
|
||||
- xdp: network data processing function, intercepting and processing network packets.
|
||||
- perf_event: performance event function, used to handle kernel performance events.
|
||||
- kretprobe: return probe function, executed when a specified kernel function returns.
|
||||
- tracepoint_return: tracepoint return function, executed when a specified kernel tracepoint returns.
|
||||
- raw_tracepoint_return: raw tracepoint return function, executed when a specified kernel raw tracepoint returns.
|
||||
|
||||
### BCC
|
||||
|
||||
BCC stands for BPF Compiler Collection. The project is a Python library that includes a complete toolchain for writing, compiling, and loading BPF programs, as well as tools for debugging and diagnosing performance issues.
|
||||
|
||||
Since its release in 2015, BCC has been continuously improved by hundreds of contributors and now includes a large number of ready-to-use tracing tools. The [official project repository](https://github.com/iovisor/bcc/blob/master/docs/tutorial.md) provides a handy tutorial for users to quickly get started with BCC.
|
||||
|
||||
Users can program in high-level languages such as Python and Lua on BCC. Compared to programming directly in C, these high-level languages are much more convenient. Users only need to design BPF programs in C, and the rest, including compilation, parsing, loading, etc., can be done by BCC.
|
||||
|
||||
However, a drawback of using BCC is its compatibility. Each time an eBPF program based on BCC is executed, it needs to be compiled, and compiling requires users to configure related header files and corresponding implementations. In practical applications, as you may have experienced, dependency issues in compilation can be quite tricky. Therefore, in the development of this project, we have abandoned BCC and chosen the libbpf-bootstrap tool, which allows for one-time compilation and multiple runs.
|
||||
|
||||
### eBPF Go library
|
||||
|
||||
The eBPF Go library provides a general-purpose eBPF library that decouples the process of obtaining eBPF bytecode from the loading and management of eBPF programs, and implements similar CO- functionality as libbpf. eBPF programs are usually created by writing high-level languages and then compiled into eBPF bytecode using the clang/LLVM compiler.
|
||||
|
||||
### libbpf
|
||||
|
||||
`libbpf-bootstrap` is a BPF development scaffold based on the `libbpf` library, and its source code can be obtained from its [GitHub](https://github.com/libbpf/libbpf-bootstrap).
|
||||
|
||||
`libbpf-bootstrap` combines years of practice from the BPF community and provides a modern and convenient workflow for developers, achieving the goal of one-time compilation and reuse.
|
||||
|
||||
BPF programs based on `libbpf-bootstrap` have certain naming conventions for the source files. The file for generating kernel space bytecode ends with `.bpf.c`, and the file for loading bytecode in user space ends with `.c`, and the prefixes of these two files must be the same.
|
||||
|
||||
Based on `libbpf-bootstrap`, BPF programs will compile `*.bpf.c` files into corresponding `.o` files,
|
||||
and then generate the `skeleton` file based on this file, i.e. `*.skel.h`. This file will contain some data structures defined in the kernel space,
|
||||
as well as the key functions used to load kernel space code. After the user-space code includes this file, it can call the corresponding loading function to load the bytecode into the kernel. Similarly, `libbpf-bootstrap` also has a comprehensive introduction tutorial that users can refer to [here](https://nakryiko.com/posts/libbpf-bootstrap/) for detailed introductory operations.
|
||||
|
||||
### eunomia-bpf
|
||||
|
||||
[eunomia-bpf](https://github.com/eunomia-bpf/eunomia-bpf) is an open-source eBPF dynamic loading tool and development toolchain designed to simplify the development, building, distribution, and execution of eBPF programs. It is based on the libbpf CO-RE lightweight development framework.
|
||||
|
||||
With eunomia-bpf, you can:
|
||||
|
||||
- When writing eBPF programs or tools, only write kernel space code, automatically retrieve kernel space export information, and dynamically load it as a module.
|
||||
- eunomia-bpf can package pre-compiled eBPF programs into universal JSON or WASM modules for distribution across architectures and kernel versions. They can be dynamically loaded and run without the need for recompilation.
|
||||
|
||||
eunomia-bpf consists of a compilation toolchain and a runtime library. Compared with traditional frameworks such as BCC and native libbpf, it simplifies the development process of eBPF programs.
|
||||
|
||||
> - eunomia-bpf project Github address: <https://github.com/eunomia-bpf/eunomia-bpf>
|
||||
> - gitee mirror: <https://gitee.com/anolis/eunomia>
|
||||
5. In the entire lifecycle of an eBPF program, what does it do in user mode and kernel mode?
|
||||
|
||||
## References
|
||||
|
||||
|
||||
Reference in New Issue
Block a user