mirror of
https://github.com/eunomia-bpf/bpf-developer-tutorial.git
synced 2026-10-30 12:15:40 +08:00
docs: improve tutorials 50 through 54
This commit is contained in:
yuxi4096
@@ -1,54 +1,91 @@
|
||||
# eBPF Tutorial: Inspecting the Executable Image After exec
|
||||
|
||||
Imagine a container running a series of nested wrapper scripts. What you see from `ps` or `/proc` is `/usr/bin/wrapper.sh --config /etc/app.conf`, but the actual business logic might be a Python interpreter or some dynamically linked binary three layers deep. When troubleshooting, you need to know which executable image the kernel finally installed, what architecture it targets, and what byte order it uses - not just the command-line arguments.
|
||||
When a process calls `execve`, the kernel replaces its memory image with a new executable. But which executable, exactly? If the command line says `/usr/bin/wrapper.sh --config /etc/app.conf`, the actual running code might be a Python interpreter or a compiled binary launched three layers deep through wrapper scripts. Security tools, container runtimes, and troubleshooting utilities all need to know what the kernel *actually* installed—not just what the user typed.
|
||||
|
||||
This tutorial shows how to achieve this using BPF task work and file-backed dynptr. The tool attaches an LSM hook to observe the `bprm_committed_creds` moment, schedules a task work callback for the target process after credentials are committed, and that callback runs in a sleepable context to reacquire the installed executable file, resolve its path, read the ELF header, and send the results to user space through a ring buffer.
|
||||
This tutorial builds a tool that captures that information at the kernel level. It hooks the moment credentials are committed after exec, schedules a deferred callback, then reads the installed executable's ELF header and reports the architecture, byte order, and file type. Along the way, it demonstrates two recent kernel features—BPF task work and file dynptr—that together solve a problem traditional eBPF approaches cannot.
|
||||
|
||||
> Complete source: <https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/54-exec-image-inspector>
|
||||
|
||||
## Why BPF task work and file dynptr are needed
|
||||
## The problem: reading file content from eBPF
|
||||
|
||||
Traditional approaches to inspecting executable files all have significant limitations.
|
||||
Why would a security tool need to read the executable's content rather than just its path? Because the path alone does not identify what code will run. A hash of the executable's content can verify it matches a known-good binary. Embedded signatures or certificates can prove provenance. Specific byte patterns at known offsets can identify packing, obfuscation, or tampering. None of this information is available from the path—you must read the file's bytes. Doing so from eBPF, at the exact moment of exec, eliminates the race window that plagues user-space approaches.
|
||||
|
||||
The most straightforward idea is to read the `/proc/<pid>/exe` symlink, but this requires the process to still be alive. Short-lived processes may have already exited before you can read it. Furthermore, `/proc` access happens in user space, creating a race window with the kernel's exec event - the same process might have already executed `execve` multiple times.
|
||||
The most direct approach to inspecting an executable is reading `/proc/<pid>/exe`, but this only works if the process is still alive. Short-lived processes exit before you can read them. Even if you catch them in time, the `/proc` filesystem is accessed from user space, creating a race window: by the time you read the symlink, the process might have called `execve` again.
|
||||
|
||||
Using a tracepoint or kprobe to observe `sched_process_exec` can capture exec events, but these hooks run in a non-sleepable context. You can obtain the file path, but if you need to read file contents to verify the ELF header or check embedded metadata, you run into a problem: the file-backed dynptr's `bpf_dynptr_read` triggers a page fault when target bytes are on a cold page, and page faults require sleep to complete. BPF programs in a non-sleepable context cannot wait for I/O.
|
||||
Tracepoints and kprobes can hook `sched_process_exec` to observe exec events synchronously, but these hooks run in what the kernel calls a **non-sleepable context**. This matters because of how Linux manages file data in memory.
|
||||
|
||||
BPF task work solves this problem. Linux 6.18 introduced the `bpf_task_work_schedule_signal` kfunc, which allows a BPF program to schedule a callback for a target task. The kernel executes this callback in a safe sleepable context. Linux 6.19 introduced file-backed dynptr, providing verifier-tracked file data access. Combining these two features, you can identify the target process in an LSM hook, record a timestamp, and schedule task work. The callback then runs in a sleepable context where it can reacquire the task's installed executable file, read content from any offset through the dynptr (including cold pages), and finally send the complete inspection results to user space.
|
||||
When you read from a file, the kernel first checks whether the requested bytes are already in the **page cache**—a memory region that caches recently-accessed file data. If they are, the read completes immediately. If they are not (a **cold page**), the kernel must issue I/O to the storage device, and the calling context must **sleep** while waiting for that I/O to complete.
|
||||
|
||||
This is the design philosophy behind `exec_image_inspector`: the LSM hook selects the child process and schedules task work, while the sleepable callback performs the actual file reading and parsing.
|
||||
BPF programs attached to tracepoints and kprobes cannot sleep. They run with interrupts potentially disabled and locks held; sleeping would deadlock the system. If a BPF program tries to read file content and encounters a cold page, the read fails with `-EFAULT` instead of waiting for I/O.
|
||||
|
||||
## How BPF task work and file dynptr work
|
||||
This creates a fundamental limitation: you can observe exec events, but you cannot reliably read the executable's content to verify its ELF header or check embedded metadata.
|
||||
|
||||
BPF task work is a deferred execution mechanism. When you call `bpf_task_work_schedule_signal(task, work, map, callback)` in a BPF program, the kernel associates the callback with the specified task. The callback does not execute immediately - it runs at a safe point before that task returns to user space, where the context allows sleeping.
|
||||
## The solution: BPF task work and file dynptr
|
||||
|
||||
`struct bpf_task_work` is an opaque 64-byte structure. The BPF program only needs to allocate storage for it; the kernel manages its internal state. This tool uses a single-element ARRAY map to store `struct exec_work`, which contains the `bpf_task_work` storage, a scheduling timestamp, and direct probe results.
|
||||
Linux 6.18 introduced **BPF task work**, a mechanism that lets a BPF program schedule a callback to run later in a safe, sleepable context. The callback executes before the target task returns to user space, at a point where the kernel permits sleeping.
|
||||
|
||||
File-backed dynptr provides a verifier-tracked file data access interface. `bpf_dynptr_from_file(file, flags, dynptr)` creates a dynptr from a file, and `bpf_dynptr_read(dst, len, dynptr, offset, flags)` reads content at the specified offset. The dynptr holds internal state, so every path that creates a dynptr must call `bpf_dynptr_file_discard` to release it, including failure branches.
|
||||
Linux 6.19 introduced **file dynptr**, which provides verifier-tracked access to file data. A dynptr (dynamic pointer) is a BPF abstraction that tracks a pointer's bounds at verification time; the file variant wraps file I/O operations so the verifier can ensure memory safety.
|
||||
|
||||
In a non-sleepable context, the dynptr can only read content already in the page cache; accessing a cold page returns `-EFAULT`. In a sleepable context, the dynptr can trigger page fault operations and wait for I/O to complete. This is precisely the value of the task work callback.
|
||||
Combining these features, the design becomes:
|
||||
|
||||
## Overall tool flow
|
||||
1. Attach an LSM hook to `bprm_committed_creds`, which fires after exec installs the new credentials
|
||||
2. In the hook (non-sleepable), identify the target process and schedule a task work callback
|
||||
3. The callback runs in a sleepable context, where it can access the installed executable, read content from any offset (including cold pages), and send results to user space
|
||||
|
||||
After user space starts, it forks the target command, but the child process blocks waiting for a release signal on a pipe. This way the parent knows the child's PID (which is also the TGID) before the child executes `execvp`, and can write this value into the BPF program's read-only data section as a filter condition. Blocking the child eliminates the attach race with short-lived commands and keeps the example self-contained.
|
||||
This separation—identify the target in a non-sleepable hook, do the heavy lifting in a sleepable callback—is the key insight.
|
||||
|
||||
The parent opens the BPF skeleton, sets `target_tgid` and the optional `probe_offset`, loads and attaches the LSM program, creates a ring buffer reader, then writes a release byte to the pipe to let the child execute `execvp`.
|
||||
## How BPF task work operates
|
||||
|
||||
After the child executes exec, the `lsm/bprm_committed_creds` hook triggers when credentials are committed. The BPF program compares the current TGID with `target_tgid` and returns immediately if they do not match. After matching the target, the program records a scheduling timestamp, optionally attempts a direct read in the current context if `--probe-offset` is set (to verify the cold page scenario), then calls `bpf_task_work_schedule_signal` to schedule the callback.
|
||||
When you call `bpf_task_work_schedule_signal(task, work, map, callback)`, the kernel associates your callback with the specified task. The callback does not run immediately; it runs later, at a safe point before that task returns to user space.
|
||||
|
||||
The callback `inspect_executable` runs in that task's sleepable context. It calls `bpf_get_task_exe_file` to obtain the task's currently installed executable file (returning a referenced `struct file` that must be released with `bpf_put_file`), resolves the path with `bpf_path_d_path`, and reads the 64-byte ELF header plus the optional marker location using the file dynptr. After a successful read, it parses the ELF magic, class, data, type, and machine fields. Finally, it packages all the information into a ring buffer event and sends it to user space.
|
||||
The `struct bpf_task_work` is an opaque structure that the kernel uses to track the scheduled callback. Your BPF program allocates storage for it but does not interpret its contents. This tool uses a single-element ARRAY map to hold `struct exec_work`, which contains the `bpf_task_work` storage plus fields for timestamps and intermediate results.
|
||||
|
||||
User space alternates between polling the ring buffer and checking child process status with `waitpid(WNOHANG)`. The loop ends when the child is reaped and at least one event is received, or when the timeout is reached. The same child might install multiple images in succession (for example, `/bin/sh -c 'exec /bin/true'`), so after the child is reaped, `drain_events` performs a zero-timeout poll to collect any remaining events in the ring buffer.
|
||||
The callback signature is `int callback(struct bpf_map *map, void *key, void *value)`. The `value` parameter points to the map element containing your `bpf_task_work`, so you can pass data from the scheduling hook to the callback through surrounding fields.
|
||||
|
||||
## How file dynptr operates
|
||||
|
||||
A dynptr wraps a pointer with bounds information that the BPF verifier can track. For file dynptrs:
|
||||
|
||||
- `bpf_dynptr_from_file(file, flags, dynptr)` creates a dynptr from a file
|
||||
- `bpf_dynptr_read(dst, len, dynptr, offset, flags)` reads content at the specified offset
|
||||
- `bpf_dynptr_file_discard(dynptr)` releases the dynptr's internal state
|
||||
|
||||
Every path that creates a dynptr—including error paths—must call `bpf_dynptr_file_discard` to release it. Failing to do so leaks internal resources.
|
||||
|
||||
In a non-sleepable context, `bpf_dynptr_read` only succeeds if the target bytes are already in the page cache. Accessing a cold page returns `-EFAULT`. In a sleepable context, the same call can trigger page fault handling and wait for I/O, so it succeeds even for cold pages. This difference is why task work matters: the callback runs in a sleepable context where file reads work reliably.
|
||||
|
||||
## Tool architecture
|
||||
|
||||
The user-space program forks a child process, but holds the child blocked on a pipe before it calls `execvp`. This gives the parent time to:
|
||||
|
||||
1. Note the child's PID (which becomes its TGID—thread group ID)
|
||||
2. Open the BPF skeleton and write the target TGID into read-only data
|
||||
3. Load and attach the BPF program
|
||||
4. Create a ring buffer reader
|
||||
|
||||
Only then does the parent release the child by writing to the pipe. This handshake eliminates a race condition: without it, a fast-exiting command might finish before the BPF program attaches.
|
||||
|
||||
When the child calls `execvp`, the `lsm/bprm_committed_creds` hook fires. The BPF program compares the current TGID with the configured target; if they match, it records a timestamp and schedules a task work callback.
|
||||
|
||||
The callback—`inspect_executable`—runs in the child's sleepable context. It:
|
||||
|
||||
1. Calls `bpf_get_task_exe_file` to get the installed executable (returning a referenced `struct file` that must be released with `bpf_put_file`)
|
||||
2. Resolves the path with `bpf_path_d_path`
|
||||
3. Creates a file dynptr and reads the 64-byte ELF header
|
||||
4. Parses ELF fields: magic number, class (32/64-bit), data (endianness), type (executable vs shared object), and machine (architecture)
|
||||
5. Sends an event through the ring buffer
|
||||
|
||||
User space polls the ring buffer while also checking whether the child has exited. A single child might exec multiple times (for example, `/bin/sh -c 'exec /bin/true'` first execs the shell, then execs `/bin/true`), so after the child exits, the tool drains any remaining events from the ring buffer.
|
||||
|
||||
|
||||
|
||||
## Code implementation
|
||||
## Code walkthrough
|
||||
|
||||
This tool consists of four files: a shared header defining event and statistics structures, a compatibility header declaring new kernel kfuncs, a BPF program implementing the hook and callback, and a user-space loader managing the lifecycle.
|
||||
The implementation spans four files: a shared header, a compatibility header for new kernel interfaces, the BPF program, and the user-space loader.
|
||||
|
||||
### Shared header
|
||||
|
||||
`exec_image_inspector.h` defines the ring buffer event structure and statistics structure shared between BPF and user space.
|
||||
`exec_image_inspector.h` defines structures shared between BPF and user space:
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -96,11 +133,11 @@ struct inspector_stats {
|
||||
#endif /* __EXEC_IMAGE_INSPECTOR_H */
|
||||
```
|
||||
|
||||
`exec_event` contains all the information needed to inspect one exec: PID, TGID, process name, path, ELF parsing results, probe results, callback latency, and various error codes. `inspector_stats` accumulates match, schedule, callback, error, and drop counts; user space reads the BSS section at the end and reports the statistics.
|
||||
`exec_event` carries everything needed to report one exec: process identifiers, the resolved path, ELF metadata, and error codes. `inspector_stats` accumulates counters that user space reads from the BSS section at exit to report success and failure rates.
|
||||
|
||||
### Compatibility declarations
|
||||
### Compatibility header
|
||||
|
||||
Linux 6.18 introduced BPF task work, and 6.19 introduced file-backed dynptr. The repository's currently generated UAPI and BTF headers predate these features, so this tool places the missing declarations in a local header `bpf_experimental.h`.
|
||||
The repository's vendored vmlinux headers predate Linux 6.18/6.19, so `bpf_experimental.h` declares the new interfaces locally:
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -134,11 +171,11 @@ extern int bpf_dynptr_file_discard(struct bpf_dynptr *dynptr) __ksym;
|
||||
#endif /* __EXEC_IMAGE_INSPECTOR_BPF_EXPERIMENTAL_H */
|
||||
```
|
||||
|
||||
This file declares kfuncs including `bpf_task_work_schedule_signal`, `bpf_get_task_exe_file`, `bpf_put_file`, `bpf_path_d_path`, `bpf_dynptr_from_file`, and `bpf_dynptr_file_discard`. This file can be removed once the repository's integrated vmlinux headers include these declarations.
|
||||
These declarations use `__ksym` to mark them as kernel symbols resolved at load time. Once the repository's vmlinux headers are regenerated from a 6.19+ kernel, this file can be removed.
|
||||
|
||||
### BPF program
|
||||
|
||||
`exec_image_inspector.bpf.c` implements the LSM hook and task work callback.
|
||||
`exec_image_inspector.bpf.c` implements the LSM hook and the task work callback:
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: GPL-2.0
|
||||
@@ -341,23 +378,23 @@ void BPF_PROG(schedule_exec_inspection, struct linux_binprm *bprm)
|
||||
}
|
||||
```
|
||||
|
||||
The program entry point is the `schedule_exec_inspection` function declared with `SEC("lsm/bprm_committed_creds")`. The `bprm_committed_creds` hook runs after the new executable's credentials have been committed, at which point `bprm->file` points to the image involved in this exec. This hook itself is non-sleepable, but we can identify the target process here and schedule task work.
|
||||
The entry point is `schedule_exec_inspection`, declared with `SEC("lsm/bprm_committed_creds")`. This LSM hook fires after the new executable's credentials have been installed, so `bprm->file` points to the file being executed. The hook itself is non-sleepable, but it can identify the target and schedule the deferred work.
|
||||
|
||||
The two `const volatile` variables `target_tgid` and `probe_offset` are in the `.rodata` section. User space writes values after `open()` but before `load()`, and the verifier treats them as compile-time constants. The program first compares the current TGID with `target_tgid`, returning immediately if they do not match to avoid observing unrelated process execs.
|
||||
The two `const volatile` variables—`target_tgid` and `probe_offset`—live in the `.rodata` section. User space writes values between `open()` and `load()`, and the verifier treats them as compile-time constants for optimization.
|
||||
|
||||
After matching the target, the program looks up key 0 from the single-element `pending` ARRAY map to obtain `struct exec_work`. This structure stores the scheduling timestamp, direct probe result, and `struct bpf_task_work` storage. This tool observes only one child process per invocation, so one slot is sufficient. When `--probe-offset` is set, `probe_file_without_sleep` attempts to read 8 bytes from the specified location in `bprm->file` in the current non-sleepable context. The test program places the marker on a cold page to verify that direct reads return `-EFAULT` in this scenario.
|
||||
When the target matches, the program looks up `struct exec_work` from the single-element `pending` ARRAY map. This structure holds the `bpf_task_work` storage plus a timestamp and optional direct-probe result. A single slot suffices because this tool observes one child process per invocation.
|
||||
|
||||
After the hook saves the timestamp and direct read result, it calls `bpf_task_work_schedule_signal`, passing the current task, work storage, map pointer, and the `inspect_executable` callback function pointer. The kernel holds the references needed for callback execution, and the callback will run in a sleepable context before that task returns to user space.
|
||||
The optional `--probe-offset` flag makes the tool attempt a direct read in the non-sleepable hook via `probe_file_without_sleep`. The test suite uses this to verify that reading a cold page fails with `-EFAULT` in the non-sleepable context but succeeds in the sleepable callback.
|
||||
|
||||
`inspect_executable` is the task work callback function, with signature `int callback(struct bpf_map *map, void *key, void *value)`. It first calculates callback latency (current time minus scheduling timestamp) for diagnostics, then fills PID, TGID, process name, and direct probe result into the event structure. It then calls `bpf_get_task_exe_file` to obtain the task's installed executable file. This kfunc returns a referenced `struct file`; every successful acquisition must be released with `bpf_put_file`.
|
||||
After saving the timestamp and direct-probe result, the hook calls `bpf_task_work_schedule_signal`. The kernel holds the references needed to execute the callback later.
|
||||
|
||||
The callback first resolves the file path with `bpf_path_d_path`, then creates a dynptr with `bpf_dynptr_from_file`. Since we are now in a sleepable context, `bpf_dynptr_read` can trigger page fault operations and wait for I/O to complete. The callback only reads a 64-byte ELF header and an optional 8-byte marker. After a successful header read, the program checks the 4-byte ELF magic `\x7fELF`, then parses `EI_CLASS` (32-bit or 64-bit), `EI_DATA` (little-endian or big-endian), `e_type` (executable or shared object), and `e_machine` (architecture). The `read_elf_u16` helper function correctly parses 16-bit fields according to byte order.
|
||||
The callback `inspect_executable` calculates latency for diagnostics, then acquires the executable file with `bpf_get_task_exe_file`. This returns a referenced `struct file` that must be released with `bpf_put_file`. The callback resolves the path, creates a file dynptr, reads the 64-byte ELF header, and parses it. The `read_elf_u16` helper handles endianness: ELF files declare their byte order in the header, and multi-byte fields must be read accordingly.
|
||||
|
||||
Every path that creates a dynptr must call `bpf_dynptr_file_discard` to release its internal state, including branches where the helper fails. Finally, `bpf_ringbuf_output` sends the event to user space; the `dropped` counter is incremented if sending fails.
|
||||
Every path that creates a dynptr—success or failure—must call `bpf_dynptr_file_discard`. Finally, `bpf_ringbuf_output` sends the event to user space.
|
||||
|
||||
### User-space loader
|
||||
|
||||
`exec_image_inspector.c` handles command-line parsing, coordinates the blocked child, loads BPF, formats events, and cleans up resources.
|
||||
`exec_image_inspector.c` coordinates the child process, loads BPF, receives events, and reports results:
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
|
||||
@@ -898,17 +935,15 @@ cleanup:
|
||||
}
|
||||
```
|
||||
|
||||
The loader's flow centers on the blocked child handshake. `start_blocked_child` forks before BPF setup, but the child blocks on a pipe read. The parent already knows the child PID at this point and can write it to `rodata->target_tgid` after opening the skeleton. After attachment, `release_child` writes one byte to the pipe to let the child continue with `execvp`. This handshake eliminates the attach race with short-lived commands.
|
||||
The core technique is the blocked-child handshake. `start_blocked_child` forks before BPF setup, but the child blocks on a pipe read. The parent notes the child's PID, opens the skeleton, writes `target_tgid`, loads and attaches, creates the ring buffer, and only then calls `release_child` to write to the pipe and let the child proceed to `execvp`.
|
||||
|
||||
The `setup_inspector` function opens the skeleton, sets read-only data, loads and attaches the BPF program, and creates the ring buffer reader. Error messages on load failure indicate possible causes to help users troubleshoot kernel version or configuration issues.
|
||||
`wait_for_command` is the main loop. It prints a `READY` line, releases the child, then alternates between polling the ring buffer and checking whether the child has exited. The loop ends when the child is reaped and at least one event has arrived, or when the timeout expires. After timeout, `reap_timed_out_child` sends SIGKILL and waits.
|
||||
|
||||
`wait_for_command` is the main loop. It first prints a `READY` line indicating the tool is ready, then releases the child, sets a timeout deadline, and enters the poll loop. The loop alternates between calling `ring_buffer__poll` to receive events and `waitpid(WNOHANG)` to check child status. The loop exits when the child is reaped and at least one event is received, or when the timeout is reached. After timeout, `reap_timed_out_child` sends SIGKILL to the child and waits for it to exit.
|
||||
A single exec might trigger multiple events if the command itself execs (for example, `/bin/sh -c 'exec /bin/true'`). After the child exits, `drain_events` does a zero-timeout poll to collect any remaining events.
|
||||
|
||||
The same child may install multiple images in succession. For example, running `/bin/sh -c 'exec /bin/true'` causes the shell to exec itself first, then exec `/bin/true`. After the child is reaped, `drain_events` uses zero-timeout poll to collect any remaining events in the ring buffer, ensuring the last image event also reaches user space.
|
||||
`handle_event` formats the output, translating numeric ELF values to readable names while preserving the raw values for scripting.
|
||||
|
||||
`handle_event` parses ring buffer events and prints formatted output. It converts ELF numeric values to readable names like `ELF64`, `LSB`, `ET_DYN`, `EM_X86_64`, while preserving original values. When `--probe-offset` is set, it also prints a `PROBE` line showing probe results.
|
||||
|
||||
## Compilation and execution
|
||||
## Building and running
|
||||
|
||||
Build from source:
|
||||
|
||||
@@ -918,22 +953,22 @@ make -C src/54-exec-image-inspector clean
|
||||
make -C src/54-exec-image-inspector -j2
|
||||
```
|
||||
|
||||
The integration test requires Linux 6.19 or newer. Before running, confirm that `bpf` appears in the active LSM list:
|
||||
The test suite requires Linux 6.19 or newer with the BPF LSM active. Check that `bpf` appears in the LSM list:
|
||||
|
||||
```bash
|
||||
cat /sys/kernel/security/lsm
|
||||
```
|
||||
|
||||
To add `bpf`, change `lsm=<existing-list>` to `lsm=<existing-list>,bpf` in the kernel boot parameters.
|
||||
If `bpf` is missing, add it to the kernel command line: change `lsm=<existing-list>` to `lsm=<existing-list>,bpf` in your bootloader configuration.
|
||||
|
||||
Run tests:
|
||||
Run the tests:
|
||||
|
||||
```bash
|
||||
cd src/54-exec-image-inspector
|
||||
sudo make test
|
||||
```
|
||||
|
||||
Repository CI only compiles this lesson. Runtime behavior was functionally tested on x86_64 with kernel `7.0.0-rc2+`. The following session output shows the test harness and tool output:
|
||||
The repository CI only compiles this lesson. Runtime behavior was functionally tested on x86_64 with kernel `7.0.0-rc2+`. Sample output:
|
||||
|
||||
```text
|
||||
TEST-MISSING matched=0 events=0 command_exit=127
|
||||
@@ -947,11 +982,15 @@ SUMMARY matched=1 scheduled=1 schedule_errors=0 callbacks=1 header_errors=0 path
|
||||
PASS: missing-command, timeout cleanup, re-exec drain, ELF decode, and deferred file read succeeded
|
||||
```
|
||||
|
||||
The first three test records cover important boundaries: a missing command exits with status 127, at which point the committed-exec hook has not yet triggered, so `matched=0` and `events=0`; the timeout case produces an exec event then exceeds the test deadline, is terminated by SIGKILL, and reaped with status 137; the successive exec case drains two image events and confirms `/usr/bin/true` appears last.
|
||||
The first three test lines cover boundary conditions:
|
||||
|
||||
The `EXEC` line shows the installed image and ELF header parsing results. The test program flushes and evicts the marker page before exec, so the direct read in the hook returns `-EFAULT` (`-14`), while the task-work callback successfully returns `454950524f424521` (the hexadecimal bytes for `EIPROBE!`). This verifies that non-sleepable context cannot read cold pages, while sleepable context can.
|
||||
- **TEST-MISSING**: A nonexistent command exits with status 127. The LSM hook never fires because exec fails before credentials are committed, so `matched=0` and `events=0`.
|
||||
- **TEST-TIMEOUT**: The command runs too long, gets SIGKILL, and is reaped with status 137 (128 + 9). One exec event was observed.
|
||||
- **TEST-REEXEC**: A shell command that uses `exec` internally produces two events, and the final path is `/usr/bin/true`.
|
||||
|
||||
To inspect other commands, omit the test-program-specific probe offset:
|
||||
The `EXEC` line shows the installed image and parsed ELF fields. The `PROBE` line demonstrates the sleepable-context difference: the test writes a marker (`EIPROBE!`) to a page, flushes and evicts it from the page cache, then execs. The direct read in the non-sleepable hook returns `-EFAULT` (`-14`) because the page is cold. The deferred read in the sleepable callback succeeds, returning the marker bytes as hex (`454950524f424521`).
|
||||
|
||||
To inspect a simple command:
|
||||
|
||||
```bash
|
||||
sudo ./exec_image_inspector --timeout-ms 3000 -- /bin/true
|
||||
@@ -963,29 +1002,32 @@ Command-line format:
|
||||
exec_image_inspector [--probe-offset BYTES] [--timeout-ms MS] [--verbose] -- COMMAND [ARG...]
|
||||
```
|
||||
|
||||
`--timeout-ms` accepts 100 to 60000 milliseconds, default is 5000 milliseconds; the loader terminates and reaps the command after the deadline. `--probe-offset` compares 8-byte direct and deferred reads at the specified location, used to verify cold page scenarios. `--verbose` outputs libbpf diagnostics for troubleshooting load or attach issues. `--` explicitly ends inspector option parsing; all subsequent arguments belong to the observed command.
|
||||
- `--timeout-ms`: 100 to 60000 milliseconds (default 5000). The tool kills and reaps the command after this deadline.
|
||||
- `--probe-offset`: Read 8 bytes at this offset both in the hook (direct) and in the callback (deferred), to verify the cold-page difference.
|
||||
- `--verbose`: Print libbpf diagnostic messages.
|
||||
- `--`: Separates inspector options from the command to run.
|
||||
|
||||
### Environment requirements
|
||||
### Requirements
|
||||
|
||||
| Requirement | Details |
|
||||
|---|---|
|
||||
| Kernel version | Linux 6.19+ (BPF task work introduced in 6.18, file-backed dynptr introduced in 6.19) |
|
||||
| Kernel version | Linux 6.19+ (BPF task work introduced in 6.18, file dynptr in 6.19) |
|
||||
| Kernel configuration | `CONFIG_BPF=y`, `CONFIG_BPF_SYSCALL=y`, `CONFIG_BPF_JIT=y`, `CONFIG_BPF_LSM=y`, `CONFIG_SECURITY=y`, `CONFIG_DEBUG_INFO_BTF=y` |
|
||||
| Active LSM | `/sys/kernel/security/lsm` contains `bpf` |
|
||||
| Active LSM | `/sys/kernel/security/lsm` must contain `bpf` |
|
||||
| Architecture | Tested on x86_64 |
|
||||
| Privilege | root |
|
||||
| Privileges | root |
|
||||
|
||||
## Scope
|
||||
## Limitations and extensions
|
||||
|
||||
This tool observes one direct child that it creates, and a single map slot is sufficient for this CLI scenario. To extend it to concurrent services, you could allocate per-task state, set admission limits, and reclaim pending callbacks. Timeout cleanup sends SIGKILL to the child; callers can add process-group management and external signal handling to cover more complex scenarios.
|
||||
This tool observes one direct child per invocation, using a single map slot. For concurrent services, you would allocate per-task state, implement admission limits, and handle callback reclamation. Timeout cleanup sends SIGKILL to the child only; callers needing process-group management or external signal handling would add those features.
|
||||
|
||||
## Summary
|
||||
|
||||
This tutorial demonstrates how to use BPF task work and file-backed dynptr to inspect the executable image actually installed after exec. Compared to reading `/proc/<pid>/exe` or obtaining paths in tracepoints, this approach can read file contents in a sleepable context, even when target bytes are on cold pages.
|
||||
This tutorial demonstrates how to combine BPF task work and file dynptr to inspect the executable image actually installed by exec. The key insight is separating two moments: identifying the target (in a non-sleepable LSM hook) and reading file content (in a sleepable task work callback). This lets eBPF programs read file data reliably, even when the target bytes are not in the page cache.
|
||||
|
||||
The core idea of this tool is to separate the moment of identifying the target process (non-sleepable LSM hook) from the moment of reading file contents (sleepable task work callback). The blocked-child handshake eliminates attach races, bounded command execution and final drain ensure all events reach user space, and explicit resource release guarantees complete cleanup. These kernel features combine into a reproducible single-command tool while leaving room for extension to concurrent scenarios.
|
||||
The blocked-child handshake eliminates attach races, bounded execution ensures cleanup, and the final drain captures all events. Together, these techniques produce a reproducible single-command tool while leaving room for extension.
|
||||
|
||||
> If you want to learn more about eBPF, check out our tutorial repository at <https://github.com/eunomia-bpf/bpf-developer-tutorial> or visit our website at <https://eunomia.dev/tutorials/>.
|
||||
> To learn more about eBPF, visit our tutorial repository at <https://github.com/eunomia-bpf/bpf-developer-tutorial> or <https://eunomia.dev/tutorials/>.
|
||||
|
||||
## References
|
||||
|
||||
|
||||
@@ -1,54 +1,91 @@
|
||||
# eBPF 教程：检查 exec 后真正安装的可执行镜像
|
||||
# eBPF 教程：检查 exec 后实际安装的可执行镜像
|
||||
|
||||
假设某个容器中运行着一系列嵌套的包装脚本，你从 `ps` 或 `/proc` 看到的命令行是 `/usr/bin/wrapper.sh --config /etc/app.conf`，但实际执行业务逻辑的可能是三层深处的 Python 解释器或某个动态链接的二进制。排查故障时你需要知道内核最终安装的是哪个可执行镜像、它是什么架构、用什么字节序，而不只是命令行参数。
|
||||
当进程调用 `execve` 时，内核会用新的可执行镜像替换它的内存映像。但具体是哪个可执行文件？如果命令行显示 `/usr/bin/wrapper.sh --config /etc/app.conf`，实际运行的代码可能是三层包装脚本深处的 Python 解释器或编译好的二进制。安全工具、容器运行时、故障排查工具都需要知道内核 *实际* 安装的是什么，而不仅仅是用户输入的命令行。
|
||||
|
||||
本教程展示如何使用 BPF task work 和 file-backed dynptr 实现这一目标。工具挂载 LSM hook 观察 `bprm_committed_creds` 时刻，在凭据提交后为目标进程安排 task work 回调，回调在可睡眠上下文中重新获取已安装的可执行文件、解析路径、读取 ELF 头部，并把结果通过 ring buffer 发送给用户态。
|
||||
本教程构建一个在内核层面捕获这些信息的工具。它在 exec 提交凭据后挂载钩子，安排一个延迟回调，然后读取已安装可执行文件的 ELF 头部，报告其架构、字节序和文件类型。过程中会演示两个较新的内核特性——BPF task work 和 file dynptr——它们组合起来可以解决传统 eBPF 方案无法处理的问题。
|
||||
|
||||
> 完整源码：<https://github.com/eunomia-bpf/bpf-developer-tutorial/tree/main/src/54-exec-image-inspector>
|
||||
|
||||
## 为什么需要 BPF task work 和 file dynptr
|
||||
## 问题：从 eBPF 读取文件内容的困难
|
||||
|
||||
传统的可执行文件检查方案都有明显的局限性。
|
||||
为什么安全工具需要读取可执行文件的内容，而不仅仅是路径？因为路径本身并不能确定将要运行什么代码。对可执行文件内容的哈希可以验证它是否与已知的可信二进制匹配。嵌入的签名或证书可以证明来源。特定偏移处的特定字节模式可以识别加壳、混淆或篡改。这些信息都无法从路径获取——必须读取文件的字节。在 eBPF 中、在 exec 的精确时刻读取，消除了困扰用户态方案的竞态窗口。
|
||||
|
||||
最直接的想法是读取 `/proc/<pid>/exe` 符号链接，但这需要在进程存活期间完成，短命进程可能在你读取之前就已经退出。而且 `/proc` 访问发生在用户态，与内核的 exec 事件之间存在竞态窗口，同一进程可能已经执行了多次 `execve`。
|
||||
最直接的可执行文件检查方法是读取 `/proc/<pid>/exe`，但这要求进程仍然存活。短命进程在你读取之前就已退出。即使抓住了时机，`/proc` 文件系统是从用户态访问的，与内核的 exec 事件之间存在竞态窗口：等你读到符号链接时，进程可能已经再次调用了 `execve`。
|
||||
|
||||
使用 tracepoint 或 kprobe 观察 `sched_process_exec` 可以捕获 exec 事件，但这些 hook 运行在不可睡眠上下文中。你可以获取文件路径，但如果需要读取文件内容来验证 ELF 头部或检查嵌入的元数据，问题就来了：file-backed dynptr 的 `bpf_dynptr_read` 在目标字节位于冷页时会触发缺页操作，而缺页需要睡眠才能完成。不可睡眠上下文中的 BPF 程序无法等待 I/O。
|
||||
Tracepoint 和 kprobe 可以钩住 `sched_process_exec` 同步观察 exec 事件，但这些钩子运行在内核所谓的**不可睡眠上下文**中。为什么这很重要？这涉及到 Linux 如何管理内存中的文件数据。
|
||||
|
||||
BPF task work 才能真正解决这个问题。Linux 6.18 引入了 `bpf_task_work_schedule_signal` kfunc，允许 BPF 程序为目标 task 安排回调，内核会在安全的可睡眠上下文中执行这个回调。Linux 6.19 引入了 file-backed dynptr，提供验证器跟踪的文件数据访问方式。把这两个功能组合起来，你可以在 LSM hook 中识别目标进程、记录时间戳，然后安排 task work；回调在可睡眠上下文中执行，可以重新获取 task 已安装的可执行文件、通过 dynptr 读取任意位置的内容（包括冷页），最后把完整的检查结果发送给用户态。
|
||||
当你读取文件时，内核首先检查请求的字节是否已经在**页缓存**（page cache）中——这是一块缓存最近访问过的文件数据的内存区域。如果在，读取立即完成。如果不在（即**冷页**），内核必须向存储设备发起 I/O 请求，调用上下文必须**睡眠**等待 I/O 完成。
|
||||
|
||||
这就是 `exec_image_inspector` 的设计思路：LSM hook 选定子进程并调度 task work，可睡眠回调完成实际的文件读取和解析工作。
|
||||
挂载在 tracepoint 和 kprobe 上的 BPF 程序不能睡眠。它们运行时可能中断被禁用、持有锁；如果睡眠会导致系统死锁。当 BPF 程序尝试读取文件内容遇到冷页时，读取会以 `-EFAULT` 失败，而不是等待 I/O。
|
||||
|
||||
## BPF task work 与 file dynptr 的工作原理
|
||||
这就形成了根本性的限制：你可以观察 exec 事件，但无法可靠地读取可执行文件的内容来验证其 ELF 头部或检查嵌入的元数据。
|
||||
|
||||
BPF task work 是一种延迟执行机制。当你在 BPF 程序中调用 `bpf_task_work_schedule_signal(task, work, map, callback)` 时，内核会把回调关联到指定的 task。回调不会立即执行，而是在该 task 返回用户态前的某个安全点执行，此时上下文允许睡眠。
|
||||
## 解决方案：BPF task work 与 file dynptr
|
||||
|
||||
`struct bpf_task_work` 是一个不透明的 64 字节结构，BPF 程序只需要为它分配存储空间，内核负责管理其内部状态。本工具使用单元素 ARRAY map 存储 `struct exec_work`，其中包含 `bpf_task_work` 存储、调度时间戳和直接探测结果。
|
||||
Linux 6.18 引入了 **BPF task work**，这是一种让 BPF 程序安排回调稍后在安全的可睡眠上下文中执行的机制。回调会在目标任务返回用户态之前、内核允许睡眠的时刻执行。
|
||||
|
||||
File-backed dynptr 则提供了验证器跟踪的文件数据访问接口。`bpf_dynptr_from_file(file, flags, dynptr)` 从文件创建 dynptr，`bpf_dynptr_read(dst, len, dynptr, offset, flags)` 读取指定偏移处的内容。dynptr 持有内部状态，因此所有创建 dynptr 的路径都必须调用 `bpf_dynptr_file_discard` 来释放，包括失败分支。
|
||||
Linux 6.19 引入了 **file dynptr**，提供验证器跟踪的文件数据访问。dynptr（动态指针）是一种 BPF 抽象，在验证时追踪指针的边界；file 变体封装了文件 I/O 操作，让验证器能确保内存安全。
|
||||
|
||||
在不可睡眠上下文中，dynptr 只能读取已经在页缓存中的内容，访问冷页会返回 `-EFAULT`。在可睡眠上下文中，dynptr 可以触发缺页操作并等待 I/O 完成。这正是 task work 回调的价值所在。
|
||||
结合这两个特性，设计变成：
|
||||
|
||||
## 工具的整体流程
|
||||
1. 在 `bprm_committed_creds` 挂载 LSM 钩子，它在 exec 安装新凭据后触发
|
||||
2. 在钩子中（不可睡眠）识别目标进程并安排 task work 回调
|
||||
3. 回调在可睡眠上下文中执行，可以访问已安装的可执行文件、读取任意偏移的内容（包括冷页）、然后把结果发送给用户态
|
||||
|
||||
用户态启动后先 fork 目标命令，但子进程会在 pipe 上阻塞等待释放信号。这样父进程就能在子进程执行 `execvp` 之前知道它的 PID（也是 TGID），可以把这个值写入 BPF 程序的只读数据段作为过滤条件。阻塞子进程消除了短命命令带来的 attach 竞态，让整个示例保持自包含。
|
||||
这种分离——在不可睡眠的钩子中识别目标，在可睡眠的回调中完成重活——是关键思路。
|
||||
|
||||
父进程打开 BPF skeleton、设置 `target_tgid` 和可选的 `probe_offset`、加载并挂载 LSM 程序、创建 ring buffer reader，随后向 pipe 写入释放字节让子进程执行 `execvp`。
|
||||
## BPF task work 的工作机制
|
||||
|
||||
子进程执行 exec 后，凭据提交时 `lsm/bprm_committed_creds` hook 触发。BPF 程序比较当前 TGID 与 `target_tgid`，不匹配则直接返回。命中目标后程序记录调度时间戳，如果设置了 `--probe-offset` 则先尝试在当前上下文直接读取（用于验证冷页场景），然后调用 `bpf_task_work_schedule_signal` 安排回调。
|
||||
调用 `bpf_task_work_schedule_signal(task, work, map, callback)` 时，内核把你的回调关联到指定的任务。回调不会立即执行；它会稍后在该任务返回用户态之前的某个安全点执行。
|
||||
|
||||
回调 `inspect_executable` 在该 task 的可睡眠上下文中执行。它调用 `bpf_get_task_exe_file` 获取 task 当前安装的可执行文件（返回带引用的 `struct file`，必须用 `bpf_put_file` 释放），用 `bpf_path_d_path` 解析路径，用 file dynptr 读取 64 字节 ELF 头部和可选的标记位置。读取成功后解析 ELF 魔数、class、data、type 和 machine 字段。最后把所有信息打包成 ring buffer 事件发送给用户态。
|
||||
`struct bpf_task_work` 是一个不透明结构，内核用它来追踪已安排的回调。BPF 程序只需为它分配存储空间，不解释其内容。本工具使用单元素 ARRAY map 保存 `struct exec_work`，其中包含 `bpf_task_work` 存储以及时间戳和中间结果字段。
|
||||
|
||||
用户态边轮询 ring buffer 边用 `waitpid(WNOHANG)` 检查子进程状态。子进程回收且至少收到一条事件时结束循环，或者等到超时。同一子进程可能连续安装多个镜像（例如 `/bin/sh -c 'exec /bin/true'`），子进程回收后还要执行 `drain_events` 取尽 ring buffer 中的剩余事件。
|
||||
回调签名是 `int callback(struct bpf_map *map, void *key, void *value)`。`value` 参数指向包含你的 `bpf_task_work` 的 map 元素，所以你可以通过周围的字段从调度钩子向回调传递数据。
|
||||
|
||||
## file dynptr 的工作机制
|
||||
|
||||
dynptr 用边界信息包装指针，BPF 验证器可以追踪这些边界。对于 file dynptr：
|
||||
|
||||
- `bpf_dynptr_from_file(file, flags, dynptr)` 从文件创建 dynptr
|
||||
- `bpf_dynptr_read(dst, len, dynptr, offset, flags)` 读取指定偏移处的内容
|
||||
- `bpf_dynptr_file_discard(dynptr)` 释放 dynptr 的内部状态
|
||||
|
||||
每条创建 dynptr 的路径——包括错误路径——都必须调用 `bpf_dynptr_file_discard` 来释放它。忘记释放会泄漏内部资源。
|
||||
|
||||
在不可睡眠上下文中，`bpf_dynptr_read` 只有在目标字节已经在页缓存中时才会成功。访问冷页返回 `-EFAULT`。在可睡眠上下文中，同样的调用可以触发缺页处理并等待 I/O，即使对冷页也能成功。这个差异就是 task work 的价值所在：回调运行在可睡眠上下文中，文件读取能可靠完成。
|
||||
|
||||
## 工具架构
|
||||
|
||||
用户态程序 fork 出一个子进程，但在子进程调用 `execvp` 之前用管道阻塞它。这给父进程时间去：
|
||||
|
||||
1. 记录子进程的 PID（它会成为 TGID——线程组 ID）
|
||||
2. 打开 BPF skeleton 并把目标 TGID 写入只读数据段
|
||||
3. 加载并挂载 BPF 程序
|
||||
4. 创建 ring buffer reader
|
||||
|
||||
只有这些都完成后，父进程才通过写管道来释放子进程。这个握手消除了竞态条件：没有它的话，快速退出的命令可能在 BPF 程序挂载之前就结束了。
|
||||
|
||||
当子进程调用 `execvp` 时，`lsm/bprm_committed_creds` 钩子触发。BPF 程序比较当前 TGID 与配置的目标；如果匹配，它记录时间戳并安排 task work 回调。
|
||||
|
||||
回调——`inspect_executable`——在子进程的可睡眠上下文中执行。它：
|
||||
|
||||
1. 调用 `bpf_get_task_exe_file` 获取已安装的可执行文件（返回一个带引用的 `struct file`，必须用 `bpf_put_file` 释放）
|
||||
2. 用 `bpf_path_d_path` 解析路径
|
||||
3. 创建 file dynptr 并读取 64 字节 ELF 头部
|
||||
4. 解析 ELF 字段：魔数、class（32/64 位）、data（字节序）、type（可执行文件 vs 共享对象）、machine（架构）
|
||||
5. 通过 ring buffer 发送事件
|
||||
|
||||
用户态一边轮询 ring buffer 一边检查子进程是否已退出。单个子进程可能多次 exec（例如 `/bin/sh -c 'exec /bin/true'` 先 exec shell，然后 exec `/bin/true`），所以在子进程退出后，工具会取尽 ring buffer 中的剩余事件。
|
||||
|
||||
|
||||
|
||||
## 代码实现
|
||||
## 代码详解
|
||||
|
||||
本工具由四个文件组成：共享头文件定义事件和统计结构、兼容性头文件声明新内核 kfunc、BPF 程序实现 hook 和回调、用户空间加载器管理生命周期。
|
||||
实现分布在四个文件中：共享头文件、新内核接口的兼容性头文件、BPF 程序、用户态加载器。
|
||||
|
||||
### 共享头文件
|
||||
|
||||
`exec_image_inspector.h` 定义了 BPF 和用户空间共享的 ring buffer 事件结构和统计结构。
|
||||
`exec_image_inspector.h` 定义 BPF 和用户态共享的结构：
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -96,11 +133,11 @@ struct inspector_stats {
|
||||
#endif /* __EXEC_IMAGE_INSPECTOR_H */
|
||||
```
|
||||
|
||||
`exec_event` 包含了检查一次 exec 所需的全部信息：PID、TGID、进程名、路径、ELF 解析结果、探测结果、回调延迟和各类错误代码。`inspector_stats` 累计匹配、调度、回调、错误和丢弃计数，用户态在结束时读取 BSS 段并报告。
|
||||
`exec_event` 携带报告一次 exec 所需的全部信息：进程标识符、解析后的路径、ELF 元数据、错误码。`inspector_stats` 累计各种计数器，用户态在退出时从 BSS 段读取并报告成功和失败率。
|
||||
|
||||
### 兼容性声明
|
||||
### 兼容性头文件
|
||||
|
||||
Linux 6.18 引入 BPF task work，6.19 引入 file-backed dynptr。仓库当前生成的 UAPI 与 BTF 头文件早于这些功能，因此本工具把缺失声明放在局部头文件 `bpf_experimental.h` 中。
|
||||
仓库的 vendored vmlinux 头文件早于 Linux 6.18/6.19，所以 `bpf_experimental.h` 在本地声明新接口：
|
||||
|
||||
```c
|
||||
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
|
||||
@@ -134,11 +171,11 @@ extern int bpf_dynptr_file_discard(struct bpf_dynptr *dynptr) __ksym;
|
||||
#endif /* __EXEC_IMAGE_INSPECTOR_BPF_EXPERIMENTAL_H */
|
||||
```
|
||||
|
||||
这个文件声明了 `bpf_task_work_schedule_signal`、`bpf_get_task_exe_file`、`bpf_put_file`、`bpf_path_d_path`、`bpf_dynptr_from_file` 和 `bpf_dynptr_file_discard` 等 kfunc。等仓库集成的 vmlinux 头文件包含这些声明后即可移除此文件。
|
||||
这些声明用 `__ksym` 标记为在加载时解析的内核符号。等仓库的 vmlinux 头文件从 6.19+ 内核重新生成后，这个文件可以移除。
|
||||
|
||||
### BPF 程序
|
||||
|
||||
`exec_image_inspector.bpf.c` 实现了 LSM hook 和 task work 回调。
|
||||
`exec_image_inspector.bpf.c` 实现 LSM 钩子和 task work 回调：
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: GPL-2.0
|
||||
@@ -341,23 +378,23 @@ void BPF_PROG(schedule_exec_inspection, struct linux_binprm *bprm)
|
||||
}
|
||||
```
|
||||
|
||||
程序的入口是 `SEC("lsm/bprm_committed_creds")` 声明的 `schedule_exec_inspection` 函数。`bprm_committed_creds` hook 在新可执行文件的凭据提交后运行，此时 `bprm->file` 指向本次 exec 涉及的镜像。这个 hook 本身不可睡眠，但我们可以在这里识别目标进程并安排 task work。
|
||||
入口点是用 `SEC("lsm/bprm_committed_creds")` 声明的 `schedule_exec_inspection`。这个 LSM 钩子在新可执行文件的凭据安装后触发，此时 `bprm->file` 指向正在被执行的文件。钩子本身不可睡眠，但可以识别目标并安排延迟工作。
|
||||
|
||||
两个 `const volatile` 变量 `target_tgid` 和 `probe_offset` 位于 `.rodata` 段，用户态在 `open()` 之后、`load()` 之前写入值，验证器将它们视为编译期常量。程序首先比较当前 TGID 与 `target_tgid`，不匹配则直接返回，避免观察无关进程的 exec。
|
||||
两个 `const volatile` 变量——`target_tgid` 和 `probe_offset`——位于 `.rodata` 段。用户态在 `open()` 和 `load()` 之间写入值，验证器把它们当作编译期常量来优化。
|
||||
|
||||
命中目标后程序从单元素 `pending` ARRAY map 查找 key 0 得到 `struct exec_work`。这个结构保存调度时间戳、直接探测结果和 `struct bpf_task_work` 存储。本工具一次调用只观察一个子进程，一个槽位已经足够。设置 `--probe-offset` 时，`probe_file_without_sleep` 在当前不可睡眠上下文中尝试从 `bprm->file` 读取指定位置的 8 字节。测试程序会把标记放到冷页位置，验证此时直接读取返回 `-EFAULT`。
|
||||
当目标匹配时，程序从单元素 `pending` ARRAY map 查找 `struct exec_work`。这个结构保存 `bpf_task_work` 存储以及时间戳和可选的直接探测结果。单个槽位就够了，因为本工具每次调用只观察一个子进程。
|
||||
|
||||
hook 保存时间戳和直接读取结果后调用 `bpf_task_work_schedule_signal`，传入当前 task、work 存储、map 指针与 `inspect_executable` 回调函数指针。内核持有回调执行所需的引用，回调会在该 task 返回用户态前的可睡眠上下文中执行。
|
||||
可选的 `--probe-offset` 标志让工具通过 `probe_file_without_sleep` 在不可睡眠的钩子中尝试直接读取。测试套件用这个来验证：读取冷页在不可睡眠上下文中以 `-EFAULT` 失败，但在可睡眠回调中成功。
|
||||
|
||||
`inspect_executable` 是 task work 回调函数，签名为 `int callback(struct bpf_map *map, void *key, void *value)`。它首先计算回调延迟（当前时间减去调度时间戳）用于诊断，填充 PID、TGID、进程名和直接探测结果到事件结构。然后调用 `bpf_get_task_exe_file` 获取 task 已安装的可执行文件。这个 kfunc 返回带引用的 `struct file`，每次成功获取都必须用 `bpf_put_file` 释放。
|
||||
保存时间戳和直接探测结果后，钩子调用 `bpf_task_work_schedule_signal`。内核持有稍后执行回调所需的引用。
|
||||
|
||||
回调先用 `bpf_path_d_path` 解析文件路径，再用 `bpf_dynptr_from_file` 创建 dynptr。因为现在处于可睡眠上下文，`bpf_dynptr_read` 可以触发缺页操作并等待 I/O 完成。回调只读取 64 字节 ELF 头部和可选的 8 字节标记。头部读取成功后程序检查 4 字节 ELF 魔数 `\x7fELF`，再解析 `EI_CLASS`（32 位或 64 位）、`EI_DATA`（小端或大端）、`e_type`（可执行文件或共享对象）和 `e_machine`（架构）。`read_elf_u16` 辅助函数根据字节序正确解析 16 位字段。
|
||||
回调 `inspect_executable` 计算延迟用于诊断，然后用 `bpf_get_task_exe_file` 获取可执行文件。这返回一个带引用的 `struct file`，必须用 `bpf_put_file` 释放。回调解析路径、创建 file dynptr、读取 64 字节 ELF 头部并解析它。`read_elf_u16` 辅助函数处理字节序：ELF 文件在头部声明其字节序，多字节字段必须相应地读取。
|
||||
|
||||
所有创建 dynptr 的路径都必须调用 `bpf_dynptr_file_discard` 释放其内部状态，包括 helper 失败的分支。最后 `bpf_ringbuf_output` 把事件发送给用户态，发送失败时递增 `dropped` 计数器。
|
||||
每条创建 dynptr 的路径——无论成功还是失败——都必须调用 `bpf_dynptr_file_discard`。最后，`bpf_ringbuf_output` 把事件发送给用户态。
|
||||
|
||||
### 用户空间加载器
|
||||
### 用户态加载器
|
||||
|
||||
`exec_image_inspector.c` 负责解析命令行、协调阻塞子进程、加载 BPF、格式化事件并清理资源。
|
||||
`exec_image_inspector.c` 协调子进程、加载 BPF、接收事件、报告结果：
|
||||
|
||||
```c
|
||||
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
|
||||
@@ -898,17 +935,15 @@ cleanup:
|
||||
}
|
||||
```
|
||||
|
||||
加载器的流程围绕阻塞子进程握手展开。`start_blocked_child` 在 BPF 设置完成前 fork，但子进程会阻塞在 pipe 读取上。父进程此时已经知道子进程 PID，可以在打开 skeleton 后把它写入 `rodata->target_tgid`。挂载完成后 `release_child` 向 pipe 写入一个字节让子进程继续执行 `execvp`。这个握手消除了短命命令带来的 attach 竞态。
|
||||
核心技术是阻塞子进程握手。`start_blocked_child` 在 BPF 设置前 fork，但子进程阻塞在 pipe 读取上。父进程记录子进程 PID、打开 skeleton、写入 `target_tgid`、加载并挂载、创建 ring buffer，然后才调用 `release_child` 写管道让子进程继续执行 `execvp`。
|
||||
|
||||
`setup_inspector` 函数打开 skeleton、设置只读数据、加载并挂载 BPF 程序、创建 ring buffer reader。加载失败时的错误信息会指出可能的原因，帮助用户排查内核版本或配置问题。
|
||||
`wait_for_command` 是主循环。它打印 `READY` 行、释放子进程，然后交替轮询 ring buffer 和检查子进程是否退出。当子进程被回收且至少收到一条事件时循环结束，或者超时时结束。超时后，`reap_timed_out_child` 发送 SIGKILL 并等待。
|
||||
|
||||
`wait_for_command` 是主循环。它先打印 `READY` 行表示工具已就绪，然后释放子进程、设置超时 deadline、进入 poll 循环。循环中交替调用 `ring_buffer__poll` 接收事件和 `waitpid(WNOHANG)` 检查子进程状态。子进程回收且至少收到一条事件时结束循环，或者等到超时。超时后 `reap_timed_out_child` 向子进程发送 SIGKILL 并等待其退出。
|
||||
单次 exec 可能触发多条事件，如果命令本身也调用 exec（例如 `/bin/sh -c 'exec /bin/true'`）。子进程退出后，`drain_events` 用零超时 poll 取尽剩余事件。
|
||||
|
||||
同一子进程可能连续安装多个镜像。例如运行 `/bin/sh -c 'exec /bin/true'` 时，shell 先 exec 自身，然后 exec `/bin/true`。子进程回收后 `drain_events` 用零超时 poll 取尽 ring buffer 中的剩余事件，确保最后一条镜像事件也能到达用户态。
|
||||
`handle_event` 格式化输出，把数字 ELF 值转换为可读名称，同时保留原始值供脚本使用。
|
||||
|
||||
`handle_event` 解析 ring buffer 事件并打印格式化输出。它把 ELF 数值转换成可读名称如 `ELF64`、`LSB`、`ET_DYN`、`EM_X86_64`，同时保留原始数值。设置 `--probe-offset` 时还会打印 `PROBE` 行显示探测结果。
|
||||
|
||||
## 编译与运行
|
||||
## 构建与运行
|
||||
|
||||
从源码构建：
|
||||
|
||||
@@ -918,13 +953,13 @@ make -C src/54-exec-image-inspector clean
|
||||
make -C src/54-exec-image-inspector -j2
|
||||
```
|
||||
|
||||
集成测试需要 Linux 6.19 或更新版本，执行前请先确认活动 LSM 列表中包含 `bpf`：
|
||||
测试套件要求 Linux 6.19 或更新版本，且 BPF LSM 处于活动状态。检查 `bpf` 是否出现在 LSM 列表中：
|
||||
|
||||
```bash
|
||||
cat /sys/kernel/security/lsm
|
||||
```
|
||||
|
||||
要添加 `bpf`，请在内核启动参数中把 `lsm=<existing-list>` 改成 `lsm=<existing-list>,bpf`。
|
||||
如果缺少 `bpf`，在内核命令行中添加它：把引导加载器配置中的 `lsm=<existing-list>` 改为 `lsm=<existing-list>,bpf`。
|
||||
|
||||
运行测试：
|
||||
|
||||
@@ -933,7 +968,7 @@ cd src/54-exec-image-inspector
|
||||
sudo make test
|
||||
```
|
||||
|
||||
仓库 CI 只编译本课。运行时行为在 x86_64 上通过功能测试，内核版本 `7.0.0-rc2+`。下面的会话记录给出测试框架和工具输出：
|
||||
仓库 CI 只编译本课。运行时行为在 x86_64 上通过功能测试，内核版本 `7.0.0-rc2+`。示例输出：
|
||||
|
||||
```text
|
||||
TEST-MISSING matched=0 events=0 command_exit=127
|
||||
@@ -947,45 +982,52 @@ SUMMARY matched=1 scheduled=1 schedule_errors=0 callbacks=1 header_errors=0 path
|
||||
PASS: missing-command, timeout cleanup, re-exec drain, ELF decode, and deferred file read succeeded
|
||||
```
|
||||
|
||||
前三条测试记录覆盖重要边界：缺失命令以状态 127 退出，此时 committed-exec hook 尚未触发，因此 `matched=0`、`events=0`；超时用例产生 exec 事件后超过测试时限，由 SIGKILL 终止并以状态 137 回收；连续 exec 用例取尽两条镜像事件，确认 `/usr/bin/true` 最后出现。
|
||||
前三行测试覆盖边界条件：
|
||||
|
||||
`EXEC` 行给出已安装镜像和 ELF 头部解析结果。测试程序在 exec 前刷新并驱逐标记所在页面，hook 中直接读取返回 `-EFAULT`（`-14`），而 task-work 回调成功返回 `454950524f424521`（`EIPROBE!` 的十六进制字节）。这验证了不可睡眠上下文无法读取冷页，而可睡眠上下文可以。
|
||||
- **TEST-MISSING**：不存在的命令以状态 127 退出。LSM 钩子从未触发，因为 exec 在凭据提交前就失败了，所以 `matched=0` 且 `events=0`。
|
||||
- **TEST-TIMEOUT**：命令运行太久，被 SIGKILL 终止，以状态 137（128 + 9）回收。观察到一条 exec 事件。
|
||||
- **TEST-REEXEC**：内部使用 `exec` 的 shell 命令产生两条事件，最终路径是 `/usr/bin/true`。
|
||||
|
||||
检查其他命令时可省略测试程序专用的探测偏移：
|
||||
`EXEC` 行显示已安装的镜像和解析的 ELF 字段。`PROBE` 行展示了可睡眠上下文的差异：测试向一个页面写入标记（`EIPROBE!`），刷新并从页缓存驱逐它，然后 exec。不可睡眠钩子中的直接读取返回 `-EFAULT`（`-14`），因为页是冷的。可睡眠回调中的延迟读取成功，返回标记字节的十六进制表示（`454950524f424521`）。
|
||||
|
||||
检查简单命令：
|
||||
|
||||
```bash
|
||||
sudo ./exec_image_inspector --timeout-ms 3000 -- /bin/true
|
||||
```
|
||||
|
||||
命令行格式如下：
|
||||
命令行格式：
|
||||
|
||||
```text
|
||||
exec_image_inspector [--probe-offset BYTES] [--timeout-ms MS] [--verbose] -- COMMAND [ARG...]
|
||||
```
|
||||
|
||||
`--timeout-ms` 接受 100 至 60000 毫秒，默认值为 5000 毫秒，超过时限后加载器会终止并回收命令。`--probe-offset` 比较指定位置的 8 字节直接读取与延迟读取，用于验证冷页场景。`--verbose` 输出 libbpf 诊断，便于排查 load 或 attach 问题。`--` 明确结束 inspector 选项解析，后面的参数全部属于被观察命令。
|
||||
- `--timeout-ms`：100 到 60000 毫秒（默认 5000）。超过此期限后工具会终止并回收命令。
|
||||
- `--probe-offset`：在钩子中（直接）和回调中（延迟）分别读取此偏移处的 8 字节，以验证冷页差异。
|
||||
- `--verbose`：打印 libbpf 诊断信息。
|
||||
- `--`：分隔 inspector 选项和要运行的命令。
|
||||
|
||||
### 环境要求
|
||||
|
||||
| 要求 | 详情 |
|
||||
|---|---|
|
||||
| 内核版本 | Linux 6.19+（BPF task work 在 6.18 引入，file-backed dynptr 在 6.19 引入） |
|
||||
| 内核版本 | Linux 6.19+（BPF task work 在 6.18 引入，file dynptr 在 6.19 引入） |
|
||||
| 内核配置 | `CONFIG_BPF=y`, `CONFIG_BPF_SYSCALL=y`, `CONFIG_BPF_JIT=y`, `CONFIG_BPF_LSM=y`, `CONFIG_SECURITY=y`, `CONFIG_DEBUG_INFO_BTF=y` |
|
||||
| 活动 LSM | `/sys/kernel/security/lsm` 中包含 `bpf` |
|
||||
| 活动 LSM | `/sys/kernel/security/lsm` 必须包含 `bpf` |
|
||||
| 架构 | 已在 x86_64 上测试 |
|
||||
| 权限 | root |
|
||||
|
||||
## 适用范围
|
||||
## 局限性与扩展
|
||||
|
||||
本工具观察自身创建的一个直接子进程，单个 map 槽位适合这种 CLI 场景。如果要扩展到并发服务，可以改为按 task 分配状态、设置准入上限并回收待处理回调。超时清理向该子进程发送 SIGKILL，调用方可以添加进程组管理和外部信号处理来覆盖更复杂的场景。
|
||||
本工具每次调用观察一个直接子进程，使用单个 map 槽位。对于并发服务场景，需要按任务分配状态、实现准入限制、处理回调回收。超时清理只向子进程发送 SIGKILL；需要进程组管理或外部信号处理的调用方需自行添加这些功能。
|
||||
|
||||
## 总结
|
||||
|
||||
本教程展示了如何使用 BPF task work 和 file-backed dynptr 检查 exec 后真正安装的可执行镜像。相比读取 `/proc/<pid>/exe` 或在 tracepoint 中获取路径，这种方案可以在可睡眠上下文中读取文件内容，即使目标字节位于冷页也能完成。
|
||||
本教程展示了如何结合 BPF task work 和 file dynptr 来检查 exec 实际安装的可执行镜像。关键思路是分离两个时刻：识别目标（在不可睡眠的 LSM 钩子中）和读取文件内容（在可睡眠的 task work 回调中）。这让 eBPF 程序能可靠地读取文件数据，即使目标字节不在页缓存中也能完成。
|
||||
|
||||
工具的核心思路是把识别目标进程的时刻（不可睡眠的 LSM hook）与读取文件内容的时刻（可睡眠的 task work 回调）分开。阻塞子进程握手消除了 attach 竞态，有界命令运行和最终 drain 确保所有事件都能到达用户态，显式资源释放保证清理完整。这些内核功能组合成了一个可复现的单命令工具，同时为并发场景留出扩展空间。
|
||||
阻塞子进程握手消除了 attach 竞态，有界执行确保清理完整，最终 drain 捕获所有事件。这些技术组合产生一个可复现的单命令工具，同时为扩展留出空间。
|
||||
|
||||
> 如果你想深入了解 eBPF，请查看我们的教程代码仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或访问我们的网站 <https://eunomia.dev/tutorials/>。
|
||||
> 要深入了解 eBPF，请访问我们的教程仓库 <https://github.com/eunomia-bpf/bpf-developer-tutorial> 或 <https://eunomia.dev/tutorials/>。
|
||||
|
||||
## 参考资料
|
||||
|
||||
|
||||
Reference in New Issue
Block a user