mirror of
https://github.com/eunomia-bpf/bpf-developer-tutorial.git
synced 2026-03-20 11:56:22 +08:00
Fix curl hanging in XDP load balancer: backend servers reject mismatched Host headers (#198)
* Initial plan * Add simple HTTP server and update docs to fix curl hanging issue Co-authored-by: yunwei37 <34985212+yunwei37@users.noreply.github.com> * Fix code review issues: remove unnecessary shutdown() and fix spacing Co-authored-by: yunwei37 <34985212+yunwei37@users.noreply.github.com> * Add test script to verify HTTP servers handle mismatched Host headers Co-authored-by: yunwei37 <34985212+yunwei37@users.noreply.github.com> * Add comprehensive solution documentation for curl hanging issue Co-authored-by: yunwei37 <34985212+yunwei37@users.noreply.github.com> * Co-authored-by: yunwei37 <34985212+yunwei37@users.noreply.github.com> * Update src/42-xdp-loadbalancer/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: 云微 <1067852565@qq.com> * Update src/42-xdp-loadbalancer/README.zh.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: 云微 <1067852565@qq.com> * Update src/42-xdp-loadbalancer/SOLUTION.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: 云微 <1067852565@qq.com> * Delete src/42-xdp-loadbalancer/__pycache__/simple_http_server.cpython-312.pyc Signed-off-by: 云微 <1067852565@qq.com> * Delete src/42-xdp-loadbalancer/SOLUTION.md Signed-off-by: 云微 <1067852565@qq.com> --------- Signed-off-by: 云微 <1067852565@qq.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: yunwei37 <34985212+yunwei37@users.noreply.github.com> Co-authored-by: 云微 <1067852565@qq.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -470,9 +470,20 @@ You can test the setup by starting HTTP servers on the two backend namespaces (`
|
||||
|
||||
Start servers on `h2` and `h3`:
|
||||
|
||||
**Important**: The HTTP servers must bind to `0.0.0.0` so they listen on all local addresses in the namespace and can accept connections arriving via the load balancer. The forwarded HTTP requests will use the load balancer's virtual IP and port in the `Host` header (for example, `Host: 10.0.0.10:8000`), which may differ from the backend's own IP address; for this tutorial we use simple HTTP servers that accept such requests without enforcing strict `Host` header checks.
|
||||
|
||||
**Option 1**: Using the provided simple HTTP server (recommended):
|
||||
|
||||
```sh
|
||||
sudo ip netns exec h2 python3 -m http.server
|
||||
sudo ip netns exec h3 python3 -m http.server
|
||||
sudo ip netns exec h2 python3 simple_http_server.py &
|
||||
sudo ip netns exec h3 python3 simple_http_server.py &
|
||||
```
|
||||
|
||||
**Option 2**: Using Python's built-in http.server with explicit binding:
|
||||
|
||||
```sh
|
||||
sudo ip netns exec h2 python3 -m http.server --bind 0.0.0.0 &
|
||||
sudo ip netns exec h3 python3 -m http.server --bind 0.0.0.0 &
|
||||
```
|
||||
|
||||
Then, send a request to the load balancer IP:
|
||||
@@ -483,6 +494,8 @@ curl 10.0.0.10:8000
|
||||
|
||||
The load balancer will distribute traffic to the backends (`h2` and `h3`) based on the hashing function.
|
||||
|
||||
> **Note**: If you experience hanging requests with `curl`, ensure the backend HTTP servers are bound to `0.0.0.0` and accept requests with any Host header. The XDP load balancer operates at Layer 3/4 (IP/TCP) and does not modify HTTP headers, so the Host header in requests will still show `10.0.0.10:8000` even though packets are forwarded to the backend IPs (10.0.0.2 or 10.0.0.3).
|
||||
|
||||
### Monitoring with `bpf_printk`
|
||||
|
||||
You can monitor the load balancer's activity by checking the `bpf_printk` logs. The BPF program prints diagnostic messages whenever a packet is processed. You can view these logs using:
|
||||
@@ -507,6 +520,24 @@ Example output:
|
||||
|
||||
### Debugging Issues
|
||||
|
||||
#### Curl Requests Hanging
|
||||
|
||||
If `curl` requests to the load balancer hang and never complete, the most likely cause is that the backend HTTP servers are rejecting requests with mismatched Host headers.
|
||||
|
||||
**Problem**: When you run `curl 10.0.0.10:8000`, the HTTP request includes a Host header set to `10.0.0.10:8000`. The XDP load balancer forwards the packet at Layer 3/4 (IP/TCP) to a backend server (10.0.0.2 or 10.0.0.3), but the HTTP headers remain unchanged. If the backend HTTP server validates the Host header and expects it to match its own IP address, it may reject or drop the request.
|
||||
|
||||
**Solution**: Ensure backend HTTP servers bind to `0.0.0.0` and accept requests with any Host header:
|
||||
- Use `python3 simple_http_server.py` (provided in this directory), or
|
||||
- Use `python3 -m http.server --bind 0.0.0.0`
|
||||
|
||||
**Verification**: Check the backend server logs to see if requests are being received. You can also use `tcpdump` in the backend namespace to verify packets are arriving:
|
||||
|
||||
```sh
|
||||
sudo ip netns exec h2 tcpdump -i veth2 -n port 8000
|
||||
```
|
||||
|
||||
#### XDP Packet Forwarding Issues
|
||||
|
||||
Some systems may experience packet loss or failure to forward packets due to issues similar to those described in this [blog post](https://fedepaol.github.io/blog/2023/09/11/xdp-ate-my-packets-and-how-i-debugged-it/). You can debug these issues using `bpftrace` to trace XDP errors:
|
||||
|
||||
```sh
|
||||
|
||||
Reference in New Issue
Block a user