Debugging Type-Based Alias Analysis optimizations in BPF

During a routine update to a BPF program, internal integration tests began failing due to packet drops. After some investigation, the issue was traced back to bad L3 checksums. The update involved switching to BTF generated vmlinux.h definitions instead of relying on specific kernel headers. It was discovered that the only change was an update to the Type of Service (ToS) field, which was immediately re-computed.

However, the checksum calculation required incrementally updating the checksum based on the difference between the old and new 16-bit words.

Upon further inspection, it was found that the code was not re-calculating the ToS value after writing the new value to the IP header. Instead, it was simply assigning the old value to the new one. This suggests that the issue might be due to Clang's Strict Aliasing Rules. Since uint16_t * and struct iphdr * are different types, Clang assumes that modifying iph- tos does not affect the memory referenced by ((uint16_t *)iph).

As a result, Clang aggressively optimizes away the second memory read and reuses the cached value of the old ToS value.

To verify this, the -fno-strict-aliasing flag was passed to Clang when compiling, which disabled these optimizations and allowed the correct L3 checksum to be recomputed. However, the team did not want to turn off all optimizations as some were useful. Instead, they used READ_ONCE, a wrapper around the barrier macro, to tell the compiler to reread the memory.

The investigation revealed that the issue was related to LLVM's Alias Analysis. The difference between the original and CO-RE generated versions of the struct iphdr led to incorrect alias analysis, causing the compiler to optimize away the necessary memory read. By using the llvm.bpf.preserve.struct.access.index intrinsic function, the alias analysis correctly identified the offset overlap, ensuring the checksum was recalculated accurately.