Static Allocation, Constant Work

Static allocation and constant work are key concepts in creating robust systems. Object pools clarify the relationship between memory safety and correctness. Using malloc and free can lead to use-after-free errors, which can result in arbitrary code execution. Introducing an object pool for dead objects of type T helps avoid type confusion and ensures deterministic behavior.

Typed allocation functions can write allocators that use type-segregated pools, improving memory efficiency and locality, though it may result in a small memory overhead. Inline enums may break this approach, but heap allocating enum variants can still work.

Generational indexes are a popular remedy for preventing bugs, but the author hasn't personally used them. Instead, they share two tricks from TigerStyle that can help avoid bugs in systems like order matching engines. One approach is to specify a maximum number of orders at startup and reject any surplus requests. This ensures the system operates at capacity without strict limits, preventing catastrophic failures.

Another approach is to design the system to always have a fixed amount of orders by introducing a no-op, neutral order. This allows for cognitive benefits, such as thinking in terms of orders circulating in the system rather than being created and destroyed. It also simplifies code and improves performance, as the compiler can better vectorize and prefetch data.

The Constant Work principle ensures consistent performance, regardless of the load, making it easier to discover insufficient performance issues during rollout rather than during critical events like Black Friday.