Upstream Rust maintenance report

In August and September 2026, the open source Rust toolchain continued to see significant progress under the leadership of the Sovereign Tech Fellow. A key focus was reducing the size of the target directory by eliminating duplicated metadata of Rust crates. This could potentially shrink real-world projects by 5-35%.

During this period, the experiment of embedding metadata by default on the nightly channel was finalized. This change, which removed the requirement to pass metadata explicitly to rustc via the --extern flag, was met with positive feedback and enabled several build systems to transition to the new mechanism. However, the team faced challenges in synchronizing this change across both rustc and Cargo, which currently operate on different versions of the flag.

The plan was to maintain support for both flags temporarily while users migrated to the stable version, after which the old flag would be phased out.

The team also experimented with applying arena allocation to various parts of the compiler, aiming to improve performance. However, this approach faced challenges due to the complex structure of the Rust compiler codebase and concerns about the language's inherently hostile design towards arena allocation. The Allocator trait is slated for stabilization, which may alleviate these issues in the future.

Additionally, the reporter explored similar optimization techniques used in other compilers, such as Zig's use of Data Oriented Design/Entity Component System for efficient parsing. Inspired by this, the reporter attempted to implement a flat representation of the token stream in the Rust compiler to avoid the overhead of allocating new Vecs for each token tree.

This change resulted in significant improvements in lexing performance. However, transitioning this approach to the parser, macro expansion, and AST handling proved to be a complex task, requiring the creation of a parallel implementation using the flat token structure and conversion functions to bridge the old and new representations.