Why we moved our Backstage platform from Yarn to pnpm

Bol, a company utilizing the Backstage platform for five years, decided to switch their internal developer platform from Yarn to pnpm. The move aimed to improve parallel development using Git worktrees and enhance the package manager's efficiency. However, the migration turned out to be more complex than anticipated.

During the switch, CI runs revealed significant issues with the package manager's cache. The cache was writing to the wrong location in GitLab, causing each job to start from scratch. The solution was to set the correct cache location explicitly in the CI configuration, ensuring jobs could reuse the store effectively.

Another challenge arose when pnpm's archive grew to 1.42 GB due to cached node_modules trees. This cache was not providing the expected performance benefits as it still rebuilt dependency layouts and native modules. By removing node_modules from the cache paths, the archive size reduced by 72% and file count by 59%, saving approximately six minutes per pipeline.

Moving to pnpm also posed challenges to the custom worktree machinery used for local development. The existing symlinking logic became incompatible with pnpm's virtual store and links, leading to ENOTDIR failures. To resolve this, the custom logic was removed, and the process was simplified to a single command: pnpm install --frozen-lockfile. This change allowed each checkout to get its own installation while still benefiting from the shared store.

In conclusion, the migration from Yarn to pnpm presented several challenges, including cache misconfiguration and compatibility issues with the custom worktree machinery. By addressing these issues and optimizing the configuration, the company successfully transitioned to the new package manager, achieving better performance and a more efficient development workflow.