{
  "id": 1606881,
  "title": "Retrofitting a build system into a compiler",
  "url": "https://urgent.news/2026/08/17/retrofitting-a-build-system-into-a-compiler",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-08-17T22:42:21.000Z",
  "source": {
    "name": "Lobsters",
    "slug": "lobsters",
    "url": "https://www.dra27.uk/blog/platform/2025/09/25/building-with-effects.html"
  },
  "original_language": "en",
  "account": "During the summer, Lucas Ma explored the possibility of using effects in the OCaml compiler itself, with the aim of converting it into a library on-demand for creating a longer-lived \"compiler service\". Though not entirely revolutionary, this task proved challenging due to the compiler's 30-year-old design, which was primarily built for single-shot separate compilation. Lucas began by focusing on OCaml's build system and specifically examining the Load_path, an internal component responsible for searching \"include\" directories for files, mainly for type information. Effects offered a way to invert the control for file lookups, enabling the program calling the compiler to modify the lookup process. This capability allowed Lucas to manipulate the dependency graph by ignoring it.\n\nOne of the most intriguing aspects of Lucas's work was the use of effects to package up a suspended compilation within a continuation. This allowed the compiler process to perform other tasks, such as compiling interface files directly instead of spawning new instances. However, the 30-year-old codebase posed a significant challenge, as many parts of the compiler relied on global mutable state, making it non-re-entrant. To address this issue, Lucas leveraged the Merlin project's snapshotting mechanism, enabling the compiler to save and restore its global state before and after performing effects. This innovation facilitated the compilation of the OCaml bytecode compiler without pre-existing components, effectively retiring ocamldep.\n\nMoreover, Lucas introduced the concept of Domains, which enabled the creation of multiple compiler processes that could compile .ml files concurrently. Each process utilized the Local_store mechanism, allowing for domain-specific storage and snapshotting. By implementing this approach, Lucas achieved a nearly functional prototype, significantly surpassing his initial expectations given the limited time available. The most exciting outcomes of this investigation were the potential for a multi-threaded compiler interface and the prospect of developing a custom build system that could outperform the existing make-based build process.",
  "summary": null,
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}