Challenges when writing gren-format

A short tour of the problems we encountered while creating a code formatter for Gren, and how we addressed each issue. For a more detailed analysis, refer to the full comment story in Putting the comments back.

The formatter was constructed using the compiler's v2 parser, which generates a syntax tree lacking comments and certain punctuation tokens. The complications stemming from this include reinserting comments accurately, maintaining consistent results on subsequent runs, and testing a tool whose only specification is "it looks right."

gren-format relies solely on the compiler's v2 parser, as Gren is a language still under development. A formatter with its own parser would eventually diverge from the compiler's interpretation of valid code. By utilizing the compiler's parser, the formatter can always accept what the compiler accepts, ensuring consistency.

Comments are absent from the syntax tree, as both -- and {- -} comments are excluded, leaving only doc comments {-| -} in the tree. Punctuation marks such as =, :, ,, |, - , then, in and friends lack nodes and positions, forcing the formatter to make assumptions about their locations without access to the source code.

To ensure consistent formatting, the formatter adopts a rule: "the comment lands after the separator." This decision is recorded as one of the roles in section 3 of the formatter's design. Unfortunately, this choice led to various bugs, as the author's row numbers were accessible to the formatter, leading to incorrect assumptions about code layout.

The primary challenge in comment placement is that it relies on source positions, while formatting invalidates these positions. Comments contain start and end positions in separate data structures, and the formatter must determine their placement based on these positions. To resolve this, the formatter decides comment placement once during the parsing stage, storing this information as a role on the comment.

This approach eliminates the need for downstream components to recompute comment positions, reducing potential errors.