Parsing Expression Grammar vs. regexes: Building Org parser in Lisp that exports to HTML (via SXML)

In this blog post, the author describes their journey in building OrgWebAlchemy, a Guile Scheme library for parsing Org-mode documents into an Abstract Syntax Tree (AST) and rendering them to HTML. Initially, the author tried using regular expressions for parsing, but soon realized that a more sophisticated approach was needed. This led them to explore Parsing Expression Grammars (PEGs) using Guile's (ice-9 peg) module and the tutorials available.

The author appreciates the readability of PEG grammars as documentation for the language and the benefit of having the grammar and HTML rendering as separate concerns. This allows for future expansion to support other formats like Markdown.

The parsing process begins with a flat sequence of list items, which are then organized into a nested structure in the AST based on indentation levels. The HTML renderer then takes the AST and produces the final HTML output. Although the library currently supports a subset of Org-mode features, the author is confident about its stability after reaching version 1.0.

The SXML format is used for representing the HTML output because, being within the Lisp ecosystem, it allows for a more natural representation of the markup tree. The author emphasizes the flexibility of allowing full customization of the output HTML through Guile parameters, enabling easy integration into various projects.

The project is open-source, licensed under the GNU LGPL v3 or later, and the code is available on Codeberg. The author invites feedback on various aspects of the project, including the grammar, AST design, parser architecture, and any Org constructs they may have overlooked. The inclusion of a test suite in the repository serves as a safety net and showcases the parser's capabilities.