My HTML-to-Markdown output passed every test I wrote — then flattened in a strict parser
An agent piped a wiki export through my HTML-to-Markdown converter into a RAG pipeline and reported that hierarchy was disappearing. Every nested task list came out flat: sub-items rendered as siblings of their parents. Sections got chunked together that had nothing to do with each other, and retrieval started returning paragraphs attributed to the wrong topic. Here's the part that stung: the…
A developer discovered an issue with their HTML-to-Markdown converter after running a series of tests. The converter passed all the tests, but the output flattened nested list items into flat lists when fed into a strict parser. The problem stemmed from the converter indents nested list items using two spaces, which is accepted by lenient parsers and most browser previews but not by strict CommonMark parsers.
The developer's test suite only checked the output against one parser, a lenient one, and this proved insufficient. The fix was simple, but the diagnosis was difficult. To ensure compatibility with different parsers, the developer now runs a small harness in CI that converts HTML to Markdown, re-parses the result with three different Markdown implementations, and compares the resulting trees. If there is any disagreement, the build fails.
This experience highlighted that Markdown is not a single format but a family of dialects with varying interpretations of indentation, heading style, emphasis, and tables. The developer's converter, available at https://x402.freeq.one/tools/html_to_markdown.html, provides options for heading style, bullet marker, and code fence to accommodate the various parsers that might be encountered.
The takeaway is that when converting HTML to Markdown for use in RAG pipelines, it is crucial to test the round trip with the specific parser that will be used at the end of the pipeline, not just the one used for initial testing.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
