How do programming languages impact token efficiency and correctness?

This post argues that dynamic languages and languages that represent concepts more concisely tend to be more token-efficient, meaning they require fewer tokens to express the same functionality compared to static languages. For example, dynamically typed languages often have lower LLM token costs than statically typed languages due to the omission of explicit type declarations, making the code more compact.

The author cites a post claiming that some concise dynamic languages can have 1/2 to 1/3 the token cost of static languages like Rust, Go, C++, etc. They note a significant gap of 2.6x between the least token-efficient language (C) and the most token-efficient (Clojure), where Clojure averages just 70 tokens, nearly half of Clojure's 109 tokens.

The author also discusses other comparisons between dynamic and static languages in terms of token efficiency. They caution that trivial problems may not accurately reflect the performance of languages when solving more complex tasks. Furthermore, they point out potential issues with some of the evals, such as tests executing the wrong path or tests running the wrong executable if a test fails.

Despite these concerns, the author suggests that running their own evaluations could provide more insight into the relationship between token efficiency and correctness.