Let Them Write RFCs

Here's my attempt at a report based on the provided source material:

For a complex project that exceeds my current expertise, I found myself spending hundreds of millions of tokens to create four versions of the same markdown file. This experience highlights the challenges of working on unknown problems, even with the assistance of large language models (LLMs).

Some argue that tackling unfamiliar tasks without sufficient understanding is unwise. However, I believe that actively working on a problem is what leads to true comprehension. Yet, LLM usage can create a cycle where friction is minimized, allowing continued work without fully grasping the underlying concepts, potentially leading to misguided solutions.

The key insights that helped me navigate this issue were:

1. Discarding early, incomplete implementations that didn't work well. Building bad drafts reveals crucial problems and accelerates learning.

2. Employing LLMs to generate RFCs (request for comments) that articulate the problem, proposed solutions, and alternatives. This forces me to explain the problem and solution in a clear, concise manner.

I posed the model with questions like "Write an RFC about this problem and how to solve it. Keep it short, include good examples." By iterating on successive RFC versions (V1, V2, etc.), I consulted documentation and researched as needed until I had a solid understanding of the problem and confidence in the proposed ideas.

An RFC serves to clarify the problem, expose hidden assumptions, and guide the development process. However, the final code implementation may still fall short of expectations. In such cases, I would review the diff, compare it to the RFC, and either revise the code or start anew with a new RFC.

While this process is demanding, with multiple RFC iterations potentially consuming hundreds of millions of tokens, I found it more useful and enjoyable than simply working on the problem directly. The method minimizes sloppily written code and promotes focused learning. It's a valuable approach for complex projects where understanding the problem is paramount.