LLM Apps in Python: A Production Stack Without Framework Bloat
LLM Apps in Python: A Production Stack Without Framework Bloat The first LLM feature I shipped that actually paid for itself was about 200 lines of Python and one SDK. The one that cost me a weekend of debugging was a framework-based pipeline where I could not tell which layer had swallowed a retry. Since then I have built agents, RAG pipelines and scheduled workers that run unattended, and my…
Building Production-Ready LLM Apps in Python: A Stack Without Framework Overhead
The author shares their experience shipping their first LLM feature, which paid for itself with just 200 lines of Python and one SDK. In contrast, a framework-based pipeline that cost them a weekend to debug proved that a minimal stack could be effective.
Starting with the raw SDK and understanding the specific problems a framework would solve is advised. Most LLM apps consist of an HTTP call with structured input, a loop over tool calls, and some state, making the core code relatively small. Owning this core means being in control of failures, which can be challenging in framework-based solutions.
A simple agent example is provided, showcasing how a framework can obscure the inner workings. The max_turns cap is essential to prevent unbounded spending loops. The author reaches for frameworks only when dealing with complex requirements, such as durable graphs, human-in-the-loop pauses, or when a team needs a shared vocabulary for orchestration.
The default stack consists of using the raw SDK for single-purpose LLM features or multi-step agents running unattended. For longer workflows with pauses and approvals, an orchestration layer is used. The core principles involve treating prompts and tool calls as versioned artifacts with a typed contract. Pydantic is used for validation, ensuring structured, bounded results. Errors should be returned as tool results with a message for the model to act upon, rather than raising exceptions.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
