Don't Enqueue What You Cannot Reserve

Enqueueing before confirming reservation is a crucial step to avoid unintended usage of limited resources. In both paid and free models, an attempt consumes input, output, wait time, and potentially a retry if the client interprets silence as failure. The hidden cost is that the HTTP client may timeout, leaving the server still generating or waiting in a queue it doesn't control.

Retry attempts compound the issue by associating two unrelated tasks, making it impossible to accurately trace which transcript corresponds to which click. Free capacity exacerbates this problem because it appears safe to disregard the issue. The duplicate still occupies the slot needed for the actual job and exhausts the generated context that cannot be easily recreated once the window closes.

Waiting for the queue to process is not a refundable deposit, nor is the prompt that has already departed. A simple fix is to estimate input, reserve an output capacity, and allocate slack for the acceptable wait time. If these reservations exceed the remaining budget, the job should be aborted in the initiating process, where failure is less costly.

This approach ensures a single reservation, a unique attempt ID, and no subsequent calls from a timeout scenario. Implement a character heuristic to estimate tokens, replacing it with the actual tokenizer used for the model once confidence in the estimation is established. The proposed control flow involves one reservation, one attempt ID, and no secondary calls triggered by timeouts.