Why Your OpenAI JSON Calls Randomly Fail with "could not parse JSON body" (And How to Fix It)

You are calling OpenAI from Node.js. Ninety-nine percent of requests work correctly. However, randomly, you may receive a BadRequestError indicating that the JSON body could not be parsed. This error may also manifest as a SyntaxError reporting an invalid JSON object or, if you are utilizing tool calling, an Invalid JSON in tool call arguments.

When you review your code, you will find no issues. Simple retries appear to resolve the problem temporarily. Upon redeployment, the errors resurface. The issue lies in transient artifacts from proxy or network corruption, where the request body arrives at OpenAI partially damaged. The SDK may parse JSON prematurely, before the stream signals completion, particularly when max_output_tokens is set to a low value.

Additionally, model-side tool call malformation can occur, leading to single quotes, trailing commas, or markdown code fences within function.arguments. These intermittent issues cannot be debugged using stack traces. The incorrect approach is to retry errors that should not be retried, such as invalid API keys, rate limits requiring backoff, or context length exceeded.

Sanitizing tool call arguments is also overlooked, resulting in downstream crashes with malformed JSON. The proper solution involves classifying and acting on different errors accordingly. Transient 400 errors, caused by could not parse JSON body, should be retried with backoff and jitter. Truncated streams, indicated by EOF while parsing an object, warrant retries, possibly with higher max_tokens.

Malformed tool arguments require sanitization without any retries. Invalid API keys should result in an immediate failure, without retry attempts. Rate limits should be managed with longer backoff periods. Implementing this manually requires approximately 150 lines of potentially flawed code. Alternatively, you can utilize the @coder12-z/llm-shield library, which provides a simple solution requiring only three lines of configuration.

This library supports OpenAI, Anthropic, Gemini, and other services that throw Error objects with status and message. However, it does not repair arbitrary JSON strings or make LLM calls itself. For more details, visit the npm page, GitHub repository, or landing page. The library is licensed under MIT and welcomes feedback.