Changing def to async def let one blocking call freeze an entire FastAPI app

The article examines the impact of changing function definitions from synchronous to asynchronous in a FastAPI application. It demonstrates that blocking calls in an asynchronous endpoint can freeze the entire server, while they have no effect in a synchronous endpoint. The author provides a setup using Python 3.14.4, FastAPI 0.143.0, Starlette 1.7.0, uvicorn 0.54.0, and anyio 4.15.1, with a slow third-party API that takes exactly one second to respond to each request.

The author creates four endpoints in the FastAPI application, each calling the slow API in different ways. They use a load generator to test the performance of the "/fast" endpoint while 20 clients hit it concurrently. The results show that the "/fast" endpoint takes only 1.6 milliseconds to respond in a plain synchronous function, but freezes the entire server in an asynchronous function, taking up to 34,130 milliseconds.

Even for a trivial endpoint that simply returns a constant, the blocking call in the asynchronous function significantly slows down the response time. The author highlights that common fixes for blocking calls, such as changing the endpoint to asynchronous without modifying the libraries used, can actually make the problem worse.

The article also explores the impact of CPU-bound tasks in an asynchronous context. It finds that moving CPU-intensive work to a thread using asyncio.to_thread or a process pool can help reduce the impact on other endpoints. However, it notes that threads are not free and offer only limited performance improvements. The author suggests a possible solution by adjusting the thread pool size at startup, which can provide additional headroom for handling a higher number of slow requests.