"A second operation was started on this context" — the EF Core bug that only shows up under load

In a recent development, a second operation was initiated regarding the EF Core bug that manifests under load conditions. Developers have encountered this issue before, known as InvalidOperationException: A second operation was started on this context before a previous operation completed. Despite working locally, the problem arises when the application faces actual production load.

The root cause lies in a service-lifetime design issue that only becomes apparent under real concurrency. The problem stems from a fire-and-forget context leak in a hosted background service. A single scoped DbContext is being reused across parallel work items, but what is being held onto is the context itself after its scope has expired. This leaves the context in an invalid state, leading to the error when subsequent operations attempt to use it.

Local testing environment lacks the necessary parallelism to reproduce this bug. Errors appear intermittently, as the context's internal state gets corrupted over time, making the error seem random. Fire-and-forget tasks are often overlooked in debugging because they are out of sight and out of mind.

A common misconception is that EF Core's thread-safety rules apply to explicit parallelism, such as Task.WhenAll. However, EF Core's thread-safety is actually about lifetime. A DbContext must only live within its scoped lifetime. If a task holds onto it after the scope ends, the context's internal state becomes invalid, leading to the bug.

To resolve this issue, the correct approach is to ensure one scope per unit of work. The context should be resolved inside the scope, not per service instance. Fire-and-forget tasks must not capture the context in their closure. If such tasks are used, the context should not be captured after the scope ends. Additionally, explicit management of scopes is recommended to ensure the context is disposed when the work is done.

The significance of this issue goes beyond just fixing an error. It's about designing applications for parallelism. EF Core's concurrency rules are centered around respecting dependency boundaries. Violating these boundaries, such as letting a DbContext outlive its scope, will lead to similar errors. To avoid these problems, developers should test under load and design for lifetime correctness. By following these principles, they can create more robust and scalable applications.