Why Your Python Cache Does the Same Work Twice (P.S: GIL Won’t Save You)
Your Python cache can run the same work twice, even with the GIL. See the check-then-act race, the lru_cache gotcha, and how locks fix it.
The source material describes a Python caching issue that does not achieve the intended caching behavior. Despite adding a cache to an expensive function, running it multiple times resulted in approximately 400 executions for 50 keys instead of the expected single execution per key. The cache function was working correctly in terms of returning the right values, but it failed to fulfill the primary purpose of reducing the number of expensive computations.
The article concludes that the problem lies not in shared data but in the gap between two lines of code: the cache check and the cache population. The explanation provided includes an example of using eight threads to call a function with shared keys. The threads run simultaneously, and upon encountering the same key, both attempt to execute the expensive function, leading to redundant computations and the observed high number of function calls.
Another example given involves five threads attempting to withdraw 100 units from a balance of 100. The use of a threading barrier and a withdraw function shows how the race condition manifests in a banking scenario. Despite all threads passing the check that balance is sufficient before any one of them performs the actual withdrawal, the balance ends at -400, showing that the intended logic was not enforced due to the race condition.
To resolve this issue, two fixes are proposed. The first involves using a single lock that encompasses both the cache check and the cache population, ensuring that these operations are atomic and executed one at a time. The second approach suggests employing a separate lock per key, allowing threads requesting different keys to proceed without waiting, while threads requesting the same key will synchronize their access via their respective locks.
In summary, the Python cache bug discussed in the source stems from the logic gap between checking if a key is in the cache and populating it if not. This gap allows for redundant computations, highlighting that the GIL (Global Interpreter Lock) does not offer protection at the level of multi-step logic. The article concludes by detailing two fixes to ensure that the cache actually works as intended, thereby preventing unnecessary computation and ensuring data integrity.
Written by urgent.news from HackerNoon's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
