A Determinism Check Has to Leave the Process
Code: Megapixel99/nondet The obvious way to check whether a Python function is deterministic is to call it twice and compare. I wrote that check, and it is blind to the commonest source of nondeterminism in the language. String hashing is randomised per interpreter, so set and dict iteration order is stable within a process and different in every new one. A function returning list({'alpha',…
This article explains an open-source Python tool called nondet, designed to detect nondeterminism in Python functions. Determinism refers to a function consistently producing the same output given the same input. The author describes how Python's built-in data structures and randomization mechanisms can lead to nondeterministic behavior even within a single interpreter instance.
To address this, the nondet tool re-runs each function with a fixed input ladder across multiple fresh worker processes, ensuring a deterministic comparison of results. The tool sets PYTHONHASHSEED to zero for the workers, preventing any environment-based seed influence. It covers various function types, including those with multiple positional arguments, variadic arguments, keyword-only signatures, and zero-arity functions, rejecting the latter.
The tool's test case, test_in_process_repetition_would_have_missed_it, ensures the in-process check misses variations in hash-order functions that the fresh process check would catch. The package includes various fixtures to test different function structures and behaviors, revealing instances where hash-order defects and other nondeterminisms occur.
Out of 19 tested functions, all nondeterministic functions were caught, while deterministic ones were not falsely flagged. One deterministic function was intentionally refused as a refusal test. The tool's gated probing finds genuine nondeterminism and costs some recall. The environment's variability, including timezone and locale, is considered, with the epoch_year fixture catching functions that depend on time zone differences.
The article highlights some bugs in the tool itself, such as issues with relative imports and function result handling, which were addressed through real-world testing. The nondet tool is unique in its approach, walking a ladder of inputs in fresh processes to identify differences between runs, a granularity not found in other existing tools.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
