My determinism check can only test one producer in seven, and the print says so
I added a check that asks one question: run the thing that produces this record twice, and do you get the same bytes? Its first output: OK_PHI kinds=1/7 reruns=2 bytes_equal=1/1 declared_agree=1/1 disagree=0 plasma_untestable=1 One of seven. That number is not a coverage gap I plan to close. It is the answer. Why six of them cannot be re-run The check reads who produces each kind of record from a…
A new determination check has been added to assess whether a record producer generates identical bytes when a record is produced twice. The result of the initial check demonstrated that only one out of seven producers can be verified. This figure is not indicative of a coverage gap the researchers plan to address; rather, it is the final result.
The reason behind the inability to run six of the seven producers is that they cannot be re-run. The check identifies the producer of each record kind from a declaration, and of the seven producers, five are programs that can be re-run, while two are devices without a declared interpreter and one cannot be invoked at all.
The key aspect of this check lies in running the record producer twice and comparing the resulting bytes. For records written by a person, this is not a complex task; it simply measures memory, not the system's determinism. Two tempting numbers emerge from this process: kinds=1/1, which skips the six unmeasurable cases, and kinds=1/7, which includes these cases in the denominator.
The first number presents a misleadingly successful outcome, as it only accounts for records that were easy to measure, while the second number accurately reflects the true determinism by considering all seven producers, even those that are unmeasurable. The latter number is crucial, as it keeps the count of unreachable cases in the denominator, thereby providing a more accurate representation of the system's determinism.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
