Proving the Tree Came Back After Breaking It on Purpose
Code: Megapixel99/restore-verified Any tool that breaks a file on purpose and puts it back (a mutation harness, a codemod, a benchmark swapping a config) has four ways to leave the tree wrong. The standard try/finally covers exactly one of them. First, an exception mid-run, which is the covered case. Second, a signal: finally does not run on SIGTERM, so a plain kill leaves the file broken. Third,…
The <source> material details a tool called restore-verified, which helps ensure that file mutations during testing are properly restored. The tool provides safeguards against four failure modes:
1. Exceptions occurring mid-run
2. Signals like SIGTERM not running the cleanup code
3. Incorrect restore operations
4. SIGKILL signal that cannot be handled within the process
restore-verified addresses these issues through a context manager in Python and a CLI tool in JavaScript. It hashes the file before and after attempts to restore it, comparing the snapshots to verify the restoration was successful. If the restore fails, the tool detects that the mutated file remains on disk.
The package is designed to work with any command line harness by providing a callback mechanism. It uses a manifest (a JSON file) to record file digests before and after mutations, allowing the tool to verify that the final mutated state matches the expected state after restoration.
The tool is particularly useful for mutation testing, where the goal is to determine if testing is effective by introducing small changes to the code and checking if the tests catch them. However, the author argues that for tools that create copies of the code during mutation (rather than modifying in-place), the need for restore verification is less critical.
The <source> also includes testing of restore-verified against four real mutation testing frameworks, demonstrating that the tool provides essential protection against restored mutations in cases where the frameworks fail to restore their mutated code correctly.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
