About 1,800 tests passed. Running the real app found 14 bugs

About 1,800 tests were passed for the Flows plugin. Running the real app revealed 14 bugs that the test suite did not catch. The Flows plugin includes an automation engine for managing collections and records. It features a visual designer, four trigger types, operations for creating or updating records, HTTP requests, email, conditions, and calling other flows, draft/publish versioning, and a step-through debugger. The test suite achieved 80% MSI (Mutation Score Index) per module.

In September, the reporter switched to testing Flows in a separate Laravel app, using it as a user would. This method exposed 14 defects in one release (v1.8.0) and one more the day before (v1.7.1). The existing test suite failed to uncover these issues. Most of the bugs exhibited similar patterns: the system reported success while performing no action or executing the wrong action.

One bug showed a flow completing in 0 milliseconds. The run page displayed "Completed," but no steps had executed. The engine walks the flow as a graph and attempts to find successors on the success branch for each node. However, the lookup used the sourceHandle field exactly. If the graph was written by hand or through the REST API, it might not contain the handle, causing the walk to stop at the trigger and record the completion as success.

Another issue involved accepting unauthorized webhooks. By default, Flows use HMAC authentication mode, where the sender signs a timestamp and body with a shared secret. A problem arose because the webhook_auth_mode column defaulted to hmac, and the webhook_secret was only generated when the trigger node's config contained auth_mode: hmac.

If the secret was NULL, the verifier accepted any signature, effectively allowing anyone to pass the check. The fix involved refusing to verify against an empty secret and reading the webhook_auth_mode column instead of node config.

Another defect involved cache persistence. Flows used a Cache::forever() call to store collection-event subscriptions. However, the optimize:clear command clears the cache, causing all collection-event flows to stop firing. The fix modified the cache to be a read-through index over what is actually published, ensuring that the subscriptions were rebuilt if the cache was lost.