Your CI is not flaky. It fails every 7 days.
Quick take Before you label a failing test flaky, write down the dates it failed. Flaky has no rhythm. Yours might. "Flaky" is the most expensive word in CI. It closes the investigation. Nobody looks at a flaky test again until it fails on the day of a release. Here is a cheap way to earn the label instead of assuming it. Pull the dates of the last failures and look at the gaps: gh run list…
Your continuous integration (CI) pipeline may not be flaky, but it may fail every seven days. Before labeling a test as flaky, it's essential to document the failure dates. A flaky test can be discouraging, often leading to disinterest in further investigation until it fails during a release. To avoid this, examine the gaps between failures instead of assuming the test is flaky.
To do this, run the command `gh run list --workflow = ci.yml --status = failure --limit 50 --json createdAt --jq .[].createdAt | sort` to list the last 50 failures' creation dates. If the gaps between failures consistently occur on the same number, it suggests a timer-triggered issue, not the test itself. A seven-day cycle is particularly worth noting, as GitHub Actions evicts cache entries untouched for a week.
Dependencies that are only restored, not rebuilt, may disappear on a schedule, causing a slow, cold, and sometimes failing test after that.
This pattern could be due to certificates, tokens, log rotations, or nightly database refreshes, all of which follow a calendar, unlike flaky tests. In one case, the CI pipeline's cache expired every seven days, causing tests to fail due to marginal timeout issues. The fix wasn't to extend the timeout but to ensure the cache was refreshed more frequently. Your CI pipeline might not be flaky; it might just follow a seven-day calendar.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
