It Ran Every Morning and Still Broke: Failure Modes of a Free-Tier AI Job
The cron log said exit 0 every morning. The summary file updated on schedule. The job was running. The output was wrong. This is an autopsy of a small automation that failed without crashing. It ran on MonkeyCode's free tier: a daily job that fetched upstream release notes, summarized them with a model, and wrote the result to a file. The setup was simple. The failure modes were not. Disclosure:…
The cron log repeatedly reported exit 0, indicating the job was running successfully. However, the output file contained incorrect information. This small automation, which fetched release notes, summarized them with an AI model, and wrote the summary to a file daily, failed without crashing. The job ran on MonkeyCode's free tier, following a simple setup with three files: watch.py for fetching data, latest_summary.md for the output, and ledger.jsonl for logging runs. The failures were not due to the simplicity of the setup.
Failure mode 1: The AI model provided a one-word answer ("Yes") without the required sections. A fix was a schema check to ensure the output contained all three sections.
Failure mode 2: The job received a quota message in a 200 response from the AI model endpoint. The script treated this as the summary, which was incorrect. To fix this, the response body should be validated against an expected shape, not just the status code.
Failure mode 3: The cron environment on the server differed from the local environment, causing the script to fail due to a missing dependency directory. This was fixed by using absolute paths and checking for all dependencies before proceeding.
Failure mode 4: Stale state data was mistakenly treated as fresh data. The script wrote the ledger entry before updating the state file, causing the script to skip work after a successful run. Writing the ledger entry before updating the state file and logging skipped runs as events resolved this issue.
Failure mode 5: Truncation of release notes could lead to important information being missed. The script recorded a warning log when truncation occurred, but it was not enforced. Implementing this check ensures that no critical information is overlooked.
After identifying these failure modes, the author created a health check script (healthcheck.py) to monitor the AI job's health. This script verifies if the job is operating correctly by checking the output size, structure, ledger entry, state file consistency, and truncation warnings. By implementing this health check, the author hopes to prevent future failures in their free-tier AI job.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
