Edtech Observability Stack — Health Monitoring Through Logs, Metrics, and Result Deadlines

An edtech app's health can be monitored by focusing on three key data points: structured logs, a limited set of metrics, and explicit job-completion events. While structured logs are useful for troubleshooting, metrics help track trends, and completion events indicate the freshness of data. To monitor success, an uptime probe should be used in addition to internal signals.

The uptime check should verify that the app delivers promised outcomes, such as the import of student records at a specified time. When monitoring import health, focus on the user-visible promise. District files are expected to arrive at a certain time, with a normal import window of 25 minutes, and downstream results should be available within a specified timeframe.

The monitor should check if a scheduled run has reached a terminal state and produced a plausible result before the deadline. Represent every expected run with a stable key combining tenant ID and schedule date. Record key details such as scheduled time, start time, completion time, terminal status, input count, accepted count, rejected count, and a correlation ID.

Counts alone do not prove data correctness, but they distinguish between empty files and worker failures. Be mindful of date-related issues, as time zone discrepancies can cause confusion around import deadlines. Model the promise once by creating expectations from the district's configured time zone and persisting both business dates and absolute deadlines.

The monitor should consist of four main components: a scheduler that creates expectations, the worker that emits structured progress logs and writes completion records, a metrics exporter that aggregates counters and durations, and a separate evaluator that compares current time with each expectation's deadline. Alerts should be generated from the evaluator and probe, while logs serve as supporting evidence.

Implement the result monitor in TypeScript, relying on storage and paging interfaces rather than a specific monitoring vendor. The evaluator uses an injected clock for deterministic boundary tests. Define the import status and import run types, along with alert and run store interfaces. The evaluateImports function checks for due imports, determines their completion status, and opens alerts accordingly.

The fingerprint ensures only one incident per run, and the resolution mechanism closes alerts once a delayed run succeeds. Keep the write path idempotent to prevent duplicate alerts. Additionally, expose aggregate measurements like expected runs, successful runs, failed runs, overdue runs, and completion duration using metrics defined by OpenTelemetry.