Web APIs and Batch Jobs Communicate Failure Differently

Web APIs and batch jobs handle failure in distinct ways. Web APIs convey success or failure to clients via HTTP status codes, such as an HTTP 500 Internal Server Error. However, batch jobs require a different approach.

Traditionally, batch jobs might return false or { status: 500 } to signal an error. Yet, Lambda functions, a common platform for batch jobs, do not consider these return values as failures. Consequently, even if the application coded a failure, the execution environment might still register it as successful. As a result, the invocation won't be logged in CloudWatch's Errors metric, and monitoring systems like alarms won't trigger.

This realization came when attempting to send Slack notifications for failing batch jobs. Initially, the thought was straightforward: monitor Lambda execution errors with CloudWatch and use AWS notification services to alert via Slack. But upon examining the existing batch code, it became clear that the error handling was inadequate. It only signaled the code level, not the execution environment.

To resolve this, the error handling was refactored to throw an exception when an error occurred: throw new Error(...). This action caused the Lambda invocation itself to fail, which was then recognized by CloudWatch's Errors metric. Only then could monitoring and alerting systems observe the failure, leading to an appropriate Slack notification.

This experience underscored the importance of considering error handling not just within the application but also in ensuring that failures are observable by the execution environment and monitoring systems. While Web APIs use HTTP status codes for communication, batch jobs necessitate a failure state that the execution environment can detect. Proper error handling for batch jobs involves both application-level corrections and ensuring the environment and monitoring systems can identify and respond to failures.