Day 47: Copy Requirements Before the Code, and SQS Has No Priority Feature

Day 47 marked the final Docker task in the Engineer platform's run, as Days 35 to 47 transitioned from installing the engine to deploying a real application image in Kubernetes. The Dockerfile for the application remained unchanged from previous days, with two COPY lines determining the order of operations: first, installing dependencies via `COPY src/requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt`, and then copying the code with `COPY src/ .`. This order ensured that rebuilding the image only occurred when changes were made to the source code. The project's configuration took advantage of AWS's Simple Queue Service (SQS) queues and Simple Notification Service (SNS) topic, allowing for high-priority and low-priority message processing via Lambda functions.

To achieve this, two SQS queues were created, an SNS topic was set up for routing based on message priority, and a Lambda function was designed to drain high-priority messages before low-priority ones. However, during deployment, an IAM permission error arose due to mismatched actions between inline and managed policies. The solution was to migrate all permissions to managed policies, which allowed for more granular control over the necessary actions.

Lessons learned from this experience included the importance of validating CloudFormation templates for syntax rather than operational validity, as well as the potential complexity and costs associated with creating stacks after a failed deployment. The final deployment of the Python application followed the prioritized message processing design using SQS queues and SNS filters, ensuring that high-priority messages were processed before low-priority ones.