{
  "id": 12549624,
  "title": "Write the Acceptance Tests Before the Agency Writes the Automation",
  "url": "https://urgent.news/2026/10/07/write-the-acceptance-tests-before-the-agency-writes-the-automation",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-07T05:02:45.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/ujjwal_dubey_9/write-the-acceptance-tests-before-the-agency-writes-the-automation-4pi8"
  },
  "original_language": "en",
  "account": "Before starting a project with an AI automation agency, it is recommended to write the acceptance tests yourself. This shifts the conversation away from vague terms like \"smart routing\" and towards concrete, behavior-driven tests. Acceptance tests describe how the system should respond to specific real-world inputs, rather than just outlining features. For AI workflows, this is crucial as the same feature can behave differently depending on the type of input, such as a voice message, a duplicate lead, or an upset customer.\n\nThe recommended approach is to create acceptance tests in plain language and attach them to the contract. Each test should have a unique identifier, a given condition (the input), an expected outcome (the desired behavior), and a flag indicating if human approval is required. For example, an acceptance test for a WhatsApp message asking about opening hours might specify that the system should auto-reply from an approved library within the business's reply window, create a lead in the CRM, and not require human approval.\n\nWhen writing acceptance tests, it's important to use real messages as input, remove any personal data, and aim for a balance between happy-path tests (positive scenarios) and failure tests (error handling, duplicates, opt-outs, etc.). Every test that involves a human should be marked with human_approval set to true. For tests involving money, prices, or apologies, human approval must also be marked as true.\n\nIt's also crucial to agree on who will run the tests. Typically, the agency will run them before the system goes live, and the client will rerun a sample after any changes are made. All acceptance tests should be kept in a shared location, such as the repository or shared drive, next to the workflow map for easy access and reference.\n\nBy following this approach, proposals become more concise and honest. Vague terms are replaced with specific acceptance tests, making it easier to discuss effort and pricing. The initial step in this process is to conduct a 72-hour audit of one workflow, during which these acceptance tests are created. If you're still unsure whether custom AI work is necessary, comparing custom AI solutions to off-the-shelf chatbots can help make an informed decision.",
  "summary": "If you are the technical person asked to \"work with the agency\", here is the one thing I would do before kickoff: write the acceptance tests yourself, in plain language, and attach them to the contract. It changes the conversation with an AI automation agency that writes acceptance tests first or with any vendor, because \"done\" stops being a feeling. Disclosure: I run NxFlowAI. We ask clients for…",
  "key_points": [
    "Write acceptance tests before agency writes automation.",
    "Tests describe system response to specific real-world inputs.",
    "Include unique identifier, condition, outcome, and human approval flag."
  ],
  "editors_take": "Writing acceptance tests upfront redefines client-agency collaboration by shifting discussions from vague feature terms to specific, behavior-driven requirements, enabling more transparent proposals and effort estimates.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}