{
  "id": 11297459,
  "title": "Item Review Desk: exam questions that can't go live until a second person signs them off",
  "url": "https://urgent.news/2026/10/01/item-review-desk-exam-questions-that-cant-go-live-until-a-second",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-01T22:16:52.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/dr_ai_578f3b4b94ad92b1cc0/item-review-desk-exam-questions-that-cant-go-live-until-a-second-person-signs-them-off-3fbi"
  },
  "original_language": "en",
  "account": "This report covers a system designed to manage the review process of exam questions for professionals, particularly in customer service and contact centre roles. The reviewer's school in Lagos creates multiple versions of questions, which then undergo a rigorous approval process. Currently, this approval process is chaotic, with questions hopping between email threads and a status spreadsheet. The Item Review Desk aims to streamline this workflow by digitizing it within the Sanity platform.\n\nEach exam item has its own state - approved, in review, awaiting changes, or in draft - along with a detailed log of every action taken. An AI agent can generate questions and request a review, but only a human reviewer can approve them. Importantly, the person who wrote the question cannot approve it themselves. The Next.js website only displays approved items, while a board tracks how close each exam blueprint is to a final version.\n\nThe review process is governed by a set of rules encoded in the lib/workflow.ts file, which defines the allowed transitions and the conditions for each move. For instance, a draft can only be submitted for review by a human, not by the AI agent itself. The reviewer must also have made at least one correction before approving. The system prevents an approved item from being submitted again for review.\n\nItem writing rules are enforced by lib/lint.ts, which checks for issues like unclear stems, improper capitalization in negatives, excessive length of options, and the use of 'all of the above'. These checks are displayed directly in the form, and errors prevent the item from being submitted.\n\nHowever, the system initially faced some issues. The AI agent couldn't access Sanity due to its restricted environment, leading to a failure in building the site. To address this, an OFFLINE_SEED mode was introduced, allowing the system to run the queries against a pre-defined dataset, enabling a preview of the site without an actual Sanity connection. Another issue was the overly strict rule about lowercase 'not' in stems, which didn't account for context. This was refined to only check the sentence in question, improving the accuracy of the checks. Additionally, the warning about the longest answer being disproportionately longer than the distractors caused problems for the agent's items, which were subsequently adjusted.\n\nDuring the live demonstration, the AI agent couldn't interact with the real Sanity project. Manual intervention was required to approve certain questions, but even then, the practice site only showed eight questions instead of the expected nine. This was due to the system attempting to publish changes in a single operation, which conflicted with the Studio's loading process. The fix involved writing the transition directly to the published document in one transaction, thereby resolving the issue.\n\nLooking forward, the reviewer suggests integrating the agent into Sanity Functions, enabling automated item drafting when an objective falls below its target. Additionally, a second reviewer step could be implemented for high-stakes exams to further enhance the review process. The Sanity project is identified by the ID x97j.",
  "summary": "This is a submission for the Sanity Challenge, Path Two: Vibe-Code Something Strange What I Built I run a training school in Lagos and have spent years writing exam questions and certification prep for working professionals, a lot of it for customer service and contact centre roles. The part of that job nobody sees is item review. A question gets drafted, a subject expert checks it, it goes back…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}