Prompts aren’t enough: Why two AI giants are evolving their approach

This summer, OpenAI and Anthropic announced features that allow their AI systems to learn a task by observing someone do it once, rather than being instructed in a prompt. In June, OpenAI introduced Record & Replay, enabling ChatGPT and Codex users to demonstrate a workflow and turn it into a reusable skill. A few weeks later, Anthropic unveiled Record a Skill within Claude Cowork, where users can record their screen while doing a task, narrate their reasoning, and Claude transforms it into a skill for repeated use.

Two competitors, simultaneously arriving at the same solution, signify that prompting alone may not be sufficient to advance AI. The author, who previously worked on Siri at Apple, believes this milestone was inevitable and that OpenAI and Anthropic have recognized the need. The gap between voice assistants and human interaction lies in retaining memory across interactions, a concept known as tacit knowledge, which is difficult for AI to grasp.

While prompts provide instructions, they cannot capture nuances and decision points inherent to a person's workflow. Demonstrations, however, capture these aspects more accurately, as the context and action are intertwined. A library of recorded skills can run autonomously and save users from repetitive prompts and iterations. As more people adopt demonstrations, workflow mining will become increasingly valuable, transforming how we automate multistep tasks and create a continuously improving library of reusable workflows.