{
  "id": 14302121,
  "title": "Your library isn't in the training data: making AI agents use it well",
  "url": "https://urgent.news/2026/10/13/your-library-isnt-in-the-training-data-making-ai-agents-use-it-well",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-13T22:04:47.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/kevinccbsg/your-library-isnt-in-the-training-data-making-ai-agents-use-it-well-5a34"
  },
  "original_language": "en",
  "account": "Providing an AI agent with knowledge about a specific library or system is crucial to ensure it can use that knowledge correctly. When an agent doesn't have information about your library, it tends to make mistakes by confidently generating code based on similar examples from its training data. If you ask an AI model to create a React component with Jest tests, it might write decent code using examples it has seen before. However, when asked to work with something it has never encountered before, such as your company's internal API client or a new version of a library, the agent will likely produce code that is mostly correct but contains small, confident errors. The AI model doesn't guess randomly; it fills in any gaps with the closest thing it knows.\n\nTo improve the agent's ability to work with your library, you should take several steps. First, provide the model with comprehensive knowledge about your library rather than just rules or warnings. Include signatures, return types, and patterns, along with complete and correct examples. Avoid providing warnings, as they don't prevent mistakes. Instead, use a programmatic check to catch errors before running the code. Load the knowledge layer by layer, with a core set of information that is always available and additional details that are loaded only when needed. This approach minimizes the amount of data the agent needs to process on each request.\n\nSecondly, leverage a typed API to teach the model about your library. When the agent makes mistakes like misspelling an option name or passing arguments in the wrong order, a type check will catch these errors immediately, preventing them from becoming bugs at runtime. Ensure your API is strictly typed with correct option names and clear return types. A compile-time error is more reliable than a runtime bug. Additionally, use a linter to catch issues that the type system can't detect, such as missing `await` statements or imports from alternative libraries. Make the linter easy to use and automatically fix obvious errors. Before relying on the linter, test it against real projects to ensure it doesn't misinterpret valid code.\n\nThirdly, design your output in a way that is understandable by both humans and the AI agent. Provide a JSON report alongside regular output, as AI agents may struggle to parse console logs or other text-based formats. Offer a JSON format option for every command that generates output, and document all exit codes clearly. Transparency in the agent's output will help both users and the AI model interpret the results accurately.\n\nFourthly, test your instructions thoroughly using evaluation cases. Instructions can change over time, and a seemingly small edit can lead to a loss of coverage for a particular aspect of your library. Regularly run evals, which are tests that set up a small project, give the agent a task, and grade the results. Make sure to use deterministic graders, such as regular expressions or explicit tool usage checks, to ensure consistent grading. Plant deliberate errors in test files to verify that the agent's tools, like linters, are correctly identifying and fixing issues. Running evals before releases and after making changes to your instructions will help maintain the quality of the agent's output.\n\nFinally, use evaluation cases to identify gaps in your instructions. These tests can reveal issues that you might have overlooked, ensuring that your instructions cover all necessary aspects of your library. Evaluate your instructions regularly, and update them as needed to maintain the effectiveness of your AI agent. By following these steps, you can significantly improve an AI agent's ability to use your library correctly and efficiently.",
  "summary": "A model that doesn't know your library doesn't fail by doing nothing. It fails by confidently writing code for a library it does know. Ask an AI agent to write a React component with Jest tests and it'll do a decent job. There are millions of examples of both in its training data. Now ask it to use something it has never seen: your company's internal API client; your design system; a niche open…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}