Stop describing UI to your AI agent in words. Point at it.
I asked my agent to move a price tag 8 px down. It edited the wrong <div> : the one in the desktop layout, three components away from the one I was looking at. I rebuilt, looked, described it again — "no, the small grey one under the title, in the card, on mobile" — and lost another five minutes. Coding agents can read your whole codebase. They still can't see which pixel you mean. So I built…
In a recent development, a new tool called layout-debug-mcp has been introduced to address the challenges of describing user interface (UI) elements to AI agents using text. This tool aims to eliminate the need for agents to infer the exact pixel locations and details of UI elements, which often leads to errors and inefficiencies in the process.
The tool works by providing a local window over the running UI, allowing users to click and drag the desired element directly on the screen. Along with the element's file and line information, box measurements, parent nodes, and changes, the tool sends this data to the AI agent. This approach ensures that the agent has precise information about the targeted element, eliminating the guesswork associated with textual descriptions.
To use layout-debug-mcp, users need to install the tool using the command `{ "command": "npx", "args": [ "-y", "layout-debug-mcp" ] }`. Once installed, users can open the layout-debug window by executing the command `Open the layout-debug window and listen for my edits`. The agent then initiates a local server on `127.0.0.1`, opening a window in the user's browser with the dev page inside (or a bundled demo page until a custom URL is set).
The tool works by allowing users to select and move elements using their mouse and keyboard. By holding the Alt key and clicking, users can select the tightest box under the cursor, which is then highlighted with breadcrumbs indicating the parents and children of the selected element. Users can then drag the selected element or use the arrow keys to nudge it, rescaling it if necessary.
When users input their desired changes in the element's chat, the tool sends this information to the agent in the form of a message, including the element's parent chain, measurements, and anchors. The agent can then make the necessary edits to the UI based on this precise data. The agent's responses are displayed in the same window, allowing users to see the changes made and continue working with hot reloads and real-time updates.
layout-debug-mcp supports both web and Android platforms, providing a normalized snapshot of nodes, boxes, and platform properties to ensure compatibility across different environments. The tool is designed to work seamlessly with various AI agents, including Claude Code, Cursor, Codex CLI, VS Code Copilot, Gemini CLI, Claude Desktop, and Zed, without requiring any additional setup or permissions.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.