{
  "id": 12188100,
  "title": "iphone-use: let an AI agent drive a real iPhone (open source)",
  "url": "https://urgent.news/2026/10/05/iphone-use-let-an-ai-agent-drive-a-real-iphone-open-source",
  "topic": "ai",
  "section": "AI",
  "published": "2026-10-05T16:32:06.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/leeguoo/iphone-use-let-an-ai-agent-drive-a-real-iphone-open-source-4jgb"
  },
  "original_language": "en",
  "account": "Many apps lack official Application Programming Interfaces (APIs), making it difficult for AI agents to interact with them. The iphone-use project bridges this gap by connecting a real iPhone to an AI agent. Using Apple's WebDriverAgent and exposing an HTTP API, the project allows agents like Claude Code to control the iPhone via remote commands.\n\nThe daemon runs on a connected Mac, communicating with the iPhone without interfering with its screen or cursor. An agent performs actions step-by-step: reading the screen, tapping elements, and observing the results. Each action reports whether it succeeded, failed, or is unknown, including specific reasons like covered by a fixed header or not drawn.\n\niPhone-use handles scenarios where an app's protected content or a white screen is displayed. It still captures the accessibility tree, labeling it for reference. Remote control through a web interface or iOS app is available, displaying the same wireframe instead of a blank screen.\n\nPerformance metrics are tracked for each action, enabling optimization. On an iPhone 17 Pro Max, the system reduced tap-and-settle time from 8.5 seconds to 4.2 seconds, and cold connections from 75 seconds to 13 seconds. The project minimizes token usage by sending screen text instead of full screenshots, returning only changed data, and batching commands when possible.\n\nTo use iphone-use, install macOS 15 or later, a free Xcode Personal Team, and an iPhone with Developer Mode enabled. After installing the provided script, access the iPhone remotely via http://mac-ip:44321/phone. The project welcomes contributions and issues through its GitHub repository.",
  "summary": "Lots of apps have no API: banking, payment, health, chat. However good an AI agent is, it couldn't operate them for you. iphone-use hands a real iPhone to an AI agent: it reads the screen as text, taps, swipes and types, and says plainly when an action didn't land. Open source, MIT. GitHub: https://github.com/leeguooooo/iphone-use 2-minute demo: https://youtu.be/k5MsBZcjUVs How it fits together A…",
  "key_points": [
    "iphone-use connects real iPhone to AI agents via HTTP API",
    "Daemon runs on Mac, controls iPhone without interfering screen",
    "Performance metrics track each action for optimization"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}