Google’s Guided Vision turns Gemini Live into a real-world visual guide

Google has introduced a new feature called Guided Vision as part of its Gemini Live update for Android devices. Guided Vision allows the AI to provide descriptions of objects and information in the real world, making it a useful tool for reading fine print, ordering from a menu, or locating household items. This enhancement builds upon the existing capabilities of Gemini Live, which gained new functionality last year with the addition of video and screen-sharing features.

These features enabled the AI to access a user's phone camera feed and screen, allowing them to ask questions about physical objects or screen content. The latest update, the September Android Drop, includes several additional improvements, such as Google Keep integration within Messages and the ability to remember items in the Find Hub.