Jarvis Control Center: Local Browser Agent Powered by Gemma 4

The Jarvis Control Center is a locally-run browser agent that utilizes the Gemma 4 model to understand and execute natural language commands for various tasks. Users can ask Jarvis to open websites, search the web, play YouTube videos, manage tasks, and interact with connected tools.

For instance, a user could say "Open YouTube and play 11K by Seedhemaut," and Jarvis would handle the entire process. It would open the browser, search for the specified YouTube video, and play the result without requiring the user to perform each step manually.

Developed using Next.js and TypeScript, Gemma 4 E2B running locally through Ollama, Playwright for browser automation, Convex for database, authentication, real-time synchronization, and server-side functions, ElevenLabs as a text-to-speech provider, Browser speech APIs for voice input, and Composio for connecting external tools and services, Jarvis is designed to work locally by default.

This local-first approach prioritizes privacy and control, as users can run Gemma through Ollama without sending their conversations to a hosted AI API. Cloud services, such as voice output or external integrations, can be enabled through environment variables when needed.

Open innovation played a crucial role in creating Jarvis, allowing users to run and customize the AI assistant on their own machines. By using an open-weight model through Ollama, the core assistant can operate locally, providing users with more privacy and control compared to a closed-hosted API.

The project showcases the potential of combining multiple open technologies instead of relying on a single provider. While GitHub Copilot helped accelerate the initial implementation, the Jarvis Control Center is a testament to the power of open-source tools and the benefits of local-first AI agents.