Urgent.News

What's breaking now, across thousands of outlets.

AI

I gave my local LLM live web search for free, and it replaced the one thing I still used ChatGPT and Gemini for

My GPU is saving my tokens and my dollars

I gave my local LLM live web search for free, and it replaced the one thing I still used ChatGPT and Gemini for

Developing a local AI model that can rival cloud-based LLMs like ChatGPT and Gemini has been a challenge for many users. One significant hurdle is the lack of native web search capabilities in these local models. Typically, users download a model's weights from platforms like Hugging Face and run them using llama-server. However, these models are limited by their training data and usually do not have access to real-time information.

This limitation often forces users to revert to cloud-based LLMs for more up-to-date information. To address this issue, I implemented a solution that allows my local LLM to perform live web searches, thus reducing my dependence on cloud AI services. I use Qwen3.6-35B-A3B, a 35-billion-parameter mixture-of-experts model, which runs on my RTX 4070 Ti Super with 16GB of GDDR6X, a Ryzen 5 7600X, and 32GB of DDR5-6000.

To enable web search, I set up SearXNG, an open-source search engine, as a custom tool within the model. This tool communicates with the model through mcp-searxng, a small Node.js program that acts as a bridge between SearXNG and llama-server. With this setup, the model can independently search the web for up-to-date information when needed.

For instance, when asked about the date of CES 2027, the model recognized the need for current information and performed a live search, arriving at the correct answer. This setup has allowed me to operate my local AI setup more independently and without relying on cloud subscriptions.

Written by urgent.news from XDA Developers's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

This story

This is one outlet's version. Read the fullest account.

Every outlet that reported this, and when →

Read the original at xda-developers.com →

More in AI

More from Wednesday 18 November →