I Gave an On-Device LLM a Search Tool. It Ignored It and Made Up Data Instead
I handed Apple's on-device model a Spotlight search tool. It never called it, invented two snippets that do not exist, and twenty answers in a row passed.
A user has reported an issue with an on-device Large Language Model (LLM) that ignored a search tool and made up data instead. The feature, which is a single sentence command in the snippet library, was supposed to provide real snippets based on the search query. However, the model produced plausible names for snippets that didn't exist in the library, and most of the answers were false.
The model did not use the retrieval tool at all, which led to incorrect documentation as the user noted down what the model seemed to be doing. The issue was eventually caught because the cosmetic half of the feature kept failing, and the model's output and the retrieved data could not agree. The main lesson is that the model's output and the retrieved data must agree, and if they don't, it indicates a problem.
The article also discusses the importance of ensuring that the UI will break loudly and mechanically if the model makes something up, rather than focusing on preventing hallucinations.
Written by urgent.news from HackerNoon's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
