We Taught a 230M Language Model to Keep Learning on Android
Small language models can now run directly on phones. But most of them stop learning the moment they ship. For personal AI, that feels like a strange stopping point. Some of the most useful signals arrive only after the model acts: Did the user dismiss the notification? Did they open it later? Did they rewrite the suggestion? Did they ask for it again? These interactions contain useful…
We have developed an open-source prototype called Online-SDFT, which fine-tunes a small language model on Android devices using delayed interactions. This allows the model to learn and continuously improve based on real-world usage patterns, rather than being limited to predetermined actions. The prototype utilizes a rank-4 LoRA adapter, ONNX Runtime training, and an on-device replay buffer to store relevant information.
By using soft targets and controlled exploration, Online-SDFT can make better decisions and adapt to the user's preferences over time. Our initial results on synthetic notification streams show that Online-SDFT outperforms other methods, achieving a preference accuracy of 70.3% compared to 28.2% for the frozen base model and 50.0% for RAG.
The replay buffer plays a crucial role in this learning process, as its removal significantly impacts the model's performance.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
