Inside Android Skills - Built for deprecation

Android Skills, introduced in April, have been an overwhelming success according to initial feedback. This blog post aims to clarify the reasoning and process behind these skills, as well as address concerns about their limited availability. The primary criterion for creating new official skills is the identification of knowledge gaps in state-of-the-art (SOTA) models; otherwise, teaching models what they already know is unnecessary.

So far, around 20 official skills have been released, targeting highly specific and rapidly evolving areas such as AGP 9, Navigation 3, advanced Camera APIs, and Perfetto SQL.

Core, more general skills are not typically considered for official release due to their potential to consume excessive tokens, which can be both wasteful and costly. Before installing a skill, it's essential to evaluate whether the LLM already possesses the required knowledge. Each skill undergoes rigorous testing through evals, which function similarly to integration tests for code. These evaluations ensure that the skill provides clear value when active and fail otherwise.

Developers are encouraged to utilize the Android Knowledge Base, either through Android Studio or Android CLI, for accessing official documentation. This approach is more efficient than relying on numerous skills. If an LLM exhibits overconfidence, prompting it to consult the documentation more frequently can be achieved by adding a directive to its configuration file or prompting the agent directly.

Pull requests are disabled due to the reliance on internal infrastructure for evaluation framework, but community feedback is actively monitored, and users can submit issues regarding bugs, optimizations, or new skill requests.

Core or basic skills may be beneficial in certain scenarios, such as when employing vague prompts, utilizing smaller, more affordable models, refactoring or reviewing legacy code, or deviating from standard Android architecture. However, these skills should be sourced from reputable developers, as unverified repositories may contain untested or even malicious instructions.