Foundation Models Framework (macOS)

The Foundation Models framework is a powerful tool for integrating large language models into applications using Swift. It provides direct access to Apple's Foundation Models on-device and through Private Cloud Compute, enabling features like guided generation, tool calling, image understanding, and Dynamic Profiles. Multimodal prompts allow for the simultaneous processing of both text and images, opening up new possibilities for visual content analysis.

The Vision framework is seamlessly integrated, offering OCR and barcode reading capabilities on-device. This framework gives developers the ability to build intelligent, private, and offline-capable experiences. It extends to server models like Claude and Gemini through the same Swift API, supporting features such as tool calling and guided generation.

Any language model provider can create a Swift package that conforms to the Language Model protocol, allowing for flexibility in selecting the appropriate model for an app. For developers starting out, the Apple Foundation Models in Private Cloud Compute can be used for free under the App Store Small Business Program, ensuring user data remains private and secure.

The new Dynamic Profiles feature enables continuous updates to a session's model, tools, and instructions, allowing for more contextual intelligence with less prompting. The Evaluations framework aids in testing prompts and ensuring features work correctly across various conditions. The fm command line tool in macOS 27 facilitates scripting and automates model workflows, making it easier to get started with AI integration.

Additional utilities packages provide experimental abstractions, such as custom skills and chat completions clients. The Foundation Models SDK for Python extends the framework to the Python ecosystem, enabling integration with popular tooling and evaluation packages. Finally, the Acceptable Use Requirements for the Foundation Models framework outline prohibited uses, and information is provided to AI system providers to comply with the AI Act.