Kolibri Has Landed: A Sovereign Open-Weight Model

On the day of German reunification, Kolibri, an English-German Mixture-of-Experts Transformer, was released. The model boasts 78B total parameters, with 3B active, and can handle context lengths of up to 1M tokens. The full weights are available on Hugging Face and can be used under the Apache 2.0 license terms. Kolibri emerged from a series of model training iterations, starting with Kolibri Origin, a 30B total, 3B active model with a 65k token context window.

Both models went through the same pipeline, from data ingestion and curation to ablations, pre-training, post-training, and final evaluations. Kolibri's design prioritizes performance in specific use cases, such as public administration, industrials, aerospace, German reasoning, math, and agentic behavior. To ensure sovereignty, the model's supply-chain integrity is maintained, with transparency in every decision made during development.

Its small and efficient size allows for on-premise deployment, enabling customers to monitor economic impact and ROI. Kolibri strikes a balance between model capability and deployment costs, sitting on the Pareto frontier for quality versus serving cost, for both English and German. The model's bilingual nature stems from a custom German/English tokenizer and a focus on incorporating organic German data during pre-training.

Kolibri incorporates abstention data and the Merlin-Arthur protocol to ensure it acknowledges when it doesn't have an answer, enhancing its trustworthiness. Developed under the EU AI Act, the General-Purpose AI Code of Practice, and GDPR, the model is transparent about its training data and decisions, with reasoning traces providing explainability.

The entire pipeline was built, trained, optimized, and controlled within Germany, ensuring full sovereignty for both Kolibri and its customers.