Yandex open-sourced an 80B model trained from scratch: what's inside and where it wins
Yandex just open-sourced a language model it trained entirely from scratch — no borrowed weights, no initialization from Qwen or Llama. It's called AliceAI-Foundation-80B-A3B-Base , it's on Hugging Face under Apache 2.0, and it's a surprisingly interesting release if you care about MoE architecture or non-English models. Here's what's inside, where it actually wins, and where the benchmark table…
Yandex has released an 80 billion parameter language model called AliceAI-Foundation-80B-A3B-Base, trained entirely from scratch and released under Apache 2.0. This model uses a MoE architecture with 512 experts and context size of 262,144 tokens, offering superior performance in Russian language tasks, math, code, and long context tasks compared to its predecessor and similar-sized models.
However, the full 80B model requires 160GB of memory in bf16, making it challenging to run locally. It is intended as a base model for further instruction tuning, and lacks chatbot capabilities. The open-source release provides a valuable foundation for Russian-language applications, particularly in legal and educational domains.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
