LFM2.5-Encoders for Fast Long-Context Inference on CPU

Hugging Face has introduced two new encoder models, LFM2.5-Encoder-230M and LFM2.5-Encoder-350M, that maintain high performance even with longer inputs. These models are now available for CPU-only inference, enabling tasks such as document-scale jobs, intent routing, policy linting, PII detection and text classification. They can be run cheaply and continuously on a laptop CPU, making them an attractive choice for long input processing.

The models are pre-trained using a masked-language objective and can be fine-tuned for various classification, token-level and search tasks. The LFM2.5-Encoder-350M model ranks fourth among 14 models evaluated on 17 tasks, outperforming ModernBERT-base and most EuroBERT models despite being smaller than most of them. The encoders inherit the fast inference speed of the LFM2 backbone and show their biggest advantage on CPU, being up to 3.7 times faster than ModernBERT-base for inputs up to 8,192 tokens.