JustFit: 200K-Token LLM Serving on a 24 GiB Laptop with Just-in-Time State Management
Capable open-weight models make local coding and reasoning attractive, but their context and execution state strain laptop memory. We present JustFit, an MLX-based inference runtime that combines KVExec for compressed KV execution, PhaseSwap for component residency, and StateTrans for state-preserving serving transitions. These mechanisms fuse reconstruction and coordinate just-in-time…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
