Prefix Sliding for efficient test-time scaling
Test-time scaling uses extra test-time compute to improve performance, such as letting language models reason longer when solving a problem. As models keep the entire reasoning trace in memory via full attention, hard tasks that need long thinking can be prohibitively expensive. However, we find most intermediate reasoning tokens lose importance as the model continues reasoning. This calls into…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
