Bold Text Doesn’t Control LLM Attention. So What Does?
A group of researchers wanted a model to pay closer attention to a span a user had marked. They did not solve it by marking the span more heavily.
A team of researchers sought to make an AI model focus on a specific section marked by the user. They did not achieve this by highlighting the span more prominently. Instead, they scrutinized the attention layer, which provided the most direct evidence of what inserting asterisks into an instruction file accomplishes. The study in question is called PASTA, developed by Microsoft Research in collaboration with Georgia Tech.
It is an inference-time method that reweights a small subset of attention heads, effectively directing the model's attention to the user-designated span. The method does not alter any model parameters; it is applied at inference, similar to how prompting functions. The researchers found a 22% average accuracy improvement on the LLAMA-7B model as a result of this approach, as it became evident that emphasis typed into the prompt is a weak signal, not a control.
Written by urgent.news from HackerNoon's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
