Controlled Decoding Attacks on Black-Box LLMs
Manipulating next-token probabilities during generation can bypass the safety alignment of large language models. Existing approaches, however, rely on access to model weights or numerical token probabilities and therefore do not apply to interfaces that return only sampled text. Reconstructing probabilities from sampled outputs offers a possible alternative, but finite sampling produces sparse…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.
