Your models agreed with each other. They were agreeing with themselves.
There is a small art project in our house that encodes a sentence as nothing but its word lengths. Each word becomes a run of some symbol, repeated once per letter; the symbol itself is chosen at random and carries nothing. "The night is long" becomes four clusters of length 3, 5, 2, That is the entire channel. A reader — human or model — gets the lengths and nothing else. The project's README…
This experiment involved encoding sentences based solely on their word lengths. The sentence "The night is long" is represented by four clusters of lengths 3, 5, 2, and 3. The purpose of this project is to demonstrate that language models (LLMs) do not recover the intended meaning from structure and bias, instead generating projections. The researchers wanted to check if independent readings of the same encoded message would agree with each other.
They conducted two experiments, one with a known sentence and another with a different sentence. Both experiments used three arms: a treatment arm with readings of the true message, a prior control arm with readings of a different length sequence, and a random basis arm with no model at all. The agreement among the readings was measured by comparing them against random chance.
In the first experiment, the readings agreed with each other more than chance would predict, suggesting that something was getting through. However, in the second experiment, the agreement reversed. This reversal is the most useful aspect of the experiment, as it highlights two different ways a baseline can deceive you when measuring whether models agree.
The first way a baseline can lie is by inflating the agreement floor, making it rise to meet you. In this case, the random texts built from the control arm's readings were near-duplicates of each other, inflating the agreement metrics and the standard error. This led to a false conclusion that there was no signal, when in fact there was a real effect.
To avoid this issue, the researchers drew the alphabet from a request independent of any message, ensuring the control arm's vocabulary could not inherit the convergence it was meant to measure.
The second way a baseline can lie is by measuring the wrong competitor. In the second experiment, the prior control arm showed higher agreement than the treatment arm, which was supposed to be the true message. This was due to the prior control arm reading a different message, resulting in higher agreement. The researchers noted that if they had only compared the treatment arm against the random basis arm, they would have concluded that independent readers converged far above chance on the encoded message, but this conclusion would have been misleading.
They emphasized the importance of using multiple arms and measuring agreement against the correct competitor to avoid misinterpretation.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
