Can Jev beat Polymarket? I asked it about 26,174 resolved markets
Can Jev beat the Polymarket price? Not from the question text, no. I sent Jev the question and resolution rules of 26,174 resolved Polymarket markets and asked one thing: will this resolve Yes? On the 9,138 markets resolved after Jev's release, so nothing it could have seen in training, it scored an AUC of 0.615. The Yes price traded a day before scored 0.844. Telling Jev the price made it worse…
The author tested whether Jev, a language model, could outperform Polymarket's prediction market prices on 9,138 resolved markets. After preprocessing the data, Jev was fed the market question and resolution rules. The model was then asked whether the market would resolve as "Yes" or "No." On 2.4% of the markets, Jev predicted a higher probability of "Yes" than the trader's implied price. The model scored an AUC of 0.615, which was worse than the average price of 0.844 observed for the 9,138 markets.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.