{
  "id": 8223503,
  "title": "OpenJev",
  "url": "https://urgent.news/2026/09/18/openjev",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-18T09:42:22.000Z",
  "source": {
    "name": "Hacker News",
    "slug": "hacker-news",
    "url": "https://openjev.com/"
  },
  "original_language": "en",
  "account": "OpenJev presents two methods for reading probabilities from a model's allowed options. One approach involves decoding probabilities without directly reading them, while the other method has the model write its option probabilities as JSON text token by token. To compare these methods, select a model size and run both on your own GPU, then measure the difference in performance. MiniCPM5 2B is the default model, but for smaller devices like phones, Qwen3 0.6B can be selected instead. Note that loading larger models may be slower and might not fit on low-end devices. The weights for these models are sourced from Hugging Face and remain in your browser's cache, ensuring that inputs never leave the page. First loading can take several minutes depending on the chosen model, network, and GPU. Despite both methods receiving the same decision, one directly reads option probabilities, while the other asks the model to generate them as JSON text. To ensure accurate comparison, read the model's choice logits and normalize across the displayed options only. Ask the model to estimate the same displayed-option distribution and write it as JSON. Observe every token as it arrives, running the methods sequentially on the same loaded model to avoid contention for one GPU. Direct runs occur first, followed by generation. Conditional probabilities are determined by a softmax over only the displayed option tokens, which are not calibrated confidence scores and do not encompass every answer the model might prefer. There are local model tiers available, with the phone model prioritizing size over accuracy and the desktop default being MiniCPM. The 4B option requires significant memory. It's important to note that none of these local models claim to match Jev. Real local timing is provided by timing setup, warmup, prompt preparation, direct execution, the first generated token, and generation completion using performance.now(). No canned results are used in this comparison. The demo utilizes quantized weights through GGUF builds by wllama. Keep in mind that quantization may affect both the quality and speed of the model.",
  "summary": null,
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}