Urgent.News

What's breaking now, across thousands of outlets.

AI

An uncalibrated classifier is worse than no classifier

What I learned building a website that rewrites itself for whoever is reading it, on ₹0 of infrastructure. Project Blog : https://santhosh-reddy.vercel.app/en/blog/8 Project Breakdown : https://santhosh-reddy.vercel.app/en/project/8 My portfolio gets very different readers. A recruiter clicking through from a LinkedIn job post wants to know whether I'm hireable for an internship. A developer…

A recent project demonstrated that an uncalibrated classifier performs worse than having no classifier at all. The project, called Evolve, used a free, Jev-style typed-decision API to route visitors with varying confidence levels. In some cases, this resulted in conversion rates lower than those achieved with no segmentation at all.

The key issue was that a poorly calibrated classifier split traffic into populations that learned from the wrong influences, leading to slower convergence and wrong outcomes. The solution was calibration rather than improving raw accuracy. To create the classifier, the project used a seeded generative model trained on honest targets, and then fine-tuned a MiniLM-L6 student model on Oracle's free micro VM.

The resulting model was able to sustain about 8 fresh decisions per second, with limited RAM and memory requirements, while achieving good performance on unseen referrer hosts.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in AI

School Wellbeing Gold Coast: What Ethical AI Tools Should and Shouldn’t Do in Schools

Artificial intelligence is finding its way into almost every part of education. Teachers are experimenting with AI for lesson planning, students are using conversational tools for study support, and…

  • AI should automate routine tasks for teachers, freeing time for human-focused work
  • AI can provide general resources for student wellbeing without assuming emotional states
  • Privacy is crucial; local processing limits data sharing and maintains student confidentiality

I Surveyed 123 People in India to Benchmark Frontier AI

This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked Current evaluations score language models using Western multiple-choice questions.

  • 123 university students in Goa surveyed to benchmark frontier AI models
  • Four key sectors evaluated: education, healthcare, justice, finance
  • Frontier and open-weight models tested using Kaggle Benchmarks SDK

More from Friday 2 October →