Urgent.News

What's breaking now, across thousands of outlets.

Tech

We lost our own benchmark the first time. Web search vs. one API call on 25 companies

If you build an agent that onboards suppliers or customers, sooner or later it has to check that a company is still registered, and that it isn't sanctioned or insolvent. We make an API that checks this against official registers, so we wanted to know if an agent needs it at all, or if a decent model with web search gets there by itself. So we ran a small benchmark, and the first run went badly…

In a benchmark test comparing web search with an API call to verify company registration status, a single API call performed better than web search for 25 companies from 10 countries. The web search method correctly identified 23 out of 25 companies, while the API method got 17 correct. The test revealed four bugs in the API method that needed to be fixed before running a second test.

After fixing the bugs, the API method improved its score to 19 correct out of 25 companies, while the web search method remained at 20 correct. The API method's accuracy in identifying sanctioned companies and insolvent companies was also higher than the web search method. However, the web search method did better on the 12 harder cases, correctly identifying 6 out of 12 companies, compared to the API method's 11 correct answers.

The web search method also performed better on the 13 more well-known companies, correctly identifying 13 out of 13 companies, while the API method only got 9 correct. Despite these differences, the overall score for the API method was higher due to the fixed bugs. The benchmark highlights the importance of addressing bugs in automated systems and the potential benefits of combining different methods for more accurate results.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

Logistics Report Retention: Browser-to-Object Storage with Presigned Uploads

Short answer: for a logistics report that must survive an ordinary customer connection, use a presigned PUT for a small object and a presigned multipart workflow for a large one; use POST when a form…

  • Employ presigned PUT for small objects, multipart workflow for larger ones
  • Maintain upload privacy and accessibility until authenticated and validated
  • Browser transmits bytes directly to object storage, application manages report state

Rust: My secret weapon for code reuse in desktops, mobile, and web

My own independent alternative to Google Docs reuses Tunc WebSocket communications to deliver co-editing features in native Hub and the browser.

  • Rust enables code reuse across desktop, mobile, and web platforms
  • Tunc uses Rust for real-time co-editing in its Hub interface and web version
  • WebAssembly allows Rust logic to function in web browsers seamlessly

More from Sunday 15 November →