Urgent.News

What's breaking now, across hundreds of outlets.

Tech

A Japanese NLI checker for the space between generation and trust (sealed 0.961, zero-shot baselines 0.79–0.81)

TL;DR — We released ja-nli-v1, a small Japanese natural language inference model: given a premise and a hypothesis, it returns probabilities for entailment / neutral / contradiction. On a sealed test of 5,000 pairs it scores accuracy and Macro F1 of 0.9612; on a 600-pair human-verified set, 0.9567, with entailment↔contradiction confusions at 3 (0.5%). Two multilingual NLI models used zero-shot…

We released ja-nli-v1, a small Japanese natural language inference (NLI) model that assesses relationships between a premise and hypothesis. On a sealed test of 5,000 pairs, the model achieved an accuracy of 0.9612 and a Macro F1 score of 0.9612. This outperformed zero-shot baselines of Japanese models, which scored between 0.79 and 0.81 on the same test set. In a human-verified set of 600 pairs, the model achieved 0.9567 accuracy with entailment/contradiction confusion at 3 (0.5%).

The model relies on synthetic data built from public-domain licensed government documents and Japanese OSS documentation under MIT/Apache-2.0 licenses. It was trained without using existing NLI datasets. The model is licensed under Apache 2.0 and available for use via Transformers. It achieves ~109 ms per pair on a CPU.

The model includes diagnostics to ensure it is actually reading the premise, such as accuracy dropping to ~0.38 when the premise is shuffled to a completely unrelated text. It also performs well on a hypothesis-only probe, scoring ~0.63 on the human-verified set. Long premises are truncated after 1024 tokens, so for longer texts, the premise should be chunked. The model was not trained on other Japanese NLI benchmarks, so performance on those should be considered separately.

Rewritten by urgent.news; may contain errors — check the original.

Read the original at dev.to →

Report a problem with this story

More in Tech

Why Small Teams Need CI/CD (and How to Set Up Your First Pipeline)

It is a common myth among small engineering teams that CI/CD pipelines are an enterprise luxury. When you only have two or three developers, setting up automated infrastructure can feel like…

  • CI/CD automates testing and deployment, reducing manual errors
  • Small teams can set up CI/CD with simple steps, not complex systems
  • Automated pipelines catch bugs early, freeing developers for feature work

Why Your Show HN Got No Upvotes (and What to Change Next Time)

A Show HN stalls for one of four reasons: readers can't try it, the title sounds like an ad, the body has no proof, or the maker isn't in the thread.

  • Readers can't try the product, leading to failed Show HN
  • Title seems like an advertisement, causing it to be skipped
  • Engage with the thread during the first hour to address concerns

Change a screenshot to the required aspect ratio in the tab before you commit the hero

The docs call for a 16:9 hero, or the OG template is 1200×630. The file in the PR is a 4:3 laptop screenshot. The layout letterboxes it, or a later object-fit: cover chops off the toolbar you meant to…

  • Hero image must be 16:9 aspect ratio, 1200×630 pixels
  • Layout team treats aspect ratio as initial check, not CSS compensation
  • Use in-tab ratio tool to adjust image before committing

Same Character, 25 Clips: How ULTRA Was Made

Originally published on the Character Lock blog . ULTRA is a 3-minute AI brand film (3:13) with one bear dad holding across more than 25 clips.

  • ULTRA is a 3-minute AI brand film with 25 clips of a bear dad.
  • Character Lock method ensures consistent character appearance across 25 clips.
  • Film achieved LUFS master of minus 14, demonstrating Character Lock method effectiveness.

How I Automated 10M+ Network KPI Records a Day with Python and Apache Airflow

Originally published on pmpdilshan.github.io . Every hour, a mobile network generates performance counters for every cell: traffic, drops, handovers, throughput, congestion.

  • Mobitel generates over 10 million network KPI records daily
  • Automated pipeline using Python and Apache Airflow reduces effort by 40%
  • Automation frees up time for optimization work and KPI centralization

More from Saturday 21 November →