Urgent.News

What's breaking now, across thousands of outlets.

AI

Dharma Field Extractor: Any Document In, Structured Fields in JSON Out

A groundbreaking 2-billion-parameter model named Dharma Field Extractor is capable of extracting precisely the fields you require from any type of document, surpassing every frontier LLM tested while being much more cost-effective. In April, DharmaOCR-LITE was introduced, an OCR-specialized model adept at handling various document types, including contracts, court filings, invoices, IDs, and forms.

The model's paper revealed that a smaller, domain-specific model, trained closer to its task, outperformed general-purpose frontier APIs at that task. OCR provides the text on a page, adhering to its original formatting, which is ideal for reading an entire contract or digitizing a manuscript. However, there exists a distinct use case where one desires specific fields from a document, presented in a structure that downstream systems can readily consume.

For instance, listing every line item on an invoice, the patient's name, exam type, and date from a medical requisition, or transforming an ID into columns for an employee record. Pass the document through OCR, and you obtain every printed word on it, without any indication of which string corresponds to the exam type, date, or total.

A general-purpose LLM can accomplish this using targeted prompting, but it becomes prohibitively expensive at a frontier API's pricing and latency for a task many businesses need to perform at high volume. Thus, we developed a specialized model for this purpose instead. The Field Extractor accepts a document image and either a request for all fields found within the document or a predefined schema to fill, returning structured JSON data: typed, field by field, nested according to the document's structure.

An invoice will return all line items as an ordered list, each with its quantity and value, alongside the total. An ID will return the fields your system already uses for storing them. The same checkpoint caters to all cases without fine-tuning for each type. When you need to push specific data into your system, such as supplier names, invoice numbers, due dates, and totals from supplier invoices, medical requisitions for a scheduler, or parties, terms, penalties, and fine amounts from contracts, the Field Extractor allows you to focus only on the essential fields.

You can request these fields in two ways: with an open request when you are uncertain about the document's contents, or by providing a schema when the output needs to match an existing structure elsewhere. Regardless of the method, the output is in JSON format, containing the fields as keys, the requested values as typed data, and null for any missing information.

When a schema is supplied, the output adheres to it precisely, with the same keys and no additional fields, allowing for seamless integration into the next system. The base model is DharmaOCR-LITE v1.1, a refined 2-billion-parameter version optimized from the one in the paper and specifically trained for OCR through supervised fine-tuning and DPO.

A LoRA adapter was then trained on 43,384 training examples covering both request modes, annotated at the field level with examples from receipts, commercial invoices, forms, bank statements, ID documents, and synthetic documents. The benchmark covers 546 documents across three scenarios, resulting in 1,638 total cases: extracting all fields without a schema, using a partial schema naming only real fields, and a schema that also includes non-existent fields.

The score is calculated using field-level F1 (matched by path, including position within a table), multiplied by the Levenshtein similarity on the matched values and the proportion of outputs that are valid JSON. The benchmark was run against four general-purpose frontier configurations, with quality gaps ranging from 1.8 points (Claude Opus 5) to 9.3 points (GPT-5.6 Luna).

The costs are significantly lower when using self-hosted models, ranging from roughly 6 times lower to 147 times lower than the frontier API costs.

Written by urgent.news from Hugging Face's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at huggingface.co →

More in AI

More from Wednesday 4 November →