Urgent.News

What's breaking now, across thousands of outlets.

Tech

Source Deduplication Patterns for a Healthcare Appointment Retrieval Architecture

Short answer: hash and version sources at ingest, group near-duplicate passages at retrieval, and require a current source ID plus page citation for every appointment claim. For a healthcare appointment assistant, deduplicate sources before the language model sees them, and keep the original document IDs beside every passage. That gives the retrieval architecture one authoritative evidence trail…

To deduplicate retrieval sources for a healthcare appointment assistant, the process should be conducted in two stages. In the first stage, document versions should be identified at ingest. This involves using exact hashes to catch repeated uploads, as well as similarity checks to identify near-duplicate passages such as PDFs with new filenames, different metadata headers, or changed footers.

The ingest record must have a stable identity, not reliant on filenames, and should include a stable content hash, source URI, timestamp, and status like active or superseded.

In a healthcare setting, an explicit supersession relationship is preferred over silently deleting older files. This allows auditors to track what was replaced while enabling retrieval to exclude outdated files based on policy. For instance, a clinic manager uploading two versions of a scheduling PDF, one on Monday and another on Tuesday with updated Saturday close times, can still maintain version control without duplicating evidence in the prompt.

At retrieval time, an extensive candidate set should be initially retrieved. Then, candidates should be grouped by canonical source and normalized passage fingerprint. The best-scoring passage from each group is selected, allowing for minor allowances when different sections of the same source are genuinely distinct. This approach prevents a lengthy policy from overshadowing other clinic hours in the context window.

The retrieval architecture must adhere to a strict, deterministic policy for citation. Every scheduling claim should require a source ID and page number, with a clear response when evidence is missing or contradictory. The deduplication process should be both consistent and straightforward. A Go implementation is provided for exact fingerprints and canonical-source selection, although it does not determine the medical authority of the documents. The responsibility for medical authority lies with upstream governance.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

Duplicacy Storage Passwords Protect More Than the Backup Job

Duplicacy Storage Passwords Protect More Than the Backup Job Duplicacy is a backup tool whose storage password can protect backup metadata and encryption keys.

  • Duplicacy uses storage password to protect backup metadata and keys
  • Web edition securely stores password-derived token, not plaintext
  • Lost password makes restoring legitimate backup impossible

63,336 agent skills, 56 paid: what the pricing data actually says

If you have been thinking about selling a SKILL.md , this is the number that should stop you for a minute: in a frozen census of 63,336 agent skills taken on 2026-10-16, exactly 56 skills carry a paid…

  • Only 1.1% of SKILL.md agent skills are paid, despite 4,902 skills with pricing labels
  • Paid skills account for 0.5% of installs, with 454,127 total installs across 56 skills

The five-minute test that would have saved me eleven hours of directory submissions

Eleven hours of directory submissions produced two followed links. Not because the directories were fake, but because I never asked the one question that decides the outcome, and it takes five minutes…

  • Author discovers five-minute test to save eleven hours of directory submissions
  • Checks three details on listing page for linking potential
  • Single script used twice for both current and future submissions

More from Thursday 19 November →