{
  "id": 5886843,
  "title": "Finding \"Duplicates Through Time\"",
  "url": "https://urgent.news/2026/09/06/finding-duplicates-through-time",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-06T01:47:18.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/denzyldick/finding-duplicates-through-time-2c56"
  },
  "original_language": "en",
  "account": "The challenge of managing a vast personal photo and video library, totaling 300GB and spanning a decade, proved daunting. Space constraints forced the author to seek a solution without resorting to external cloud storage providers or purchasing additional hardware. The compression route, while tempting, risked compromising the quality of precious memories. The author settled on a more selective approach: deleting redundant large files and targeting individual photographs.\n\nA key realization came from a friend's request for a single picture from their New York visit, only to find the author had captured seven nearly identical shots. This prompted the idea of identifying and eliminating duplicate photos taken in quick succession. The author's solution was Siegu, a local tool that finds burst-mode photographs across extensive libraries.\n\nSiegu operates differently from traditional deduplication tools, which rely on exact byte-for-byte comparisons using MD5 or SHA256 hashes. Instead, Siegu considers both temporal proximity and visual similarity. By analyzing EXIF metadata to group photos shot within seconds or milliseconds of each other, and comparing them using perceptual hashes like pHash or dHash, Siegu efficiently identifies duplicates. This approach reduces unnecessary CPU cycles, focusing only on candidate groups that were actually captured in short bursts. In the next installment, the author will delve into the technical aspects of Siegu's Rust implementation, EXIF metadata extraction, and the perceptual hashing algorithm employed to flag these duplicates.",
  "summary": "Finding \"Duplicates Through Time\": How I Cleaned Up 300GB of Photos Without Losing Quality Lately, I have been playing around with my personal photo and video library (around 300GB of files, spanning 10+ years of memories). I had a specific type of problem: I needed to save space on my SSD, but I did not want to pay any third-party cloud provider for extra storage. To make things trickier, I was…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}