{
  "id": 18777972,
  "title": "Large Image Uploads: A Node.js Sharp and Multipart Retention Runbook",
  "url": "https://urgent.news/2026/11/07/large-image-uploads-a-node-js-sharp-and-multipart-retention-runbook",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-11-07T19:48:25.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/elvrythn486209/large-image-uploads-a-nodejs-sharp-and-multipart-retention-runbook-3gi4"
  },
  "original_language": "en",
  "account": "Managing large image uploads on a gaming platform requires careful consideration of backups, restore procedures, and thumbnail generation. To ensure tenant data integrity and prevent accidental deletion of recoverable copies, a runbook outlines a structured approach.\n\nKey principles include:\n- Deletion should be a deliberate policy decision, not an automatic operation\n- Completed multipart uploads are the sole input to the Node.js Sharp thumbnail pipeline\n- Originals remain private while derivatives are stored separately\n- Restore access is verified through tenant authentication before serving bytes\n\nOperational constraints dictate treating thumbnails as separate entities from originals. A comprehensive audit record should be maintained for each candidate for deletion, capturing:\n- Tenant ID\n- Asset ID\n- Upload session\n- Object key\n- Retention class\n- Completion time\n- Restore hold status\n- Policy version that enabled removal\n\nThis record provides on-call staff with a clear audit trail to explain deletions, ensuring accountability beyond mere storage provider logs. Verification of the restore process involves confirming three critical aspects:\n1. The multipart session was uploaded under a valid upload ID\n2. The final object is restorable\n3. No thumbnail has been generated for that object\n\nRestore verification confirms that the tenant can access the original snapshot via authorized paths, and that all derivatives remain intact. The policy must account for scenarios where:\n- Snapshots are within their recovery window but derivatives have already met deletion eligibility\n- Incomplete multipart sessions may be abandoned post-client retry policies, yet the original remains protected\n- Abandoned multipart sessions should not automatically trigger disposal of the original object\n\nStorage lifecycle rules should only be applied for coarse cleanup when their behavior aligns with the defined policy, ensuring the final eligibility determination is made within the application's own ledger. If the storage provider lacks the necessary audit and retention controls, it is unsuitable for this workflow. Instead, a storage boundary that can be operationally tested should be maintained. This testing should include:\n- A held snapshot\n- An abandoned multipart session\n- A derivative whose source is still protected\n\nThe Node.js Sharp thumbnail pipeline should be designed as a separate contract from the upload coordinator and restore API. The coordinator handles multipart session management, records completion or abort events, and maintains two distinct namespaces for originals and derivatives. The thumbnail worker processes completed source keys with Sharp, generating revisioned derivative keys. The restore API authenticates tenants and snapshots before permitting private reads.\n\nCritical to this process is ensuring that:\n- Upload completion does not imply coordinator commitment\n- Workers starting from part lists can handle incomplete data\n- Workers starting from persisted completion transitions have a stable gate\n- Transformation revisions and derivative keys are deterministic\n- Originals and derivatives are stored in separate namespaces to prevent accidental deletion\n- Private read URLs are short-lived and only created post-authentication\n\nThe coordinator's output remains minimal, consisting of a simple completion gate check in Go, without dependencies on storage SDKs. The key takeaway is maintaining this process as boring and straightforward as possible, focusing on deterministic outcomes and thorough verification steps.",
  "summary": "Short answer: for a gaming platform keeping per-tenant backups and restoring a selected snapshot, let deletion be a reviewed policy decision, and let a completed multipart object be the only input to the Node.js Sharp thumbnail pipeline. Keep originals private, keep derivatives separately named, and make the restore path prove tenant ownership before it returns bytes. The operational constraint…",
  "key_points": [
    "Deletion is deliberate policy decision, not automatic",
    "Completed multipart uploads feed Node.js Sharp pipeline",
    "Originals private, derivatives stored separately"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}