10 GB Edtech Throughput with Node.js Private Browser Upload Signed URLs
Short answer: send each training artifact straight from the browser to private object storage with a short-lived signed upload URL, keep authorization and retention state in Postgres, and let Node.js coordinate the transfer without carrying the file bytes. Bytes dominate. Start with the bill, because retention multiplies every mistake. For a planning example, take 400 active courses, four 10 GB…
To efficiently transfer large training artifacts using Node.js and private object storage, the process involves sending each artifact directly from the browser to the storage with a short-lived signed upload URL. The Node.js application coordinates this transfer without handling the file bytes. Storage costs are based on retained bytes and time, while upload traffic, validation reads, student downloads, and deletion operations are separate terms.
The main adjustable factor is retained byte-months. The Node.js application should control the intent, not the bulk data, after authenticating the instructor and checking course membership. A signed upload URL is created, and the browser uploads the artifact to the storage. A confirmation step verifies object metadata before the record becomes downloadable.
A signed download URL is issued after a fresh authorization check. Treat presigned URLs like bearer credentials and keep the bucket private. The Postgres row contains enough information to make authorization decisions without parsing object names. Large-file throughput is a capacity budget, and the system's throughput is bound by browser egress, storage endpoint, concurrent connection limits, and retry behavior.
Measure upload performance using a small capacity model before load testing. An idealized lower bound for upload time can be calculated using the provided Python snippet, but real transfers may take longer. Choose a suitable expiry time based on measured p95 completion time and add a bounded margin or use a resumable or multipart protocol.
Monitor progress using XMLHttpRequest.upload listeners, but only confirm the download on the server-side. Admit large uploads by rejecting intents exceeding course or tenant limits and capping concurrent pending uploads per tenant. Implement idempotency on intent creation and confirmation to handle retry storms and prevent duplicate artifact revisions.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.