Urgent.News

What's breaking now, across thousands of outlets.

Tech

10 GB Edtech Throughput with Node.js Private Browser Upload Signed URLs

Short answer: send each training artifact straight from the browser to private object storage with a short-lived signed upload URL, keep authorization and retention state in Postgres, and let Node.js coordinate the transfer without carrying the file bytes. Bytes dominate. Start with the bill, because retention multiplies every mistake. For a planning example, take 400 active courses, four 10 GB…

To efficiently transfer large training artifacts using Node.js and private object storage, the process involves sending each artifact directly from the browser to the storage with a short-lived signed upload URL. The Node.js application coordinates this transfer without handling the file bytes. Storage costs are based on retained bytes and time, while upload traffic, validation reads, student downloads, and deletion operations are separate terms.

The main adjustable factor is retained byte-months. The Node.js application should control the intent, not the bulk data, after authenticating the instructor and checking course membership. A signed upload URL is created, and the browser uploads the artifact to the storage. A confirmation step verifies object metadata before the record becomes downloadable.

A signed download URL is issued after a fresh authorization check. Treat presigned URLs like bearer credentials and keep the bucket private. The Postgres row contains enough information to make authorization decisions without parsing object names. Large-file throughput is a capacity budget, and the system's throughput is bound by browser egress, storage endpoint, concurrent connection limits, and retry behavior.

Measure upload performance using a small capacity model before load testing. An idealized lower bound for upload time can be calculated using the provided Python snippet, but real transfers may take longer. Choose a suitable expiry time based on measured p95 completion time and add a bounded margin or use a resumable or multipart protocol.

Monitor progress using XMLHttpRequest.upload listeners, but only confirm the download on the server-side. Admit large uploads by rejecting intents exceeding course or tenant limits and capping concurrent pending uploads per tenant. Implement idempotency on intent creation and confirmation to handle retry storms and prevent duplicate artifact revisions.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in Tech

I wanted my PC in Home Assistant without an MQTT broker, so I wrote my own integration

I have a Snapdragon laptop, a desktop PC and a Home Assistant box on a Raspberry Pi. I wanted the computers in Home Assistant like any other device: sensors, power buttons, notifications, media.

  • Developer created Deskmate integration for Home Assistant without MQTT broker.
  • Deskmate uses encrypted WebSocket connection with X25519 key exchange per session.
  • Features include file sharing, screen time management, sensor monitoring, and power management.

Separate decisions from context, and let agents propose but not ratify

When I build with AI coding agents, I keep running into the problems classical projects have. A lot of talk and notes, and no clear focus on what the decisions actually are. Who said it?

  • Smeldr separates context from reasoning and decision-making
  • Decisions proposed then ratified by authorized person
  • Ratified decisions become basis for further work

More from Monday 16 November →