Your Demo Server Should Never Hold the Model Key
A public demo needs a model, but it does not need the credential that pays for it. When free model access and a free server option push the cost of shipping a demo close to zero, the cheapest possible architecture also becomes the riskiest one: paste the provider key into the host's environment variables, forget about it, and hope nobody looks. I want to walk you through a different arrangement,…
Free demo services often provide a model and server options for free, which can be tempting for creators looking to showcase their work. However, this approach can be risky as it combines two different trust relationships into one process. The first boundary is between the public internet and your demo, where anonymous users can send requests.
The second boundary is between your demo and the model provider, which handles your credential, prompt text, and request volume. When both boundaries are in the same container, a mistake in one can compromise the other. To mitigate this risk, it's recommended to separate the key from the public server. The key should live in a controlled process, while the server only holds a token that expires.
The provider key should be stored in its own process and communicated via a separate token that can only reach the broker, which answers with text and usage counts instead of headers or tracebacks. This approach ensures that even if the demo leaks, the leaked information is worth a token that dies quickly and can only reach the broker, which doesn't have direct internet access.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
