Three Security Boundaries for Hosting Untrusted, AI-Generated HTML

The story revolves around the development of a platform that allows users to upload HTML documents containing interactive elements, such as reports or dashboards. These documents are rendered inside a secure environment to prevent unauthorized access to confidential data. The primary concern is the risk of prompt injection, where a malicious actor could manipulate the HTML file to execute unauthorized actions once it is rendered.

To mitigate this risk, the platform employs three main boundaries:

1. The generation agent receives its own context window, receiving only a portion of the conversation history rather than the entire chat. This prevents the agent from incorporating potentially malicious text from previous parts of the conversation into the generated HTML.

2. The tool prompt requires the generated artifact to be self-contained, meaning it contains only one HTML file with no external requests. This eliminates the possibility of the model making unauthorized network calls to fetch additional resources.

3. A separate model verifies the generated artifact before it is linked and shared. This second model checks the artifact for any errors or malicious code, ensuring that it adheres to the platform's security guidelines.

The platform enforces these boundaries differently:

1. The server enforces the first boundary by checking the user's authorization before allowing access to the rendered artifact.

2. The browser enforces the second and third boundaries through its built-in security mechanisms. These include preventing unauthorized network access, restricting access to sensitive data (like cookies), and disallowing code execution without proper authorization.

By separating the trust verification process into these distinct stages, the platform ensures that even if an attacker manages to bypass one layer of security, they will still face additional obstacles when attempting to exploit the system. This layered approach significantly reduces the risk of a successful attack, protecting both the platform and its clients from potential data breaches or unauthorized actions.