What actually gets sent to an AI provider when you ask?

When you interact with an AI model through a service, the provider receives the entire request, not just the sentence you typed. This request includes hidden instructions, the conversation history, any attached files or information retrieved, and the definitions and results of tools the model may use. The model operates independently in each request, so all necessary information must be included in every request.

This can lead to more data being transmitted than might be expected. Retrieval-augmented generation involves the application finding relevant passages from files and including them in the request so the model can draw from them when answering. This can result in additional pages or content being sent to the provider. AI agents operate in a loop where they request actions from the model, perform tasks using tools, and return results, which then become part of subsequent requests.

This means that files, directory listings, command outputs, and more can be transmitted to the provider as part of the task. It's important to note that only content that the application places in the request is sent, not any files or data the model itself might access on its own. Users should be aware of what their applications are allowed to read and transmit, as well as any intermediary servers that might handle the data.

To minimize the amount of data sent, users should start new conversations for new topics, only include relevant sections of documents, and remove unnecessary information. Keeping secrets out of files and outputs is crucial, and using a proper secret store is recommended. Limiting the scope of agents and preferring tools that show the request can also help manage the data transmitted.