Alex Karp was right: you don’t own your data

Alex Karp, CEO of Palantir, made a point on CNBC about data ownership in AI that many people missed. He questioned who owns the data, where it is cached, and if anything is transferred back to the provider. As Karp pointed out, OpenAI and Anthropic tell commercial customers they won't train on their data, but after switching to a different AI provider, he found a loophole in the contracts.

According to OpenAI's Services Agreement, they won't use customer content to develop or improve their services, unless the customer agrees. However, the definitions are vague. "Customer Content" consists of the input and output, where "input" is what the customer sends to the model, and "output" is what comes back based on the input.

But the hidden tokens generated during intermediate reasoning steps aren't clearly defined as part of the output. These tokens are valuable and could be considered new data derived from the customer's input. Neither OpenAI nor Anthropic clearly states ownership of these intermediate reasoning tokens. OpenAI's agreement states that customers retain ownership of their input and the output, while OpenAI assigns its interest in the output to the customer.

This is similar to handing a consultant your confidential financial forecast and receiving the final answer while they keep the calculations. This is precisely what happens with AI models, as they generate intermediate reasoning tokens that aren't included in the visible output. The labs profit from these tokens, and their conduct proves their value.

However, they don't clearly assign ownership, which could mean they are using the tokens to train their next models. Until the companies clarify ownership rights, Karp's assertion that you own the prompt and the answer, while the labs keep the notebook, holds true.