Inside ‘Project Lily’: The Humans Reading Your ChatGPT Chats

OpenAI is recruiting hundreds of contractors to read and review real users' ChatGPT prompts. These prompts may contain sensitive personal information, as they can include entire conversations between users and the chatbot, which many users might not be aware are being read by actual people. The objective of these prompt review teams is to enhance the quality of ChatGPT's responses, with contractors rating and critiquing the chatbot's generated replies.

Internal documents reveal contractors are training ChatGPT to avoid anthropomorphizing itself and to be less sycophantic, addressing a critical issue for OpenAI, as excessive sycophancy was partly responsible for multiple suicide cases, according to various lawsuits.

Although OpenAI states it removes personal information before prompts reach reviewers, sensitive details may still slip through. Users often use ChatGPT as a therapist, professional assistant, or digital friend, providing it with intimate details about their lives. Contractors do not see ChatGPT usernames, but personal information can sometimes still get through.

OpenAI acknowledges that sensitive details can sneak past their Privacy Filter model, which is designed to detect and remove personal information. However, the model can still miss uncommon identifiers or ambiguous private references, and over/under-redact entities when context is limited.

Despite OpenAI's claims that it tells users about the possibility of human review, 404 Media has not found any explicit disclosure from the company to this effect. The only mention of human review is in the context of content that violates the site's terms of service or poses a safety risk. OpenAI's privacy policy states it may use personal data to improve its models, but the specific details of how and when this happens remain unclear.

If a user chooses to delete their ChatGPT conversations, OpenAI says it will remove these from its systems within 30 days, unless they have already been de-identified and disassociated from the user's account. Users need to proactively turn off the "improve the model for everyone" setting to ensure their chats are not used to improve the company's models, as it is turned on by default for free, Plus, and Pro plans.