{
  "id": 1692253,
  "title": "LLMs and Contextual Integrity",
  "url": "https://urgent.news/2026/08/18/llms-and-contextual-integrity",
  "topic": "ai",
  "section": "AI",
  "published": "2026-08-18T10:40:16.000Z",
  "source": {
    "name": "Schneier on Security",
    "slug": "schneier-on-security",
    "url": "https://www.schneier.com/blog/archives/2026/08/llms-and-contextual-integrity.html"
  },
  "original_language": "en",
  "account": "Recent research has shed light on the challenges surrounding the contextual integrity of large language models (LLMs). Two papers, \"CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMs\" and \"Contextual Integrity in LLMs via Reasoning and Reinforcement Learning,\" delve into this pressing issue.\n\nThe first paper introduces CIMemories, a benchmark designed to assess how LLMs appropriately control the flow of information from memory according to the task context. By utilizing synthetic user profiles with over 100 attributes each, paired with diverse task contexts, the authors found that frontier models exhibit up to 69% attribute-level violations. In other words, sensitive information is being leaked inappropriately. Moreover, these violations tend to accumulate over time, with GPT-5's violations rising from 0.1% to 9.6% as the number of tasks increases from 1 to 40. Even when the same prompt is executed multiple times, the model demonstrates arbitrary and unstable behavior, leaking different attributes each time.\n\nInterestingly, the paper highlights that privacy-conscious prompting does not solve this issue. Instead, models tend to overgeneralize, sharing either everything or nothing, rather than making nuanced, context-dependent decisions. This finding demonstrates that the problem lies in the fundamental limitations of LLMs, requiring contextually aware reasoning capabilities rather than just better prompting or scaling.\n\nThe second paper explores a potential solution to this problem by proposing a reinforcement learning (RL) framework that instills reasoning about contextual integrity (CI) in LLMs. By training LLMs to reason explicitly about CI when deciding what information to disclose, the authors were able to reduce inappropriate information disclosure while maintaining task performance across multiple model sizes and families. Importantly, the improvements achieved using this synthetic dataset with only 700 examples transferred to established CI benchmarks such as PrivacyLens, which relies on human annotations to evaluate privacy leakage of AI assistants.\n\nThese studies emphasize the need for LLMs to exhibit contextually aware reasoning capabilities to ensure contextual integrity. By addressing this challenge, we can pave the way for more privacy-conscious AI systems that make informed decisions about the appropriate information to share in various contexts.",
  "summary": "I have been thinking a lot about AI and integrity. Part of that is contextual integrity. I recently found two papers on the topic. “ CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMs “: Abstract: Large Language Models (LLMs) increasingly use persistent memory from past interactions to enhance personalization and task performance. However, this memory…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}