Every Mac comes with an (undocumented) family member classifier

Since macOS 13 released in 2022, every Mac computer has been equipped with two hidden daemons whose sole purpose is to create a detailed graph of your personal relationships and activities. This system builds a comprehensive graph by cross-referencing data from various sources such as Contacts, Calendar, Photos, Messages, Mail, location, Screen Time, and numerous behavioral signals throughout your device. The graph is stored in a queryable format that allows Large Language Models (LLMs) to retrieve information from it.

Apple's researchers published a paper in 2023 detailing the techniques behind this graph construction, but there is limited official documentation on how it is integrated into macOS, which versions run it, and the exact list of collected and inferred data. The closest official description is the man page (man 8 knowledgeconstructiond).

The authors of this report conducted a security research investigation into the Apple Intelligence bundle in macOS 26 (26.6.2) and also discovered a similar personal knowledge graph from previous work by Nic Lydon.

The knowledge graph is stored in the ~/Library/IntelligencePlatform directory, containing fifty-one SQLite databases protected by TCC (Transparency, Consent, and Control) Full Disk Access. The most interesting database is named graph.db, which is a triple store containing 181 entity classes and 278 predicates. Each row represents a subject, predicate, and object relationship. The subjects can be entities (people, apps, events), while the objects include plain values or other entities, forming the edges of the graph.

The daemons populate these predicates by collecting data from various sources, such as Contacts, Photos, Calendar events with attendees, Family Sharing information including ages, Medical ID emergency contacts in Health, Siri Suggestions from Mail and Messages, App Intents, and installed apps. The graph also includes information about who appears together in photos, communication logs, location and movement history, and a wide range of device signals like screen usage and ambient light.

There are 57 predicate classes describing various activities, with a focus on what you are doing rather than just who or where you are. However, it is unclear if all classes are actively used, as some may be placeholders. The report flags those classes for which mechanisms could potentially be responsible, but this should be considered as inferred rather than definitive information.

The predicates listed in the graph describe a wide range of behaviors and relationships, including sleep, exercise, mindfulness, bedtime routines, commutes, dining, and other activities with matching types. Twelve embedding models within the system are used to determine when two records represent the same person, while XGBoost classifiers assign family relationships based on 144 behavioral features.

The report also discusses the implications of this system, including potential privacy risks if such a personal knowledge base were to fall into the wrong hands, as well as the consequences of relying on a single point of failure in the privacy sphere.