Federated AI Still Has a Central Coordinator Problem

Federated learning is an attempt to make AI training more decentralized, but it still suffers from a central coordinator problem. The issue arises because most AI training starts with pooling data, which isn't feasible when data is sensitive or legally restricted. For example, in 2023, Samsung limited employee use of ChatGPT after engineers inadvertently pasted internal source code into it.

Later that year, Italy's data protection authority fined OpenAI €15M for mishandling personal data during training. Hospitals possess valuable data for improving diagnostic models but can't simply combine patient records.

Federated learning addresses this by moving the model to the data instead of moving data to the model. Each site trains locally and shares only model updates with a coordinator, which aggregates them. This approach has been successfully utilized by Google for on-device keyboard prediction in Gboard and in a multi-hospital study (EXAM) that trained a COVID-19 oxygen-needs model across approximately 20 institutions without sharing patient records.

However, "data never leaves" isn't the same as "nothing leaks." Model updates are computed from private data, and researchers have demonstrated that training examples can sometimes be reconstructed from shared gradients (Deep Leakage from Gradients, Zhu et al., 2019). Membership inference attacks can also reveal whether a specific record was part of the training set.

To achieve true privacy, extra layers are necessary, such as secure aggregation, which ensures the coordinator only sees the sum of updates, not any single site's contribution, and differential privacy, which adds calibrated noise to updates and provides a measurable privacy guarantee at the cost of some accuracy. While these techniques offer some protection, they come at an accuracy cost. Claims of "private AI" should be accompanied by this accuracy tradeoff.

Despite federated learning's decentralizing potential, control remains centralized. Often, one party runs the coordinator, owns the global model, and determines who participates. In most federated systems, data is distributed, but control is not. This centralization contradicts the goal of open, user-owned AI. To truly decentralize AI, additional components are required, such as verifiable records of what happened, independent compute, and governance mechanisms that can be contested.

Fortunately, the necessary building blocks for a practical federated learning system already exist. Federated training can be implemented using tools like Flower or NVIDIA FLARE, running training across separate real machines. Privacy can be enhanced using Opacus, which adds differential privacy to PyTorch, and Nosana, a GPU marketplace for client nodes run on independent operators, not a single cloud vendor.

The accountability layer can be established using Arweave, which permanently stores each round's model hash and training configuration, allowing anyone to verify provenance.

However, a critical aspect often overlooked in federated learning is causal structure. While most federated learning focuses on prediction—classifying images or predicting the next word—decisions in medicine and policy depend on causal questions. Learning causal structure across silos is more challenging due to differences in measured variables, confounders, and local graphs. Federated causal discovery work exists but is less mature than federated prediction.

One practical approach is to learn a causal graph locally at each site using tools like pgmpy, merge the graphs in an aggregation step, and test whether the merged structure remains consistent across sites. Addressing this gap requires measuring the privacy-accuracy tradeoff, handling messy data across sites, and making governance something people can contest.

Building tools like Flower can make a federated prototype achievable in an afternoon, but the real challenge lies in knowing what to measure and ensuring accountability in the system.