Open weights are not open source: Why AI's favorite label is under dispute

The term "open" is frequently misused within the AI industry, appearing in product releases, research papers, policy debates, and investor presentations. A company may publish model files on Hugging Face, developers run them on their own GPUs, and the release is quickly labeled as "open source." However, this is often not the case.

Open weights, which refer to the final weights and biases of a trained neural network, are publicly available. They enable self-hosting, fine-tuning on internal documents, and avoiding proprietary APIs. Yet, receiving open weights does not equate to truly open source AI. Open source AI necessitates full transparency, allowing others to inspect, reproduce, alter, and redistribute the entire system.

Without access to training data, documentation, or sufficient details about the model's creation, others cannot fully test, reproduce, or challenge the work. The Open Source Initiative (OSI), responsible for the Open Source Definition (OSD), explicitly states that open weights expose only a fraction of the information required for full accountability.

James Landay, director of the Stanford Institute for Human-Centered AI, explains that open weights offer progress but do not constitute open models. In contrast to conventional open source software, which revolves around source code, large language models (LLMs) combine code, architecture, and numerical weights derived from proprietary or undisclosed training datasets.

Efforts like the Open Model, Data, and Weights (OpenMDW) license, submitted to the OSI by the Linux Foundation, aim to define separate terms for model architecture, training data, and weights, ensuring they are covered under one agreement. However, the OpenMDW license faces objections from critics, who claim it is tainted by ideological bias against big tech and AI.

The debate highlights the need for licensing terms that encompass code, data, and weights together to establish genuine open AI.