Inside GitLab AI Gateway: How to Register Next-Gen LLMs with GitLab Duo

The GitLab Duo AI Gateway is a privacy-focused, decoupled architecture that allows software development platforms to integrate next-generation language models (LLMs) directly into the developer workflow, without hardcoding them into the main GitLab Rails monolith. This approach enables easier model updates, enhanced data privacy, and multi-provider fallback capabilities.

The AI Gateway's model lifecycle is configuration-driven and validated using Pydantic schemas. Two core configuration files govern every supported model: models.yml for hardware parameters, context limits, provider endpoints, and cost indicators, and unit_primitives.yml for mapping specific GitLab Duo features to approved model engines.

To add a new foundation model, such as Google's next-generation Gemini 4 Argon (gemini_4_argon_vertex), to the GitLab Duo, you must first define the model in models.yml, specifying its provider, model family, description, engine, endpoint, context window, output token capacity, temperature, top-p, and other capabilities. Then, bind the model to unit primitives in unit_primitives.yml, associating it with specific Duo features like Duo Chat and Advanced Code Generation.

Finally, add validation specs and regression coverage tests to ensure the configuration is valid and functions as expected within the AI Gateway's CI pipeline.