Who owns the LLM? 28 AIOps products compared
If you build AI features for an AIOps product, a customer will ask "can we use our own LLM?" sooner or later. What they want is clear: keep the data private and under their control, and still get useful answers from the LLM. The strongest models mostly sit behind public APIs, and that is also where vendors can test and evaluate their features properly. A model running inside the customer's own…
When AIOps vendors integrate large language model (LLM) features, customers often inquire about the possibility of using their own LLM. Ideally, customers want to keep their data private and under their control, while still receiving valuable answers from the LLM. Typically, strong models reside behind public APIs, which are where vendors test and evaluate their features.
However, running the model inside the customer's own network allows all alerts and logs to remain within their walls, even in air-gapped sites. This option makes it challenging for vendors to manage and often leads to struggles with multi-step tool calls.
Some vendors attempt a middle ground by hosting open models in their own cloud instead of relying on a public API. While this keeps data from going to a third-party model provider and allows the vendor to test the exact model it runs, open models are usually weaker than top API models, and the vendor pays for the GPUs.
A comprehensive analysis of 28 AIOps products revealed that most of them run the model on behalf of the customer. Only 13 offer no real control over the model, 10 ship a default and allow users to plug in their own, 5 expect customers to bring their own, and three are open source with two commercial options still offering the model to users.
The vendors that manage the models include Dynatrace, New Relic, PagerDuty, LogicMonitor, Datadog, AWS, Traversal, NeuBird, Metoro Guardian, Rootly, Robusta, and HPE OpsRamp. Hybrid options are available from BigPanda, ServiceNow, BMC Helix, IBM Cloud Pak for AIOps, and Traversal, which allow customers to swap in their own models.
Customer-provided options are available in OpenObserve, Ollama, Aurora, OpenSRE, and HolmesGPT, among others.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
