100% Local AI Meets Self-Hosting: From GitHub Repo to Production Container Stack Without Cloud APIs

Self-hosting enables data sovereignty, privacy and complete control over one's digital infrastructure. However, bringing new open-source tools from GitHub to a home server or edge cluster can be a laborious process involving Dockerfiles, port conflicts, volume permission issues and dealing with Docker vs Rootless Podman engine differences.

While cloud-based LLMs like OpenAI and Anthropic can help with DevOps tasks, using them poses a problem because they require sending internal server details, private network IP addresses and system configurations to commercial cloud APIs, which directly threatens data sovereignty.

To resolve this issue, we incorporated local open-weight LLMs (operating via Ollama on consumer hardware) into NjordDeploy, an open-source, agentless self-hosting orchestrator. The experimental design involved running three distinct open-weight models (Meta Llama 3, Mistral and Qwen 2.5) within a deterministic Air Traffic Control system, ensuring zero data leaves the private LAN.

The system evaluated three local models against a single Git repository, which resulted in 100% determinism, meaning all models produced identical port allocations, volume mounts and Jinja2 conditionals. There were no cloud leaks, and the entire process was executed offline, incurring zero recurring API costs. NjordDeploy builds upon the PiSelfhosting project and supports heterogeneous Linux environments, featuring an agentless SSH architecture, dual container engine support, Proxmox hypervisor cross-validation, and a curated stack of 46+ services in domains such as Smart Home, Privacy, Productivity and Local AI.