How to Save 70% on Cloud Costs: Re-architecting AWS to Proxmox LXC

As the VMware Exodus intensifies following Broadcom's acquisition of VMware, which has led to significant increases in licensing fees, businesses are seeking viable alternatives. While a typical response is to migrate workloads to AWS or Azure, this often results in shifting from licensing expenses to compute expenses. In this analysis, I examine the process of re-architecting a high-traffic SaaS application from a heavily reliant AWS EC2/EKS setup to a bare-metal Proxmox LXC cluster, achieving a 70% reduction in monthly cloud costs while enhancing input/output performance.

The Issue: Excess Compute Expenses at AWS The client's business operated a microservices architecture on AWS, utilizing EKS (Elastic Kubernetes Service) for orchestration, PostgreSQL databases hosted on RDS, and Redis cached via Elasticache. Their monthly expenditure approached $12,000, with the majority of costs stemming from compute instances (EC2), NAT Gateway data transfer, and premiums for managed services.

The Resolution: Proxmox Virtual Environment We shifted towards a collocated bare-metal strategy using Proxmox VE. This platform supports both KVM-based virtual machines and lightweight LXC (Linux Containers). 1. EKS Substitution with LXC and Docker Swarm Rather than employing resource-intensive KVM virtual machines, we deployed stateful services directly onto Proxmox LXC containers.

LXC shares the host kernel, delivering bare-metal performance without any virtualization overhead. For stateless microservices, we utilized a lightweight Docker Swarm cluster operating across three VM nodes. 2. Storage Architecture: ZFS In contrast to the expensive AWS EBS volumes, which charge for every I/O operation, we configured NVMe drives into a ZFS RAID10 arrangement on the bare-metal servers.

ZFS offers native compression (lz4), snapshot capabilities, and exceptional read/write speeds that far surpass standard EBS volumes, all without the per-IOPS charge. 3. Networking and Entry Points We replaced AWS ALB with a highly available HAProxy + Keepalived configuration spanning two Proxmox nodes. This substitution eliminated the hourly costs associated with load balancers while offering greater control over SSL termination and request routing.

The Results By transitioning away from managed public cloud services and harnessing the full potential of bare-metal infrastructure with Proxmox, the overall infrastructure costs decreased from $12,000 per month to roughly $3,500 monthly (including colocation and bandwidth). Additionally, latency improved due to the removal of the AWS virtualization layer.

If you're considering escaping cloud vendor lock-in, I specialize in Proxmox architecture and Kubernetes migrations. Reach out to discuss your infrastructure needs.