Crossplane vs Terraform: Provisioning S3 Buckets from Kubernetes Without Losing Control

Every platform team eventually faces the same challenge: their application teams require an S3 bucket and a Postgres instance, and they need these resources promptly. Traditionally, teams often resorted to opening a ticket (a slow process), granting them Terraform access (potentially risky), or building a self-service API (a costly endeavor). Crossplane offers a third option, leveraging Custom Resource Definitions (CRDs) that teams are already familiar with.

This article presents a practical example: an S3 bucket and an RDS instance, presented to developers as a single namespaced Claim. It objectively compares this approach with the traditional Terraform plan/apply workflow. By the conclusion of the article, readers should be able to create an XRD, a Composition, and hand a Claim to their teams, while also understanding when Crossplane might not be the optimal solution.

The article delves into the core problem of lifecycle ownership. Terraform's model revolves around a transaction: it computes a difference against the current state, a human reviews the changes, and then the apply command executes. This model works exceptionally well for building a VPC once and reviewing every modification thoroughly. However, it becomes less effective when fifty teams create and destroy buckets on a weekly basis, as each operation necessitates a pipeline run, state lock, reviewer, and merge.

Crossplane, on the other hand, adopts an inverted model. It installs controllers within the cluster that manage AWS resources as Kubernetes objects. There is no plan phase or state file; instead, the desired state is represented by the Kubernetes object, while the actual state resides in the AWS API. A reconciliation loop continuously closes the gap between these two states, consistently and indefinitely. This fundamental difference permeates all other aspects of Crossplane.

When compared to Terraform, Crossplane offers a distinct approach to managing desired state. In Crossplane, the desired state is defined using Kubernetes objects, whereas Terraform relies on an HCL configuration file stored in S3 or Terraform Cloud. Execution differs as well: in Crossplane, the desired state is applied on demand, typically within a continuous integration (CI) pipeline, while Terraform executes on demand or upon schedule.

Terraform provides a native preview through the `terraform plan` command, whereas Crossplane offers a dry-run function or the `crossplane render` command. In terms of drift detection, Terraform only identifies discrepancies when a `plan` is executed, while Crossplane automatically corrects any drift it detects. Lastly, Crossplane's blast radius is limited to one managed resource at a time, whereas Terraform can affect an entire state file.

Installing Crossplane and the AWS provider involves several steps. Crossplane can run on OpenShift, and the Upbound provider images are compatible with the restricted-v2 SCC, eliminating the need for additional SCC exceptions. The installation process involves adding the Crossplane stable Helm repository, updating the repository, and then installing Crossplane using Helm.

The AWS provider is no longer a monolithic entity; Upbound releases separate providers for each service family, which helps manage the number of CRDs and maintains small API server performance.

To use Crossplane, you first install the provider-aws-s3 and provider-aws-rds providers, specifying the runtime configuration to reference the installed IRSA (Identity-Based Service Account). The credentials are sourced from the pod identity, eliminating the need for static API keys. If you're not operating on EKS/ROSA with STS, you can use other credential sources like WebIdentity with an OIDC-federated role or a Secret, although the security implications should be carefully considered.

The article transitions to defining the API your teams will interact with: the XRD (CompositeResourceDefinition). The XRD serves as the contract between Crossplane and the team. It generates two CRDs: a cluster-scoped Composite Resource (XR) and a namespaced Claim. Developers only interact with the Claim. The design principle for the XRD schema is to expose product decisions, not AWS-specific fields.

If a developer can set `blockPublicAcls` to `false`, it indicates that the tool has merely wrapped the AWS API without adding value.

By the end of the article, the reader should have a clear understanding of how to create an XRD, a Composition, and hand a Claim to their teams, along with the knowledge of when Crossplane might not be the ideal solution.