Ordinary Experts

GitOps with CloudFormation: How We Run It

GitOps usually gets discussed in the context of Kubernetes. We've been running a GitOps approach with CloudFormation for years across our marketplace patterns and client engagements - here's the structure that works.

GitOps gets discussed in the context of Kubernetes more than CloudFormation. The principles transfer cleanly. We’ve been running what amounts to a GitOps approach to CloudFormation for years across our marketplace patterns and client engagements, and the structure has held up well.

This post is a tour of how we run it.

The two-repository pattern

Almost everything in our approach builds on a structural choice we wrote about in 2021: splitting infrastructure code into module repositories and config repositories.

A quick recap:

  • Module repositories define reusable infrastructure components: a VPC, an RDS pattern, an ECS service shape. Versioned with semver. Use git-flow with a CHANGELOG.
  • Config repositories instantiate modules for specific environments. Trunk-based development. No versioning beyond git history.

This separation is the foundation. Without it, GitOps for CloudFormation collapses into a tangle of “is this the production version of the template or not?”

The control loop

The GitOps loop in our setup looks like this:

  1. Engineer opens a PR on a config repository.
  2. CI runs cfn-lint, cfn-nag, and aws cloudformation validate-template on every changed template.
  3. CI runs aws cloudformation create-change-set for each affected stack and posts the changeset summary as a PR comment.
  4. Reviewer approves based on the changeset, not just the template diff.
  5. On merge to main, CodePipeline picks up the change and applies it stack by stack.
  6. Drift detection runs nightly against deployed stacks; any drift produces a Slack notification.

Each step matters. The changeset comment is the single most useful thing we do: it shows what CloudFormation will actually change rather than what the template says, which are often different things.

CHANGELOG and semver for module repos

Module repositories follow strict semver. Every change goes through a CHANGELOG entry. Releases are tags.

Why this matters: a config repo references a specific module version. When you bump the version, you know exactly what changed by reading the CHANGELOG. No guessing whether a “fix” was actually a breaking change.

We’ve watched teams skip this step and pay for it later: every infrastructure change becomes archaeology because no one remembers what was in version 1.4.7 vs 1.4.8.

Trunk-based for config repos

Config repositories are different. They’re trunk-based, no versioning. The branch is main. Changes deploy on merge.

This works because config repos don’t have downstream consumers. They are the deployment. Versioning a config repo is like versioning your deploy script: it adds ceremony without value.

Drift detection

The piece that often gets skipped, and shouldn’t:

aws cloudformation detect-stack-drift --stack-name my-prod-stack
# wait for completion
aws cloudformation describe-stack-resource-drifts --stack-name my-prod-stack

We run this nightly across every managed stack. Drift indicates either: someone made a change in the console (which violates the GitOps model), or CloudFormation’s understanding of state has drifted from reality (which happens with some resource types).

Either way, you want to know early. A drifted stack that no one noticed for six months is much harder to reconcile than one caught the next morning.

What we deliberately don’t do

A few patterns we’ve seen others adopt that we’ve avoided:

Auto-apply to production on merge. We require an explicit promote step from staging to production. The friction is the point: production deploys are slightly more deliberate than dev deploys.

Single mega-stack. We split deployments into multiple stacks (network, data, compute, app) so that a failure in one doesn’t roll back the others. The dependencies are managed explicitly via SSM Parameter Store or Stack outputs.

StackSets for everything. StackSets are the right answer for genuinely fleet-wide deployments. They’re overkill for “I have three environments.” Reach for them only when the fan-out is large enough to justify the operational complexity.

Tooling we use

The toolchain is unglamorous and stable:

  • cfn-lint for static checks
  • cfn-nag for security checks
  • AWS CLI for changeset operations
  • CodePipeline for orchestration
  • A small custom Lambda that posts changeset summaries to PRs
  • Slack for drift alerts

We’ve evaluated CDK and Terraform several times. For our pattern repositories, where we want to ship CloudFormation as a deliverable to clients, raw CloudFormation remains the right choice. For greenfield client work, we now also support Terraform. The GitOps structure transfers, with Terraform Cloud or Atlantis playing the role CodePipeline plays in our CloudFormation flow.

Closing

The mechanics of GitOps with CloudFormation aren’t complicated. The discipline of actually doing it consistently (module versioning, changeset reviews, drift detection) is where the real value lives.

If you’d like help adopting this pattern in your team, we’re glad to talk it through.