Ordinary Experts

Adding Amazon Q Developer to Your CodePipeline

A practical walk-through for putting Amazon Q Developer to work inside CodePipeline - reviewing CloudFormation diffs in pull requests, flagging IAM regressions, and where we draw the line between assistance and automation.

TL;DR

You can wire Amazon Q Developer into CodePipeline (or CodeBuild) to review pull requests, run security checks on CloudFormation changes, and flag regressions automatically. The piece worth getting right is where in the pipeline you place it, and what you do with its output.

Why integrate AI into CI/CD

In a previous post we described how we use Q Developer interactively in the IDE. That covers the authoring side. But the same model is useful at PR time, where the change is already drafted and the question is “should this merge?”

The interactive use case helps the author. The CI use case helps the reviewer. They complement each other.

Note: Since this post was drafted, AWS has shipped a native Q Developer GitHub integration for pull-request review. For teams on GitHub, that’s a lower-friction starting point than the hand-rolled CodeBuild approach below. The CodeBuild pattern still applies if you’re on a different SCM or want more control over prompts and findings.

Where to put it in the pipeline

We’ve found three useful integration points:

  1. PR-time review. A CodeBuild job triggered on PR creation that runs q chat (or the API equivalent) over the diff and posts a comment with findings. Non-blocking; advisory.
  2. Pre-deploy validation. After cfn-lint and cfn-nag run, an additional step that asks Q to look at the changeset for IAM regressions, missing tags, or drift from team conventions. Blocking on critical issues.
  3. Post-deploy monitoring. A scheduled job that summarizes recent changes and flags anomalies. Reporting, not gating.

We don’t recommend putting Q in the deploy path itself. The latency, occasional failures, and cost don’t justify what you get there.

A minimal CodeBuild example

Here’s a buildspec.yml for the PR-review use case:

version: 0.2

phases:
  install:
    commands:
      - pip install awscli
      - curl -L https://desktop-release.q.us-east-1.amazonaws.com/latest/q-x86_64-linux.zip -o q.zip
      - unzip q.zip && ./q/install.sh --no-confirm

  build:
    commands:
      - git fetch origin $BASE_BRANCH
      - DIFF=$(git diff origin/$BASE_BRANCH...HEAD -- '*.yaml' '*.yml' '*.json')
      - |
        echo "$DIFF" | q chat --no-interactive \
          "Review this CloudFormation diff. Flag IAM scoping issues, \
           missing tags, deprecated resource types, or drift from \
           AWS Well-Architected guidelines. Be brief." \
          > q-review.md        
      - gh pr comment "$PR_NUMBER" --body "$(cat q-review.md)"

The example uses the GitHub CLI; adjust for Bitbucket or self-hosted SCMs as needed. The substance is the same.

What we tell Q to look for

The prompt matters. We’ve iterated to a fairly specific instruction set:

  • Flag IAM resources with * actions or * resources unless clearly justified.
  • Flag missing required tags (we have a per-client list).
  • Flag use of deprecated CloudFormation resource types.
  • Don’t comment on style or formatting (linters cover that).
  • Be terse. One bullet per finding. Cite line numbers.

Generic prompts produce generic, low-signal output. Specific prompts with clear “do not comment on” exclusions are much more useful.

The trust boundary

Two rules we live by:

Q’s review is advisory, not authoritative. We don’t auto-merge based on Q’s verdict, and we don’t auto-block. A human reviewer reads Q’s findings the same way they’d read findings from a junior teammate: useful input, not a final decision.

The deploy decision is human. The temptation to let Q gate deploys (“if score > X, deploy”) is real. We resist it. The cost of one bad call is too high relative to the velocity gain.

Cost

For a moderately busy team (say, 30 PRs a week, average 200-line diff), the Q Developer Pro cost lands in the range of a few dollars per developer per month. Less than a single hour of engineering time on a misconfigured production deploy.

Closing

The mental model that’s worked for us: Q in the pipeline is a teammate doing first-pass review while a human is in another meeting. It catches obvious problems, frees humans to focus on architectural review, and never claims more authority than that.

If you’d like help building this out for your team’s pipelines, we’d be glad to help.