Review sites live and die by repeatability. If two testers measure the same laptop and get different battery figures, readers lose trust. The machines that publish those reviews pose the same puzzle for engineers. The Ops Atlas applies the name InfraOps to keeping infrastructure written down as code and verified against what is running. What follows converts that idea into an afternoon audit of a publishing stack.
For definitions we rely on the Ops Atlas page about InfraOps in the infrastructure-as-code lifecycle. It says the label is casual, which means no credential exists to chase. The audit below is ours; each item is anchored in a point the Atlas makes.
Working method
Take the sections one by one. For each line, answer Yes, No or Unknown. Unknown counts as No, because an answer you cannot prove is not an answer. Record the evidence next to each Yes: a link to a repository, a log line, a screenshot of a pipeline run. Reviewers will recognise the method: it is the same discipline as noting test conditions beside every benchmark.
Section A: What exists
- Is every server, database, storage bucket and DNS record in scope listed in code?
- Can you produce a count of in-scope resources and a count under code?
- Do you know the number that is not under code, expressed as an absolute figure rather than a percentage?
- Is there a list of things deliberately out of scope, with a reason for each?
The Atlas works through an example where roughly four fifths of the in-scope estate is under code, then advises reporting the leftover as a raw count, because a percentage can hide how large the blind spot really is. Apply the same lens to a review database: “82 percent of products tested” is less useful than “328 products with no test record”.
Section B: How changes happen
- Does every change reach production through a reviewed request?
- Is a plan, a preview of what will change, attached to each request?
- Does a person read the plan before it is applied?
- Are applies run from a shared pipeline rather than from individual laptops?
- Is there a rule that saved plans are discarded after any other apply on the same state?
The Atlas lists skipping plan review, giant single states and laptop applies among its anti-patterns. If you answered No to more than one item here, fix Section B before spending effort anywhere else.
Section C: What can go wrong at once
- Is each state limited to a size you can plan in a few minutes?
- Is there a guard against deleting more than a defined amount in a single run?
- Are data stores protected by provider-side deletion protection as well as code-side settings?
The last item matters because, as the Atlas notes, a lifecycle rule that blocks destruction fails once the configuration that carries it has been removed. Combining it with a provider setting closes that hole. The Atlas also looks back at a widely discussed 2017 cloud storage outage that began with one mistyped input, and notes the provider later throttled how fast capacity can be removed. Your own audit should ask what your equivalent brakes are.
Section D: Secrets and access
- Do you know whether state files and saved plans contain secrets?
- Are plan credentials treated with near-apply-level care?
- If your CI platform lets branch authors trigger the plan role, have you restricted what that role can read?
- Does the production apply role trust only the production environment?
The Atlas observes that newer tooling offers ephemeral resources, write-only arguments and state encryption as mitigations. Record which, if any, you use. If none, note it as a risk with an owner.
Section E: Drift and verification
- Is there a scheduled comparison of code against live resources?
- Does it use an exit status your pipeline can alert on?
- Is there a target for how quickly a drift finding gets an owner?
- Do you measure drift age, not only incidence?
In its illustration, drift showed up in a small minority of root modules during a month, lingered for a couple of days at the median and reached about seventeen days at worst. That longest-lived finding is the figure that reveals what your process really tolerates.
Section F: Machine images and rebuilds
- Are base images rebuilt on a schedule?
- Do you know the share of instances on recent images?
- Has anyone rebuilt a production component from scratch as a drill?
The Atlas sketches a fleet where three quarters of the instances run on images under a month old. If you cannot calculate your own equivalent, that is a finding in itself.
Who should run it
Ask someone who did not build the system. People who built it tend to answer from memory and intention, while an outsider will ask for the evidence line by line. If you are a small team, swap audits with another site you trust and exchange findings. The cost is a few hours each, and fresh eyes reliably find the item that everyone inside the team assumed was handled. Keep the finished checklist in the repository itself so the next audit can diff against the last one and show the direction of travel.
Scoring
Count Yes answers with evidence. Out of the twenty-three items above, a score below ten means the practice is mostly informal. Between ten and eighteen means you have a pipeline with gaps. Above eighteen, you are ready to set a coverage target and rehearse a rebuild. The Atlas frames this as a ladder of five maturity levels, again its own proposal, ending at a sustained state held over two quarters.
Reading list from our own archive
Auditing a tool stack is similar to choosing software in the first place. Our comparison of ten performance management platforms and their trade-offs shows how we structure a features-and-drawbacks assessment, and the roundup of event venue management tools compared demonstrates the same approach on a different category. Use their method for evaluating the tools your audit turns up.
What to do with the results
Sort the No answers into three piles: cheap and fast, valuable and slow, and not worth doing. Assign each item in the first two piles an owner and a date, then re-run the audit in a month. A repeatable measurement, run on a schedule, beats a one-off heroic cleanup every time.