After reviewing and working with a wide range of platform teams, I can say this with confidence. The CI/CD tool always has a name, but the portfolio around it never does. That gap is the entire article.
Key Points
- DevOps automation is a collection of bets. Every CI/CD job and deployment workflow carries an initial investment and an ongoing maintenance cost, and most tech leaders don’t know what those costs are.
- The four‑question filter decides what gets built. Before I fund any automation, I run it past a four‑question card: is it stable, does it run often enough to matter, what’s the blast radius when it misfires, and who owns it after launch. I want all four clean, not three out of four.
- Failure modes are predictable, and tool sprawl is the worst of them. I watched it eat a platform team’s roadmap for a full year. The rest, from false‑confidence pipelines to configuration drift and brittle tests, clear inside a quarter.
- Platform engineering’s job is to own automation across every software team. The usual hurdle is finding capable platform engineers. Engineers who can hold a pipeline, a Terraform mono‑repo, and a developer platform in one head are rare.
This piece walks through how I run that portfolio call for platform teams, from a few dozen engineers up to organizations supporting thousands of developers. It’s a practitioner’s frame, not an audit checklist: what DevOps automation actually covers, where the return erodes over time, and the operating model that keeps automation trustworthy instead of turning it into a second unpaid job for whoever’s on call.
None of what follows argues against automation. It argues against pretending automation is free once it ships. Treat it as a budget conversation you can have with your team and with finance the next time someone asks why headcount is going toward maintaining pipelines instead of shipping features.
What Is DevOps Automation?
DevOps automation refers to the recurring work that development and operations teams offload to software: provisioning infrastructure, integrating and testing code changes, deploying releases, and monitoring production environments. It’s a superset of a CI/CD pipeline. The continuous integration and continuous delivery process that automates the movement of a code change from commit to production. DevOps automation also encompasses infrastructure as code (IaC), configuration management and system configuration, version control, observability, automated monitoring, alerting, and incident response across the entire software development lifecycle.
When I start working with a new platform team, most have already implemented the basics around CI/CD. GitHub tooling (Actions or CI), Terraform, automated testing, dashboards, and an alerting setup are the norm. This is where I pivot to looking at the efficacy and cost around those parts. I always ask the same set of questions right out of the gate:
- Which automation tools still pay rent?
- Which ones quietly became a second system the team maintains for free?
- What do we do about the gap?
Automation isn’t a one‑way ramp. More pipelines, more IaC, and everything ships faster isn’t the truth of it. Every automation you ship is a bet. It has a setup cost, a maintenance cost, and a decay risk. Some bets pay off for years. Others rot in eighteen months and start charging you interest.
The Checklist View: CI/CD, IaC, and Everything Else
When I ask leaders what share of their platform team’s sprint goes to maintaining automation that already exists, most can’t answer. The truth is usually 30% or more, and they’re surprised by their own number.
That’s the blind spot the checklist view creates. CI/CD, some Terraform, automated tests, a dashboard or two, the usual. Everyone treats the list as finished the moment it’s stood up. It isn’t. I trust DORA’s research more than most industry data because nobody there is selling anything, and their findings hold up over time.
Somebody still has to keep the promise behind every alerting rule, and I’ve never once seen that written into anyone’s job description.
That’s the reframe I’m arguing for. Treat DevOps automation like infrastructure. Budget it, staff it, and eventually retire it.
Configuration Drift, Pipeline Rot, and Other Ways Automation Breaks
If you want the cold plunge of reality, ask yourself one question: when did a human last verify that the safety nets catch anything? The dread you feel means it’s time to take a much closer look at your DevOps pipeline.
Failure modes at scale are predictable, and configuration drift is the classic. Somebody made a manual change during an incident and never backported it, so Terraform says one thing while the cloud console says another. HashiCorp’s own drift-detection guidance treats this as routine rather than exceptional, which matches what I see. A Jenkins box nobody dares upgrade, still running a deploy that the one person who understood it took with them two reorgs ago, is pipeline rot, the quieter cousin to configuration drift.
Then there’s tool sprawl, where every team stands up multiple tools and its own CI systems. You’re paying a coordination tax with every cross-team release. Alert fatigue rounds it out; if your on-call is lacking pages without acting on them, your monitoring is training people to ignore it. For this, Google’s SRE workbook chapter on alerting is the best corrective I know.
All of that is real, but handleable. The failure mode I think is the real landmine, and the one with the widest blast radius, is the false‑confidence pipeline. That’s where automation reports success while verifying nothing.
When to Automate and When to Stay Manual
Four things have to be true before I’ll automate a process, and I want all four, not three out of four:
- The procedure is stable, meaning no real changes for months.
- The frequency is real and occurs often (really often).
- The blast radius is understood. You need to know what a misfire means and how to back out the work (i.e., it’s reversible).
- Someone real owns it. A person or team counts.
The SRE book’s definition of toil (manual, repetitive tasks that scale with headcount) is central to my decision making. If the work isn’t that, I steer away from the urge to automate. It isn’t hard to apply, because the four-question list makes the call clear. Unless it’s frequent and stable, just say no.
One thing I always bake in early is security scanning. It passes all four tests without debate: stable, runs on every commit, obvious owner. I consider it the highest‑return automation most teams underfund.
| Signal | Automate | Stay manual for now |
| Stability | Steps unchanged for 3+ months | Procedure still shifting per run |
| Frequency | Weekly or more | Quarterly or rarer |
| Blast radius | Understood and reversible | Unknown or irreversible |
| Ownership | Named team on the hook | “Whoever wrote it” |
The table comes straight from my platform review deck. Pair it with real pipeline failure rates, and most arguments fade quickly. The ownership row keeps the point front and center, because people tend to sideline it when they’re focused on the pipeline’s action scope.
Ownership, Platform Engineering, and the Operating Model
Unowned automation is a liability on a delay timer. I repeat that line in every platform review I run. The day the engineer who wrote it leaves, fifty people are stuck depending on a pipeline nobody can safely touch. The straightforward organizational fix is to assign ownership at the platform level and have that team treat automation as a product with internal customers. The CNCF platforms whitepaper lays out this model well, and it’s blessedly vendor neutral.
That capability gap is now the norm, not the exception. DORA’s 2025 research found that internal developer platforms have gone from a nice-to-have to a near-default: 90% of organizations report using one, and three-quarters now have a dedicated platform team. Having a platform isn’t the differentiator anymore. Platform quality is.
The org chart’s the easy part. The operating model’s what teams skip, and it’s what actually decides whether the automation outlives the person who built it. What I want on paper, with a name next to each, is who runs it day to day, who takes the 2 a.m. page when it breaks, and what capacity is budgeted to keep it alive.
On my teams I plan for 25% to 30% of platform capacity going toward keeping the existing estate healthy, and I put that number in front of leadership alongside new‑feature capacity so nobody pretends the cost isn’t there. This is also where the SRE connection lives. Blow the error budget and new features wait. Toil caps make the same call on maintenance time.
The honest constraint for most 200 to 5,000 person companies is people. Engineers who can own CI/CD pipelines, IaC, and an internal developer platform end to end are scarce, and the common stall at “we have CI/CD and some Terraform” is usually a capability gap. You close it by hiring, growing your own, or bringing the skills in from outside.
Choosing Automation Tools: Buy, Build, or Shell Script
The last team that pushed me for an internal developer platform had about forty engineers. I told them to wait. That’s usually the answer under a few hundred engineers, and the bar to change my mind is higher than teams expect.
I don’t take the build question seriously until a team’s big enough that a paved road pays for its own product team. In practice, that means Kubernetes, a few hundred developers, and often distributed systems complex enough to justify a dedicated platform team. Below that, buy the commodity parts: source control, CI runners, and anything nobody differentiates on. Spend the flexibility you give up on never having to maintain it.
Backstage is where everyone points, and it earns the reputation, but it’s a much bigger commitment than the demo suggests. The teams that get burned treated it like a tool to install when it’s really a product to staff, and the demo never shows you the crew it takes to keep it alive.
Plenty of times the honest answer’s still a shell script and a CI runner. If the surface is small and stable, I don’t reach for a platform that sells “self‑service” and quietly needs a full‑time team behind it. You end up staffing a team to babysit the platform you adopted so you wouldn’t need one.
AI in DevOps Automation: Real Use Cases and Current Limits
Every leadership meeting I sit in now has an AI agenda item. Log analysis and anomaly detection is the one I actually lean on. The models surface patterns in telemetry that people skim right past. Test generation earns its keep for scaffolding test automation coverage on legacy code.
The catch I keep pointing out is that the generated tests inherit whatever the code already assumes. This means you can just build a false‑confidence suite faster. Predictive alerting I still file under roadmap. It’s promising, and in my experience still too noisy to trust the vendor pitch.
DORA’s 2025 research backs up what I’m seeing in the field: heavier AI adoption still correlates with more delivery instability, not less. There’s more rework, more change failures, and longer cycle times to resolve issues, alongside the throughput gains.
I don’t read that as an argument against AI. I read it as evidence that AI generates code and configuration faster than most teams can review it, which means it amplifies whatever discipline gaps you already have. If your test suite’s brittle, AI will help you write more brittle tests at scale. Pilot it where the review cost is low and the feedback loop is fast. Don’t reorganize your operating model around it yet.
The Portfolio Review I Run Twice a Year
Twice a year, usually right after planning, I block a half day to review everything we’ve automated as a portfolio. Not the roadmap, but the existing estate. A healthy portfolio looks boring. Every automation has a named owner, a maintenance line in the budget, and a retirement path if it stops earning its keep.
The leading indicators that the estate’s starting to cost more than it returns:
- Unplanned fixes consume more than about 20% of the platform team’s sprint, and the share’s rising.
- Pipeline duration creeps up quarter over quarter and nobody owns the performance number.
- On‑call fatigue shows up as acks without action or standing snoozes.
- Tool duplication appears when more than one tool does the same job across teams (e.g., two CI systems, three deployment mechanisms).
- Orphaned automation exists for anything whose author has left and whose current owner’s still TBD.
If you’re not watching these data points, you won’t notice the slow bleed until it’s expensive to fix. The estate audit’s dull work, and it’s the difference between a portfolio and a junk drawer.
Budget the Trust, Not Just the Build
If you take one idea from this, make it the reframe that DevOps automation is infrastructure. Infrastructure gets budgeted, staffed, and retired. Live by the four‑question test before you automate anything.
A new tool gets one question: who will own it in two years? Put the portfolio review on your calendar before the next planning cycle, because the automations you already run are quietly voting on your delivery speed every day, whether or not anyone’s watching them.
None of this slows you down. Owned automation is what lets teams move faster, because they trust it enough to build on top of it. Budget for that trust the way you budget for uptime.
