Why Most AI in Infrastructure and Operations Never Scales

The problem is in the middle.
Emily Hill
Contributing Writer
concept stand out from the crowd with lots of retro robots and one unique one

Most of the conversation around AI in infrastructure and operations (I&O) right now sounds the same, with pipelines, copilots, and automation layers all moving forward in parallel. But if you spend time with teams actually running it, a different pattern shows up pretty quickly.

There’s a quiet sorting happening.

Some of the work sticks. It gets used, it holds up, and over time, it just becomes part of how things run. Other pieces don’t break, but they don’t settle either. They stay in that “almost useful” state, where they’re hard to justify expanding but not bad enough to shut down.

Gartner’s latest data puts numbers to that split.

Only 28% of AI use cases in I&O are fully meeting ROI expectations, while 20% fail outright. The rest sit in that middle ground, exactly where most teams feel the drag.

That’s where a lot of the real work is now, deciding what in the current mix actually earns the right to stay.

Why It Matters: As AI spending moves further into core IT budgets, tolerance for ambiguity drops. Every use case is effectively competing for survival, measured by whether it consistently improves operations in a way that holds up over time. For CIOs, the challenge is enforcing discipline across a growing set of initiatives and making sure resources concentrate around what actually performs.

  • The Middle Tier: The overall distribution of outcomes is uneven enough to require active portfolio management. About 28% of use cases are delivering expected ROI, while 20% fail outright. The majority sit in between, functional, but not yet strong enough to justify broad scaling. That middle tier is where most of the evaluation effort is now focused.
  • Where AI Holds Up: The use cases that consistently deliver are the ones operating inside defined systems. IT service management and cloud operations continue to produce the clearest returns because workflows are structured and inputs are more predictable. AI has less ambiguity to resolve, which leads to more stable performance.
  • Beyond the Vision: The gap shows up quickly in high-autonomy scenarios. Initiatives that depend on AI making independent decisions across complex environments, like self-healing infrastructure or full remediation, are still difficult to stabilize. The vision is there, but the issue is the variability of real systems.
  • Reality Is in the Data: Environmental constraints are still doing a lot of the filtering. Data quality, accessibility, and fragmentation continue to determine whether a use case matures or stalls. At the same time, internal capability, teams that can operate and adapt these systems, remains uneven across organizations.
  • What is Really Driving Results: The organizations getting consistent returns are making deliberate structural choices: embedding AI into existing workflows, aligning ownership across functions, and managing use cases as a portfolio with clear prioritization criteria. That operating model is what’s driving repeatable results.

Go Deeper -> Gartner Says AI Projects in I&O Stall Ahead of Meaningful ROI Returns – Gartner

Trusted insights for technology leaders

Our readers are CIOs, CTOs, and senior IT executives who rely on The National CIO Review for smart, curated takes on the trends shaping the enterprise, from GenAI to cybersecurity and beyond.

Subscribe to our 4x a week newsletter to keep up with the insights that matter.

☀️ Subscribe to the Early Morning Byte! Begin your day informed, engaged, and ready to lead with the latest in technology news and thought leadership.

☀️ Your latest edition of the Early Morning Byte is here! Kickstart your day informed, engaged, and ready to lead with the latest in technology news and thought leadership.

ADVERTISEMENT

×
You have free article(s) left this month courtesy of the CIO Professional Network.

Enter your username and password to access premium features.

Don’t have an account? Join the community.

Would You Like To Save Articles?

Enter your username and password to access premium features.

Don’t have an account? Join the community.

Thanks for subscribing!

We’re excited to have you on board. Stay tuned for the latest technology news delivered straight to your inbox.

Save My Spot For TNCR LIVE!

Thursday April 18th

9 AM Pacific / 11 PM Central / 12 PM Eastern

Register for Unlimited Access

Already a member?

Digital Monthly

$12.00/ month

Billed Monthly

Digital Annual

$10.00/ month

Billed Annually

Would You Like To Save Books?

Enter your username and password to access premium features.

Don’t have an account? Join the community.

Log In To Access Premium Features

Sign Up For A Free Account

Name
Newsletters