Pilot to production: why most AI stalls — and the operating model that ships it
Only ~48% of AI projects reach production. The blocker isn't the model — it's the operating model. Here's the playbook that ships.
AMDIM · Operations
June 14, 2026
You've seen the demo work. You've probably also seen that demo never reach production — and in our experience it's almost never a model problem.
The technology mostly works. AI stalls on the operating model.
Across RAND, Gartner, McKinsey and MIT, the conclusion is the same: unclear objectives, inadequate data, missing production infrastructure, and processes left unchanged. Every one of those is an operating-model decision — which means a leadership team can fix every one of them.
Why pilots stall
RAND's 2024 study produced the most rigorous ranked list of causes — and what's striking is how little of it is about the model.
Six Ranked Causes of AI Project Failure (RAND · 2024)
- 01
Stakeholders misunderstand or miscommunicate the problem AI is meant to solve
The number-one cause — not data, not infrastructure. The business problem handed to the technical team is not the actual problem that needs solving. By the time the mismatch surfaces, months of development have been spent on the wrong thing.
- 02
Inadequate or poorly governed data
Not just insufficient volume — the wrong data: incomplete historical records, inconsistent labelling, data that captures what was measured rather than what needed to be measured. Data adequacy assessment must happen before architecture decisions.
- 03
Chasing technology for its own sake
Selecting the most sophisticated model or framework rather than the simplest one that solves the problem. Complexity is a liability in production systems — it compounds debugging time, increases costs, and makes governance harder.
- 04
Missing production infrastructure
Building the model without building the pipeline: no monitoring, no retraining triggers, no integration with existing systems. The model works in the notebook. It was never designed to work anywhere else.
- 05
The problem is genuinely too hard for current AI
A genuine capability limitation — the task requires reasoning, reliability or consistency that current models cannot provide at production quality. Rare, but real. The honest response is to scope a simpler version that current models can handle.
- 06
Layering AI on unchanged workflows (McKinsey)
Automating a step inside a broken process produces a faster broken process. McKinsey found this to be the single strongest predictor of whether bottom-line impact materialised — and 80% of organisations make exactly this mistake.
/ The McKinsey finding
80% of organisations bolt AI onto existing processes. Workflow redesign is the strongest predictor of impact.
McKinsey found roughly 80% of firms layer AI onto their existing processes without redesigning the workflow — even though workflow redesign is the single strongest predictor of bottom-line impact. Automate a step inside a broken process and you get a faster broken process.
The operating model that ships
Three Foundations of a Scalable AI Operating Model
Hub-and-spoke structure
Central hub owns standards, platform, and governance — embedded spokes inside business units execute locally with autonomy. McKinsey finds most successful organisations land here. Avoids the bottleneck of a pure centralised CoE while maintaining quality and governance consistency.
Pods that own the outcome
Business, operations, and technical people in a standing pod with end-to-end ownership and accountability for the outcome — not for delivering features. Outcome accountability changes what gets prioritised and what gets cut.
Value gate on everything
Tie each initiative to a specific value driver and KPI before it starts. If you can't answer the value question, don't launch. This is the cure for pilot sprawl — and it keeps the portfolio honest at each funding gate.
AI Organisational Structure: Trade-offs
| Feature | Hub-and-spoke (recommended) | Pure centralised CoE |
|---|---|---|
| Avoids bottleneck — BUs can execute without central queue | ||
| Maintains quality standards and platform governance | ||
| Enables local business context in each initiative | ||
| Scales beyond 3 simultaneous use cases | ||
| Governance stays consistent across business units |
A pilot is a notebook. A product is a pipeline.
As Andrew Ng puts it, in production systems the machine learning code is just a small piece of the puzzle — the model is the small part. The machinery around it is the actual product.
MLOps Maturity: The Climb from Pilot to Product
Level 0 — Manual
Notebooks and hand-offs
Every step is hand-run. Results move around as files. No automation, no monitoring. This is where stalled pilots live — impressive as a demo, invisible as a product.
Level 1 — Automated Pipeline
Continuous training
The model retrains itself on fresh data automatically. This is the piece ordinary DevOps doesn't have — because models decay in a way software doesn't. Silent degradation is the risk managed here.
Level 2 — CI/CD/CT
Full automation
Integration, delivery and continuous training fully automated. Fast, reliable updates at scale. Input distribution monitoring — catching drift before output quality drops.
Silent degradation is the hidden production risk
A normal deployment pipeline reports 'success' while the model's predictions quietly rot — because upstream data shifted or customer behaviour changed. Accuracy erodes invisibly until a business metric drops. Mature teams watch input distributions, not just outputs, and set drift alarms on the inputs — because input drift shows up before output failure.
The last mile is adoption — and it's the biggest gap
The question to put to any business case isn't 'what's the ROI of change management?' It's 'how much of this project's ROI depends on adoption — and what, exactly, are we doing to earn it?'
— AMDIM Pilot-to-Production Practice
/ The shadow AI signal
Shadow AI is the symptom of a deployment that didn't land.
MIT found that while only ~40% of companies had bought an official AI subscription, employees at over 90% were already using personal AI tools — because the sanctioned ones didn't fit how they actually work. When the official tool doesn't land, people route around it, and your governance, security and value capture walk out the door.
Frequently asked questions
Gartner's 2024 research puts it at approximately 48% — and those that do reach production take on average around eight months. RAND found AI projects fail at more than twice the rate of comparable non-AI IT projects, driven primarily by operating model failures rather than technical ones.
RAND identifies five recurring causes: stakeholders miscommunicating the actual problem, inadequate data, chasing technical sophistication over simplicity, missing production infrastructure, and the problem being harder than current AI can handle reliably. McKinsey adds a sixth: layering AI onto an unchanged workflow — which 80% of organisations do.
A pilot runs in a notebook, is operated manually, produces outputs that go into files, and has no monitoring, no retraining process, and no integration with business systems. A production system runs on an automated pipeline, retrains automatically when performance drifts, integrates natively with operational systems, and is monitored continuously. The gap between the two is the MLOps maturity climb from Level 0 to Level 1.
Prosci's benchmarking data is stark: projects with excellent change management hit their objectives at 88%. Projects with poor or absent change management succeed 13% of the time — a 7× gap driven entirely by how the human side of the transition is managed. The ROI case for change management is not philosophical — it is the single highest-leverage investment available once the technical architecture is sound.
Stuck in pilot purgatory?
The MLOps Maturity Scorecard identifies exactly which layer of your operating model is blocking the climb to production — and gives you a prioritised roadmap in about ten minutes.
Take the MLOps Maturity Scorecard/ go deeper
Put this to work on your actual numbers.
A ten-minute assessment maps exactly where you are today — and what to do first.
/ keep reading
Agentic AI · 11 min
Agentic AI in production: architecture, guardrails, and evaluation
Most teams reaching for agents don't need one. A field guide to agent architecture, guardrails and evals — and the production maths that decide if it ships.
Article · 7 min
Cloud-ready isn't AI-native — and that gap is where pilots die
Most AI pilots stall on infrastructure, not models. Here is the real gap between cloud-capable and AI-native, and how to close it without a rebuild.
Agentic AI · 14 min
The Agentic AI Playbook: Architecture, RAG, Multi-Agent Systems, and What Actually Works in 2026
A practitioner's guide to the full agentic AI stack — from the agent reasoning loop to multi-agent orchestration, RAG pipelines, and the evaluation frameworks that keep it all honest.