Bridging the AI scalability gap: From experimentation to enterprise impact

Government agencies have historically been slow adopters of new technology and with good reason: Much is at stake in managing limited taxpayer resources and protecting public trust. It’s not surprising, then, that agency leaders are taking a cautious approach to adopting artificial intelligence, arguably the most disruptive technology in decades.

However, that cautious approach presents a different set of risks with potentially costly consequences. While agencies carefully evaluate AI through repeated pilots and proofs of concept, there’s a growing opportunity cost: Citizens continue to wait longer for services, staff remain constrained by manual processes, and investments in promising pilots may not translate to scaled impact.

Across state and local governments, we see a flurry of what appears to be productive activity: pilots, proofs of concept (POCs), and small-scale experiments. While these are necessary first steps to understand how AI models work and what they can do, they frequently don’t go far enough. To truly capture the value of AI, agency leaders must bridge the gap between experimentation — “testing the tech” — and operationalizing AI as a core enterprise capability.

The challenge of scaling purpose-built pilots

Despite this promising activity, a common challenge emerges. The difficulty in moving from a successful pilot to full-scale production usually stems from two fundamental issues:

First, many pilots are intentionally designed as isolated, controlled experiments; they are not purpose-built for long-term scalability or integration with existing systems.

Second, and perhaps more critically, these experiments often focus solely on the model’s performance while ignoring the essential elements of production: data governance, security, and operational ownership.

In the complex environment of state and local government, data is frequently fragmented and trapped within legacy systems. A model might perform brilliantly in a controlled pilot using a clean dataset, but it will likely fail when scaled across departments or integrated with a 20-year-old application. Successful agencies recognize early on that “operationalizing” AI requires more than a good algorithm — it requires data readiness, good governance, and a robust architectural plan.

AI as a mission-driven transformation

The agencies that successfully make the leap from pilot to production share a common trait: They view AI as a mission-driven transformation effort rather than a technical curiosity. They tie AI initiatives to clear, measurable operational outcomes, whether accelerating benefit processing, reducing fraud, or improving citizen service delivery.

When AI is connected to the mission, it becomes much easier to secure the leadership support and funding necessary for a robust, long-term deployment. One state agency we worked with moved from a 6-month pilot to full production in under a year by establishing cross-functional governance from day one and tying their AI initiative to a specific metric: reducing benefit application processing from 45 days to 15 days. By connecting AI to a measurable mission outcome, they secured both funding and organizational buy-in.

These leaders establish cross-functional governance from day one. AI isn’t just an IT project; it involves legal, privacy, cybersecurity, and program operations — disciplines that are frequently at the margins of pilot projects. By bringing these stakeholders together at the start, agencies can avoid the late-stage delays and setbacks that often kill a project just as it’s ready to scale.

The three dimensions of readiness

A common question we encounter is, how do you know when an AI solution is truly ready for the “go” button? We’ve developed a practical framework that examines three critical dimensions. Leaders should assess each using specific, measurable criteria:

Technical performance: The model must meet defined metrics for accuracy and reliability while demonstrating the ability to scale within the agency’s existing infrastructure. It must be cleanly integrated with enterprise systems via APIs or middleware rather than operating as a standalone tool.

Ask: Can this system handle 10x the pilot volume without degradation? Does it integrate with our existing technology stack?

Operational sustainability: You must identify who will own and support the solution once it’s live. This includes monitoring for “model drift” and having a clear plan for how the AI will be updated over time.

Ask: Who owns this after go-live? What’s our plan for monitoring and updating as needed?

Governance consistency: The solution must align with security standards, data policies, and responsible AI practices. Transparency is non-negotiable; the system must be auditable, and bias monitoring must be built into the deployment.

Ask: Can we audit every decision this system makes? Have we tested for a variety of biases, such as human bias, data bias and outcome bias?  Agencies must not only “test for bias” once, but also establish continuous monitoring with clear fairness metrics, regular audits and stakeholder feedback loops.

Empowering the workforce

Perhaps the biggest obstacle to AI adoption and implementation is the “human factor.” There’s an ongoing concern that AI will replace jobs. Leadership must directly address this by explaining that AI is meant to enhance agility and productivity, not replace people.

The best way to foster support for adoption is to provide the workforce with the tools and training they need to use AI in their day-to-day jobs. When staff experience how AI helps them personally become more productive, they quickly grasp how it can transform the entire agency. One effective approach: Start with “AI assist” tools that handle routine inquiries or document processing, freeing staff to focus on complex tasks requiring human judgment. This demonstrates value quickly while building workforce confidence and competence.

By involving your workforce in automating routine manual tasks and seeing AI in action, rather than in a pilot, agencies can accelerate productive outcomes and deliver services to citizens faster.

The role of enterprise architecture

There’s another essential factor for successfully moving from experimentation to operationalizing AI that agencies need to consider enterprise architecture. Without a unifying foundation, AI can quickly fragment into disconnected, one-off solutions that deliver short-term value but fail to transform the organization. The difference lies in how architecture is used.

Incorporating architecture often means retrofitting structure after the fact, bolting standards or integrations onto already siloed AI initiatives. This reactive approach limits scalability and reinforces fragmentation.

Embracing enterprise architecture, by contrast, means designing for integration from the outset. It establishes a deliberate framework that connects AI across platforms, data ecosystems, and security environments. With this foundation, agencies can build shared, reusable capabilities such as standardized data pipelines, model hosting, and governance layers that multiple programs can leverage consistently.

Rather than attempting costly rip-and-replace strategies, an architectural mindset enables agencies to layer AI capabilities onto existing systems through modern data platforms. The result is not just incremental improvement, but a resilient, scalable foundation, one that turns AI from a collection of isolated wins into a continuous engine for innovation.

The path forward

State and local agencies face increasing pressure to translate AI experimentation into operational reality, delivering improved services to citizens and more efficient processes for the workforce.

The agencies positioned to lead in the next decade are making this transition now from treating AI as a technical experiment to embracing it as a fundamental enterprise capability. They’re building the necessary architectural foundation, preparing their workforce, establishing governance frameworks, and most importantly, connecting every AI initiative to measurable mission outcomes.

The transition from pilot to production isn’t about technology; it’s about committing to a more agile, responsive government that meets the needs of 21st-century constituents.

The question is no longer whether AI works in government. It’s whether your agency has the strategy, architecture, and leadership commitment to make it work at scale.

___________________________________________________________________________

Request your consultation today to evaluate your current AI technical and operational readiness, identify the highest-impact opportunities for scaling your AI investments, and develop a practical roadmap from experimentation to operationalization.

Beth Howen is president of state, local and education at NTT DATA North America.  She has over 30 years of leadership experience in the technology sector, having held executive roles at Capgemini, Telus International, Atos and the City of Indianapolis.