States move to formalize AI governance as adoption expands, report finds

As state agencies continue experimenting with generative artificial intelligence tools, a growing number are building formal governance frameworks to manage the technology’s risks and opportunities, according to a report published last month by the UC Berkeley School of Information.

Eric Hysen, a former chief information officer of the Department of Homeland Security, argues in the report that AI governance is quickly becoming a core part of states’ IT operations, rather than a standalone policy exercise. The report found that 43 states have established some AI governance, but with wide variations in scope, structure and transparency.

The report includes a playbook for governments deploying AI systems, with best practices like creating governance councils and establishing clear rules around human oversight and data privacy. The playbook recommends integrating AI governance directly into procurement, cybersecurity and operational workflows, instead of treating it as a separate compliance process. It also calls for agencies to establish clear accountability structures and continuously monitor AI systems after deployment.

The report also stresses the importance of agencies conducting risk assessments to prevent AI-powered cyberattacks and attacks targeting AI systems. It recommends routinely scanning for failures in AI design and implementation that can mistakenly cause harm or bias.

“These risks start with the challenges inherent to managing any information technology system (such as cybersecurity and system reliability) and expand to include unique AI impacts,” the report reads. “When governments do not effectively manage these risks in their own uses of AI, the impacts on constituents’ lives can be severe.”

Amid limited federal regulation, several states have launched AI pilot programs and developed governance structures that mirror the report’s recommendations.

Pennsylvania launched one of the nation’s first statewide generative AI pilots after Gov. Josh Shapiro’s 2023 executive order directed agencies to responsibly explore the technology’s use cases. The state’s pilot project included guardrails prohibiting employees from entering sensitive or personally identifiable information into AI systems.

California recently launched a statewide rollout of Engaged California, a public participation platform designed to gather resident feedback on how AI is affecting workers, government services and the broader economy across the state. The feedback will help guide decisions by state officials, according to the website.

Colorado and North Carolina have focused heavily on security and data governance as agencies test AI applications, before deploying the tools at scale.

While many of these efforts remain focused on pilot programs and internal governance planning, the report suggests these early frameworks could shape how states adopt AI technologies long-term.

“AI’s capabilities and risk landscape are constantly evolving,” the playbook reads. “AI risk management must address not only extreme misuse scenarios but also the subtle, cumulative ways in which AI systems can shape human decision-making, wellbeing, and trust.”