Florida tries (again) to fix ‘absolute unmitigated tire fire’ that is state IT

The Florida Senate is considering a bill that would — for at least the fifth time in recent years — yet again reorganize the state’s technology bureau. And despite the enormity of the state’s IT problems, some current and former state officials are saying that unlike past efforts, this one might stick.

The 111-page bill would attempt to solve many of Florida’s longstanding organizational snags, including politicization of the technology office and its leader, the state chief information officer. It would absorb the functions of the current incarnation of the state IT agency, the Florida Digital Service, and form a centralized outpost for oversight and governance, like approving major technology procurements before agencies spend tens of millions of dollars on projects that in some cases drags out for decades.

And with the new agency proposed to receive $177 million in general funding, it might also wind up better staffed and more stable than past reorganization efforts. When Florida Sen. Gayle Harrell, a Republican, introduced the bill to committee last week, she sounded ebullient about the prospect of the state’s technological dysfunction being remediated.

“This is a consensus that is coming together that truly is going to transform how we do information technology, how we interoperate,” Harrell said during a presentation of the bill. “… We are going to have one system, enterprisewide, that is going to be interoperable. The agencies are going to have independence, but with standards across the enterprise.”

The bill would create the Agency for State Systems and Enterprise Technology, or ASSET. But the changes the reorganization would bring wouldn’t happen overnight, Harrell admitted, noting that the majority of its operations wouldn’t begin until July 2026 and that it likely wouldn’t be fully operational until 2028.

But the slower cadence could be cause for optimism. Some sources told StateScoop that the quick fixes thrown together in past reorganizations were part of why they failed, as the core problems plaguing the state’s IT — like its massive procurements, lack of centralized governance and failure to consider interoperability — weren’t addressed.

Harrell recalled joining the legislature’s first technology committee in 2001 and creating Florida’s original State Technology Office. Over the next two-plus decades, the division was the repeated casualty of political tugs-of-war as governors and lawmakers disagreed on where the office should sit and what sort of authority it should be allowed to have.

All the while, IT contracts to replace major systems, some totaling in the hundreds of millions of dollars, plodded along with no apparent end. Florida’s new Medicaid information system, called Florida Health Care Connections, or FX, is expected to cost the state more than $1 billion.

Florida’s IT agency was moved and renamed, again and again, each time given a new acronym and leadership structure, until in 2020, then- state Rep. James Grant, a Republican, reluctantly ended his re-election campaign and agreed to head the latest iteration of the agency, the Florida Digital Service, as its CIO. In an interview, he recalled to StateScoop his initial reaction when the governor’s office asked him to head the agency: “‘I’m not doing that. Hell no.’”

‘Gross, offensive stuff’

Though the legislation doesn’t yet have a companion bill in the House, it seems to be well supported in the Senate, including by its Republican president, Sen. Ben Albritton. The legislation would also need to clear the desk of Gov. Ron DeSantis, a potential challenge given one key change that would move the technology office out of the governor’s office and place it under the authority of the governor and the Cabinet.

“Over the years, we’ve had iterations of trying to put together a systemwide approach to our IT, to establish, standards, architecture, procurement. And you know what?” Harrell said during last Thursday’s hearing. “It hasn’t worked. It has. Not. Worked.”

Grant, who served as state CIO until the summer of 2023, turned out to be right about not wanting the job. He called it “the most hellacious three years of my career, and it’s not even close.” On his LinkedIn profile, Grant describes his time in Florida’s executive branch as “a three year bar fight with bureaucracy.” 

With StateSoop, he shared numerous challenges he believes the bills must solve, citing a rotating door of “gross, offensive stuff” he observed during his tenure as CIO. The “gross stuff” was what he now publicly calls out as “fraud, waste and abuse” and, sometimes, “outright corruption.” But just how much of those things are there in Florida?

“Infinity? Eleventy-trillion? I don’t even know how to quantify it,” Grant said. “A single unelected House staffer could write proviso language to require [the Florida Department of Management Services] to renew an almost $20 million Microsoft purchase.”

Grant said such “harmless-looking language” regularly slides through the legislature because lawmakers are termed out of office by the time they grasp just how vendors and agency leaders have been hoodwinking them.

“They get the runaround from the agencies. They get the runaround from the industry. They’re just getting a runaround no matter how you cut it,” he said of Florida’s legislators. “Now I realize that the wrong person was at the podium. Now I realize what I didn’t know to ask. I could dive in on technical stuff. I could dive in on policy stuff. But I failed to understand that the legislature is akin to a board of directors, and the actual operation of government is at the agency level.”

Grant, who spent roughly a decade in Florida’s legislature, cited a technique used by some agency heads of entering into a procurement right before the legislative session to legally avoid answering questions about their over-budget or otherwise troubled IT projects.

‘Who’s actually responsible?’

And Florida has had many troubled IT projects over the years. The state abandoned an overhaul of its financial management system in 2007 after dumping $89 million into the project. Florida’s aged unemployment insurance system repeatedly crashed during the COVID-19 pandemic. Florida has had continued issues with its electronic toll collection system, SunPass, including bugs that led to the state overbilling drivers, losing the state $50 million in revenue, to say nothing of the project’s development itself.

In October 2020, the pop star and actress Ariana Grande urged her fans in Florida to vote in the presidential race between Joe Biden and Donald Trump, crashing the state’s voter registration website. Grant, six weeks into his new CIO job, was ordered by the governor to fix the problem immediately. But, he said, he didn’t have any access to the application and he didn’t have credentials to access the floor of the data center where it was housed. And it was a colocation data center, which meant the data center’s operators couldn’t give him the access he needed.

“Who’s actually responsible for that application, network and infrastructure? In Florida, the answer is almost always lots of faceless people,” Grant said. “And then the state CIO has to sit at a podium and answer all the questions.”

Grant said Florida IT’s dysfunction is a product of the state’s “stakeholder islands” — agencies, politicians and lobbyists each with competing incentives and varying blind spots. He said some people mean well, but others cynically capitalize on disorganization and misunderstandings.

He recalled an episode in which he’d angered an industry lobbyist by opting to use the state’s pre-negotiated cooperative purchasing agreements, rather than pursue a large IT procurement. Such agreements, like one offered by the nonprofit National Association of State Procurement Officials, are common in state governments and favored by some CIOs for their lower costs and faster speed. Grant said the lobbyist began complaining to the legislature that he’d broken procurement law. Grant said the lobbyist found credulous ears and he “lost a lot of traction.”

‘Tire fire’

But those large, protracted projects are at the root of the state’s technological woes.

“Everybody is sick and tired of projects that are running 25% over timeline, 25% over budget, oftentimes never going live,” Grant said. “They’re just these time and material boondoggles. They’re all the exact same playbook. Some big [service integrator] wins a deal, it fails and then they get blacklisted, the other SI shows up and says ‘We’re the good guys that’ll do it.’ They get the next one and the same thing happens over and over and over again.”

The proposed reorganization is designed to fix the broken cycle, to put a CIO at the center of the state’s IT organizational chart, someone who can spot problems before they spiral out of control. Harrell said agencies will retain “independence,” but be held accountable to standards set by the new agency.

Grant said his power to influence outcomes as CIO was extremely limited.

“What I didn’t know was how much the agencies could obstruct the state CIO,” he said. “They could hide applications, they could not report metadata to a data catalog that was statutorily required and they could just sit back and let the lack of understanding get the state CIO in trouble. I had no idea [the Department of Management Services] could steal $1.8 million of my rate and prevent me from being able to hire without ever seeing it. I had no idea DMS could underfund a data center that’s federally funded by 20% by screwing up the math. When I say absolute tire fire — absolute unmitigated tire fire.”

Grant recalled another incident in which the governor’s office asked him to spend $8 million he didn’t have. Confused, Grant looked into the request and discovered Florida’s budgeting system didn’t account for the money he’d already committed to spending, which meant the office didn’t know how much money he had to spend.

“I tell people that if the Kremlin or [Chinese Communist Party] tried to develop an org chart and structure to sabotage technology in Florida, this would be too extreme,” he said. “They’d be like, ‘Man, they’ll never fall for that.’”