Generative AI is speeding procurement, but not without risks

Procurement, the process of purchasing products and services from commercial bidders, is a heavily regulated and lengthy affair that sometimes takes months or years to complete. But as cities and states explore uses of generative artificial intelligence, a few, like the State of Idaho and the City of Murray, Utah, are finding early success when applying the rapidly advancing technology.

In February, the Idaho House of Representatives introduced a bill that established the Idaho Artificial Intelligence Advisory Council, a body of legislative members and governor appointees that would monitor and assess AI systems employed by state agencies, with an emphasis on transparency and accountability.

The move opened the door for the state’s Office of Technology Services to pursue incorporating generative AI into the procurement process, which, Idaho Chief Information Officer Alberto Gonzalez told StateScoop in a recent interview, can sometimes take years to finalize.

“Technology moves way too fast for us to be doing a 18 to 24 months procurement policy,” Gonzalez said of adopting generative AI tools. “Now we’re running to keep up with the assessments.”

In a typical procurement, state agencies first identify a need and draft a request for proposal. Vendors submit proposals, and then an agency, usually general services, evaluates the offers and awards a contract to the supplier best suited to the job. Agencies finalize contracts that include compensation, duration and terms and conditions.

“I can come up with an idea today in June and hopefully can sneak it into the next budget cycle which begins in January,” Gonzalez said. “Hopefully they approved to let me have the money in July of the following year. Then, procurement’s going to be like a two to three year process for us to get from idea to implementation. So we thought, we cannot do that — that’s unfair to Idaho.”

‘The bare minimum’

For the past 18 months, Andrea Patrucco, an assistant professor at Florida International University’s business college who specializes in public sector procurement, has been speaking with procurement officials across the country to learn how cities and states are using generative AI. His work is done with support from the National Association of State Chief Information Officers and the National Association of State Procurement Officials, which plan to publish a report on the topic sometime this year.

“This is the technology of the moment, so everyone is talking about the possibility to use generative AI to support procurement,” Patrucco told StateScoop.

Patrucco has found that government agencies are using generative AI in several areas of procurement, including automation, analysis and communication, adopting tools that increase efficiency, reduce menial tasks and accelerate parts of the process from years to months, months to weeks, weeks to days, or days to hours.

“When there are multiple parties involved, this is where the duration of the procurement process becomes massive,” said Patrucco, who added that more complex projects require more communication, which can delay contracts.

Patrucco said he’s collected empirical data from procurement officials but because generative AI is a relatively new, he can’t yet conclude exactly how much time the technology saves. But he did predict that “the time will be absolutely cut down to the bare minimum.”

For instance, in Idaho, for each IT procurement request made, the Governor’s Office of Information Technology Services performs a formal enterprise architectural and security review before giving final approval. In April, the technology department added AI to the security assessment, which, the CIO said, expedited the process.

“We would normally have 6-12 months, and that’s a fast pace, to evaluate a vendor and what they’re doing,” Gonzalez said, “We’ve compressed that time and we’re accelerating assessments.”

Legwork

City agencies, like the recorders office of the Salt Lake City suburb of Murray City, Utah, are also seeing accelerated results with generative AI.

Brooke Smith, a recorder who manages procurements for Murray City, said the first request for proposal she published when she stepped into the role six years ago was to build a new city hall. It took five years to complete.

“There’s a significant amount of legwork that goes in behind each RFP request and getting supplies that you need for your city,” Smith said. “It’s a big project.”

Smith said there was a team of evaluators looking at vendor bids, and that reaching a consensus was tedious and time consuming.

“It was such a big purchase for our city, we had a lot of people that wanted buy-in and [to have] input,” she said. “… Smaller contracts, on a good day, could take three months.”

Smith was tasked last year with drafting an RFP for a 70-foot Christmas tree that needed to be approved within a month. She said that’s when she first reached for ChatGPT for help.

“Had I not had generative AI, we wouldn’t have had that holiday plaza, because there’s no way I would have been able to fulfill that procurement role in the timeframe we needed,” Smith said.

She said her proposal was approved the next day, a vendor selected 10 days later and a contract was signed by the end of the month.

Smith said generative AI tools also help procurement officers communicate better with subject matter experts.

“Generative AI helps you speak specific subject matter expert language,” she said. “And it helps those subject matter experts speak in the procurement language, in understanding the procurement codes that they need to follow.”

Smith said the City of Murray is now using generative AI to update its procurement forms with language that better communicates project expectations and budgets.

‘Gun-shy’

The goal of procurement is to ensure that public funds are spent efficiently, effectively and ethically. However, in addition to the benefits of increased efficiency and a reduction in repetitive tasks, generative AI also poses risks of cybersecurity, bias and compliance.

Procurement, from proposal to contract, involves collecting data on government agencies and private vendors that is vulnerable to cyberattacks. Gonzalez, the Idaho CIO, in April signed a generative AI policy requiring his agency to use a framework that acknowledges bias, transparency and privacy concerns that generative AI can produce.

“As excited as I am about AI, there’s enough people that we know and respect that are telling us to slow down, put in governance, framework, decide how this is going to work out,” he said.

Generative AI can help provide recommendations during the vendor selection process, but these suggestions may show bias against historically marginalized suppliers, such as minority-owned or other small businesses.

“If you’re asking [AI] for something general, are you eliminating vendors that don’t meet the cookie-cutter vendor,” said Smith, the Murray recorder. “I think we need to train our procurement professionals to look for the biases, look for miscommunication or errors or hallucinations.”

The Utah Department of Commerce this month announced the official opening of an Office of Artificial Intelligence Policy to explore AI regulation, assess how policies can clear regulatory burdens for AI companies and to protect the public from potential harms.

Smith said the City of Murray is also forming an AI committee, to “make sure [the city] is educating our employees who may or may not currently be using AI on how to move forward and and reduce those risks.”

Patrucco, the assistant professor, pointed out that, additionally, a lack of cohesion between local, state and federal generative AI policies can lead to compliance issues.

“You have cities within states, and a state government and a federal government,” Patrucco said. “The main issue right now is that the different levels [of government] must establish a clear, shared framework for the use of their AI systems.”

Gonzalez said not every agency in Idaho is raring to use generative AI in procurement for fear it might not meet federal requirements in an audit.

“They’re gun-shy to move into a different direction with some of these services that are being made available to them,” Gonzalez said. “… I think the [lack] of federal governance is keeping some of our agencies from really grabbing some of these AI solutions.”