Stretched thin, San Jose, Calif., reaches for generative AI

Many view the city government of San Jose, Calif., as an excellent career “launchpad,” said Stephen Caines, the city’s chief innovation officer, but he and others in City Hall want to change that.

For at least a decade, San Jose’s leaders have prided themselves on running one of the leanest operations among U.S. local governments, a designation remarkable particularly because of the city’s size and influence. Today the Silicon Valley hub consists of about 9,000 staff who provide services — everything from issuing parking permits to extinguishing fires — to a greater metropolitan population of nearly 2 million residents.

If managing (with varying degrees of success) to serve so many with so few has instilled pride, it’s also generated fatigue. Caines said many staff move on to nearby jobs that provide similar pay but that require less of them. Providing powerful new technologies to ease workloads is a key part of the city’s strategy for retaining more of its personnel. To that end, the city on Friday released a request for proposal for a generative artificial intelligence platform, in hopes of providing its staff with a tool they can use to create their own digital helpers.

San Jose Mayor Matt Mahan, who this summer was highlighted by the local press for his use of large language models in daily work, including directing his aides to use ChatGPT to help write his speeches, said in a press release that the city is “working to integrate generative AI into everything we do at City Hall because you deserve government that is fast, responsive, personalized and cost-effective.”

According to the city, the new software the city seeks will be able to help staff with a broad range of repetitive tasks, like analyzing and summarizing policy documents, reviewing computer code and generating charts. 

“The platform should provide secure access to conversational AI tools, have the ability to connect with internal systems and data, and include enterprise-level administrative and privacy controls,” the city’s release reads. “The solution must also support low-code and no-code features so employees with different levels of technical experience can use it effectively.”

San Jose has already begun offering its staff two basic training courses, for the use of generative AI and data analysis. Caines said about 400 staff have taken at least one of the hour-long IT Training Academy courses so far, progress toward the city’s goal of training 1,000 staff in the ways of LLMs by the end of the year.

“Our constant challenge is how do we retain great workers and in our opinion the way to do that is to give them tools to allow them to have a better work-life balance by making their workload more manageable,” Caines said.

According to the city’s website, some staff have used what they learned at the academy to develop customized AI assistants to assist with tasks like taking meeting notes and writing policy drafts, “freeing up time to focus on helping our community.”

The technical training isn’t only for technical roles, but for anyone who works behind a computer, Caines said. A platform that would allow many different types of city staff to create assistants could be useful given the city’s deficits. Caines pointed to the city’s police department, which has more than 100 vacancies and must cover 182 square miles.

“The result of that is we are not always hitting our priority 1 and 2 calls,” he said, referencing incidents that can include life-threatening or serious injuries. “We can’t be everywhere all at once.”

Part of the city’s public-safety response has been to install automatic license plate readers, which aren’t loved by privacy and civil rights advocates, and are limited besides. ALPR cameras can generate leads and help the city better cover its broad geography, but officers are still needed to do the follow-up police work.

LLMs and other new software could help reduce the paperwork burden police face. At least one California law enforcement office has started automating document redaction, and a major police technology firm has begun offering a body-camera transcription service that some departments are using to generate incident reports. The American Civil Liberties Union is worried that the latter could introduce new biases and provide cover for corrupt officers.

San Jose isn’t only looking at tech for police, but for a wide range of jobs. The RFP is hoped to lead to a purchase that will make staff feel they aren’t being stretched quite so far.

“We recognize that technology moves fast,” Caines said. “We recognize that the field of vendors who provide these LLMs has grown rapidly, almost exponentially, in the last few months and we want to make sure we’re casting the broadest net to find what works for San Jose.”