AI now powers street hazard detection in San Jose, Calif.

On Monday, San Jose launched phase two of its road safety conditions pilot project, which automates the detection of street hazards using artificial intelligence and is designed to allow crews to respond to road issues more quickly and efficiently.

The program, which launched in 2023 and uses cameras mounted on city vehicles to detect potholes, trash and other hazards, is designed to reduce manual reporting through the city’s 311 system, which receives roughly 300,000 tickets every year, according to Stephen Caines, the city’s chief innovation officer.

Based on the initiative’s early success — it identified potholes with 97% accuracy and debris with 88% accuracy — the city will expand the AI camera system to street sweepers and focus on spotting bike lane and sidewalk problems, such as traffic cones and construction signs.

“The IT director specifically came to me early on in his term and said, Stephen, it shouldn’t be manual. We shouldn’t have to rely on people to tell us what’s wrong with our infrastructure. The city should have a way of proactively identifying our hazards, and then we can deploy our resources much more quickly,” Caines said.

Funded by a $260,000 grant from the Toyota Mobility Foundation, the initiative, Caines said, was designed with strict privacy safeguards, automatically blurring faces, license plates and addresses.

“We’re very proactive with the community. We thought about things like automated license plate readers and going to the communities where they’re deployed so we can hear any issues before they actually go into full deployment has been a very big key to that object detection,” Caines said. “As long as we continue to display the responsibility and commitment to privacy that we have in the past, I think that will continue to preserve our residents trust in our initiatives.”

He said the city has also started start building a database of road safety images and eventually plans to share its AI tools with other governments.

“The idea is that although roads differ greatly across the U.S., you can still have certain similarities between a California pothole in the north and a pothole from Boston, where ice is much more of a condition,” Caines said.

One measure of the pilot’s success, he said, would be a dip in manual reporting with consistent resident satisfaction rates. He said that data is still being collected and that the city is migrating its 311 system from Oracle to Salesforce.

“Once we have this whole transition, we’ll be able to connect everything, and so we’ll be able to actually start measuring some of these results and then seeing that decrease in tickets,” Caines said.