California is building a ‘portal of portals’ for its data

Data is only useful if people can find it. With that in mind, technology officials in California are developing what they’re calling a “portal of portals,” a search tool that will provide the public a single authoritative source for locating all state-owned data.

The goal is to eliminate the need to search individual government websites with the hope of finding the desired information, said Michael Wilkening, an adviser to Gov. Gavin Newsom on digital services and innovation. If users can’t find what they’re looking for with a single search on the new portal, which is tentatively slated to be launched before Christmas, that will mean the data isn’t available, he said.

“Today we have different entry points for all our different data types,” Wilkening said. “What we’re doing is rethinking that whole concept. What are our entry points?”

Eventually, the state plans to use its existing open data portal as the sole entry point for all of the state’s data, including datasets available to the general public, geospatial information systems data and datasets intended only for use by government or accredited researchers.

“We ultimately want to have an environment where you can do the research somewhere that’s a controlled, secure environment,” Wilkening said.

The project is part of a broader effort by Newsom’s administration to update California’s aging enterprise computer systems, enhance existing digital services and throw new technologies at big challenges, like housing and wildfires. Using a lightweight procurement vehicle called “RFI2,” the state in September awarded two contracts for new firefighting technology, including software that predicts where fires will spread and a fire-detection system that uses aerial sensors to beam warnings down to the state’s computer-aided dispatch systems.

Mapping data underpins those projects and many others in California. Amy Tong, the state’s chief information officer, said she believes interest in GIS data is currently experiencing a resurgence. The California Department of Technology quickly launched a website in October to share information — including maps of fires, air quality and road conditions — with those affected by the power outages and the many wildfires stirred up by heavy winds around the state.

Tong also said GIS data can be used to enhance many other state-government functions, including public-safety policy surrounding the state’s legalization of recreational cannabis and its efforts to protect of natural resources. She said three agencies that are heavy users of GIS data will be featured prominently when the portal-of-portals launches: the California Health and Human Services Agency, the California Natural Resources Agency and the California Environmental Protection Agency.

“We were very purposeful in asking those three agencies to be on this initial launch,” Tong said. “They have some of the best layers that can be showcased on this portal.”

Scott Gregory, the state’s chief technology innovation officer, called the visual tools made possible by GIS data “really exciting.” He said visitors to the new wildfire website found relevant information quickly.

“I’ve been in GIS for some time, but that incident shows the massive impact of what a picture can tell,” Gregory said.

The portal-of-portals is being built using ArcGIS, the ubiquitous mapping platform developed by Esri, and CKAN, an open-source platform commonly used to develop open data portals. The fact that many California state agencies use the same technologies to handle their data made integration into a common search tool much easier, Gregory said, but that doesn’t mean the project is being rushed.

“The way that we’re doing this is a very methodical approach,” he said. “What we’re driving toward is the authoritative source. By understanding the universe of data, we can refine our data offerings to the user.”

Convincing agencies to share new datasets — which might contain sensitive or personal information that must first be scrubbed, or which some officials worry could affect their funding — is difficult, but Wilkening and Tong said California’s development of a new data-sharing policy framework is smoothing the path.

Wilkening, who ran California Health and Human Services before being hired as Newsom’s digital services adviser in April, said the agency used a similar agreement providing guidance on which data should be shared, how it could be shared legally and securely, and who is responsible for each aspect of the sharing process. He said that framework “vastly” reduced the amount of time it took for agencies to share data.

Tong said developing comprehensive written policies eliminates uncertainty and doubt, and can inculcate a data-sharing culture.

“That’s a culture change from a de facto ‘no, it’s hard’ to ‘let’s do it, yes, and go from there,’” she said. “You gotta do the work now or later. Do the legwork up front so the rest of the process is a lot easier.”