Open gov groups team up on new database of local open data policies

A pair of open government groups are teaming up to post dozens of open data policies from cities around the country in a searchable, machine-readable format, giving local leaders a new resource for understanding how other localities are making their information more accessible.

The Sunlight Foundation and OpenGov Foundation announced their new collaboration on “Open Data Policies Decoded” last week, unveiling a beta version of the new database with policies from 48 cities posted and ready for analysis.

Stephen Larrick, Sunlight’s open data project lead, told StateScoop that he sees the new web portal as addressing one of the great ironies of the open data movement at the local level.

“Open data policies themselves are by and large not released as open legislative data, and there is a lot of opportunity being missed as a result,” Larrick said. “The way ideas spread in the first place is through replication, through tweaking, and that’s a natural process we can take advantage of and connect the good ideas to the people that need them.”

Larrick noted that Sunlight has experience working to open up government data posted in static formats like PDFs — notably, the group runs “scrapers” to pull data from state and federal legislative documents — and as they worked to examine how cities were developing their open data policies, they noticed the same sort of trends when it came time for localities to post those texts.

Seamus Kraft, the OpenGov Foundation’s executive director, said his group has been thinking about similar issues for years now as they’ve worked to “get paper-based legal information into a format we can work with.” His team ended up working on a software called the “State Decoded” to open up state and city statutes, and through “regular sit-downs” with Sunlight, he said the two groups started to think about ways to blend these two efforts.

“We think this project is a wonderful love child between these two organizations,” Larrick said.

Indeed, the new platform blends the technical backbone of OpenGov’s software with Sunlight’s open data expertise and offers a bevy of features for users, Kraft said.

“You can search all across [the policies] all at once, you can directly link to sections,” Kraft said. “You have inline definitions automatically generated to break down that barrier of legalese that exists between not just the public and their law, but many government officials and their law.”

[Read more: A year after a policy update, NYC grapples with ‘the next level’ of open data]

Kraft also points out users can comment on specific parts of the policies, and even download all of the data in bulk formats (a feature he calls “the cherry on top” for prospective developers).

Though Larrick notes that some of the features are “still buggy,” he sees this proving to be an incredible resource for transparency advocates as they work to understand how open data policies actually affect communities.

“We need to use data to evaluate open data if we’re really going to be embodying the ideas we’re evangelizing,” Larrick said. “There’s a limited number of metrics to evaluate open data implementation. Our best practices come from a point of view of activism and pure ivory tower theorizing of what is best, and that’s an important part of the conversation to bring, but being able to test those and those premises like ‘OK, these policies do X, Y and Z’, we can then track that empirically.”

But he also thinks the database could prove hugely beneficial for cities hoping to embrace open data, but that are unsure where to start.

“To be frank, the people who understand open data are not often the city solicitors, or even mayors,” Larrick said. “This gives the idea that much more argumentative ‘oomf,’ to start a conversation at a place where the starting point is given respect or actual consideration.”

Yet Larrick also said the tool can enable more than just the rote copying of open data policies. He hopes it enables a “remix culture.”

He notes that his research shows that 14 different localities followed New York City’s lead, lifting language from the city’s 2012 open data law as they crafted their own policies. That sort of replication makes sense in some respects, but he cautioned that the city’s definition of open data as “non-narrative data” — information easily displayed as statistics — was a bit limiting.

Yet when Montgomery County, Maryland, drafted its own policy, it expanded the definition to data that can be assembled in “a structured format,” which Larrick said was a necessary fix that led the locality to post more valuable information. Nevertheless, he laments that other cities have yet to catch up to that change.

“These policies aren’t liquid, they aren’t easily surfaced, they’re not all in one place, so everyone continues to look at New York City’s,” Larrick said. “Even though the fix has been made in the community of practice, everyone’s using the NYC program, because it’s the largest open data program in the world. Where else are you going to look?”

Accordingly, Larrick hopes the new database will more easily bring best practices to the forefront, helping more people engage with the political process.

“You can search for something like privacy or the phrase ‘chief data officer’ and return the eight cities that use that phrase,” Larrick said. “So the vision is one where policymaking is no longer just the purview of legislators, lawyers and lobbyists, but of anybody who is a subject matter expert or advocate. And these people are empowered with not just access to the policies for mindless replication, but also with data about the context of how these ideas spread, what’s normal versus what’s an outlier.”

Moving forward, Larrick said the groups will work to load more policies into the portal, and he’s optimistic that other cities will start making their open data documents in useful formats as a result of their efforts.

Kraft even sees the tool generating a feedback loop — once cities get a taste of how useful it can be, he hopes they’ll try to pay it forward.

“They can say, ‘Here’s the result, the mayor passed it, and we’re going to contribute that back,’ because that’s open source in a nutshell,” Kraft said. “You take something, add your value, reuse it, build on top of it, then you contribute back what you achieve. That’s how I see this growing.”

More broadly, he thinks that process can even bring long-term changes to a city by “inculcating that open spirit” within the government itself.

“It’s the difference between giving somebody an open fish and teaching them how to be open,” Kraft said.