Following years of prodding, Virginia opens up corporate data for free

After two years of persistent, pastry-fueled prodding by an open data advocate, Virginia’s State Corporation Commission is making a pair of data sets freely available for download in a common format.

The commission announced last week that, starting Aug. 1, it plans to let people download its data on businesses registered with the state and tax liens placed on businesses that have defaulted on debts. Previously, anyone hoping to download that data had to pay the commission anywhere from $150 to $300 each month and could only access raw data — with the change, the data will be free to all, and available in CSV format, which is commonly used for spreadsheets.

“It’s public information, and there’s been lots of requests for making the [databases] more accessible,” Ken Schrad, the director of the commission’s Division of Information Resources, told StateScoop. “And it’s going to be provided in a format that’s more useful to people. The people who had been subscribing to the service were basically getting raw data and had to be somewhat sophisticated to know how to manage the data or manipulate it for their particular uses.”

The change is particularly validating for Waldo Jaquith, director of the nonprofit U.S. Open Data and a soon-to-be member of the General Services Administration’s 18F development team.

He’s spent more than two years buying the data from the commission, then giving it away for free in a more useful format on a custom website he built, hoping to prevent anyone from paying for information that he believes should always be free. From posting needling messages on social media to even sending a sheet cake to the head of the corporation commission, he’s done his best to publicly prod the agency to change its ways and open up its data.

“This certainly wasn’t a case of Waldo versus the big corporation commission, because ultimately, we’re both pulling toward the same thing, I just had to help them,” Jaquith said. “I do feel like I won, but I also feel like the state corporation commission won.”

Jaquith said he started thinking about the issue of access to corporate data when he met Chris Taggart, the co-founder of the open data site OpenCorporates, at an open government event in January 2014.

“Almost everything we do has some element of a transaction with a company,” Taggart said. “You need to know, does this company exist? If I’m doing business with it, can I trust it?”

So when Jaquith went home and discovered that the commission sold this data, he felt “really angry to see that Virginia sells public records,” and decided to start giving it away. Expecting to be sued by the state, he set to work whipping the data into shape.

He found that the “data was a horrible mess,” and wrote some code to convert it into more useful formats and geocode it, so that users could see where business addresses are located within the state. He made it available to download on a simple web page, told the commission about his plans and waited to hear from the state’s lawyers.

“But instead, they didn’t do anything,” Jaquith said. “So I’m stuck buying the data. I didn’t really want to do that.”

But by the end of 2014, he heard from a tax collector in small Virginia town asking him for help getting access to a list of all the businesses registered within the town’s limits.

“She explained that they only have a list of the businesses who pay their business license tax,” Jaquith said. “If you don’t pay the first time, they don’t know you exist, because the state doesn’t share this data in any way with localities. So it’s an easy way to avoid paying this tax: Just don’t pay it.”

[Read more: Open gov advocates face challenges when publishing state codes]

So, he started breaking up the data by locality, and more municipalities began using the data to audit their own records. Jaquith even agreed to help his city of Charlottesville, Virginia, investigate the issue, and found that 30 percent of the city’s businesses were unlicensed.

Based on his results, he crunched the numbers and calculated that Virginia’s localities were missing out on $100 million annually without access to the data.

“I thought me announcing that would immediately cause the corporation commission to open up this data, but a year and a half went by and they didn’t do anything,” Jaquith said. “But it’s not the state corporation commission’s $100 million, it’s not even the state of Virginia’s, it’s divided up among more than 100 localities. So it’s a lot of money collectively, but it doesn’t make any difference to anybody’s lives at the corporation commission.”

Jaquith was already writing messages to the commission on checks he sent in each month and posting pictures to his thousands of Twitter followers, but resolved to try “a stunt” instead. He worked with a friend in Richmond and got a chocolate sheet cake delivered to the head of the commission to drive his point home.

“The bad news is I’m mocking you, the good news is, I’m giving you cake,” Jaquith said. “I figured they’d put it in the break room, he can’t eat a sheet cake, and dozens or hundreds of employees will hear, ‘Hey, free cake,’ and think, ‘What? We sell our data?’”

Schrad demurred on commenting on his memory of that incident (saying “You can rely on his information for that”), but Jaquith thinks it did have some effect.

He says he started having informal conversations with an IT staffer at the commission, and he soon began getting calls from people who said the commission advised them to contact Jaquith rather than pay for the data.

“The first time I was a little offended,” Jaquith said. “But I also realized, this is a gesture of good faith, them acknowledging, ‘We can’t take your money.’”

Now Jaquith is celebrating the commission’s decision, though he said he’ll continue geocoding the data until the state adds that capability as well. Even still, he’s a bit mystified why it took this long for the commission to make the change.

Schrad notes that they were “aware” of Jaquith’s efforts, but suggested that it took awhile for the agency to do away with its policy of charging for the data, since it originated from when they could only share the data on tapes.

“Keep in mind, we’re still working off a system that was first built in the mid 1980s, so as the commission improves the databases and continues to upgrade its own technology, the capabilities become more possible as result,” Schrad said.

That claim puzzled Jaquith and Taggart, neither of whom believes there’s any connection between technology concerns and releasing the data for free.

“I have never heard any government make such a claim before, probably because the claim is untrue on its face,” Jaquith said. “I’m not sure why he’d say that, unless it’s that the agency just doesn’t want to admit that they changed their minds about something.”

But, ultimately, Taggart did praise Virginia for being part of “the leading pack” of states making this data available, noting that it’s “quite common” that a state might charge for this information. He believes only a dozen currently make this data freely available, and that number “has doubled in the last year.”

“The bottom line here is, Virginia’s move is to be welcomed,” Taggart said. “Hopefully we’ll see more states moving in this direction, since there’s virtually no good reason at all not to do it.”