Congress leads state legislatures in open data efforts — panel

Although Congress is still struggling with open data, it remains far ahead of state and local legislatures when it comes to releasing data in machine readable and downloadable formats, according to a panel of experts and Congressional staffers.

At a discussion hosted by the nonprofit Center for Data Innovation Tuesday, open data advocates lamented that state and local lawmakers have so far been unable to match the progress that legislators and staffers at the federal level have made in the area.

“Right now, there’s almost no trickle down effect at the state level, outside of a couple of bright spots,” said Tim Hwang, CEO of the legislative analytics company FiscalNote. “It tends to be the states with more resources that are making more data open. It’s no surprise that New York and California are leading the way.”

Hwang said that should be a troubling trend for people interested in making government more transparent, especially since “Congressional gridlock is pushing many of these controversial [regulatory and legislative] issues to the states at a really high velocity.” 

He noted that his company, which specializes in collecting and analyzing legislative data from both Congress and state legislatures, is able to ingest information posted online as static PDFs only thanks to advances in technology.

“Technologies used to extract data from state and local levels are extremely complex, because you need an algorithm to scrape them, but making things searchable requires pulling text out of an image or PDF,” Hwang said. “The technology has improved, but it’s a significant investment.”

Increasingly, that’s not a problem at the federal level, according to Daniel Schuman, policy director at the online open government advocacy group Demand Progress.

“I see it as a bit of a trickle up effect,” Schuman said. “People bring their experiences at the state level to Congress with them, and want more open data…That impulse travels up to the House, then over from the House to the Senate.”

[Read more: D.C. leaders urge civic hackers to throw support behind open data legislation]

However, Congress is not without its flaws when it comes to posting open legislative data, the panelists agreed. 

Steve Dwyer, senior policy advisor and digital director for House Minority Whip Rep. Steny Hoyer, D.-Md., pointed out that Congress has only recently moved away from a system for sharing legislative data with staffers developed by a civic hacker to a more standardized online portal at Congress.gov.

“The entire institution is behind it, it’s no longer just something built by one person,” Dwyer said. “But there’s still so many different legislative agencies, they have so many different bad, outdated systems and data, and they want to update them and open them.”

Dwyer used the controversy on the House floor last week over an amendment to a bill that would’ve barred discrimination against LGBT employees by federal contractors as a perfect example of how more advanced open data practices could make Congress more transparent. 

Though the measure was on the verge of narrowly passing, seven Republican lawmakers switched their votes at the last minute. Dwyer and his staff were on the floor trying to identify which legislators changed their minds, but could only do so by printing vote tallies from a bank of antiquated computers that aren’t even connected to the internet.

“This is real-time information coming from the floor, and you would’ve thought it was already public, but that data stream goes to C-SPAN and nowhere else,” Dwyer said. “That’s a perfect test case for why we need more transparency.”

Jessica Seale, digital director for Sen. John Cornyn, R.-Texas, said she’s observed similarly analogue techniques on the Senate floor stymie open data efforts. If a lawmaker introduces an amendment to a bill in a committee session, then she said a great deal of the paperwork for that process is written out by hand and then scanned later into Congress’ system.

“A lot of times, we’ll have amendments scanned in with sticky notes on the front, and that’s not only not searchable, but if something happens in the manual scanning process, that could be an issue,” Seale said. “That’s a huge issue with these budget bills that have 40 or 50 amendments pending.”

Schuman also expressed frustration that Congressional Research Service reports still aren’t posted online for public consumption. He’s championing a bill introduced in March that would change that, and he charged that there’s “no good argument for not making them available.”

Currently, CRS reports are only supposed to be available to lawmakers and their staffs — although they are allowed to pass them on to their constituents. In practice, they can be obtained by lobbyists or others with connections in Congress, but they are not available to the public except when obtained and posted by government transparency advocates. 

“If you have money it’s not that hard to get them, but for anyone else, it’s much more difficult,” Schuman said. “You’re entitled to your opinion, but the underlying facts shouldn’t be in dispute.”

Seale also wants to see the reports become available, but she cautioned that releasing them immediately could cause lobbyists and concerned citizens to deluge CRS staffers with questions about them, when they’re already overwhelmed with work.

“There’s a lot of support for open data, but not a lot of support for money to expand Congressional staff budgets, and those two things are in conflict,” Seale said.

Contact the reporter at alex.koma@statescoop.com, and follow him on Twitter @AlexKomaSNG.