Report: Cross-agency data sharing fraught with challenges, rewards

State government leaders have been talking about “knocking down silos” and sharing data for years — and those same state leaders have a new resource to meet that goal in the form of a paper published Monday called Data Sharing Across State Agencies.

The paper, written by Gabriella Lively, senior data analyst of CapTech, a technology consulting firm, outlines the challenges facing agencies interested in developing data sharing programs and recommendations for solving those challenges.

Recent History

Recent numbers show that state IT is taking data silos seriously. This year’s National Association of State Chief Information Officers (NASCIO) survey revealed that CIOs rank “Business Intelligence and Data Analytics” as their fourth largest priority for the coming year. “Consolidation/Optimization,” an area that can enable data sharing, ranked third.

State lawmakers are also engaging in data integration. Since 2015, 42 states have introduced 318 bills specific to data sharing. The federal government is also engaged, with $42 million in grants going to state-level data sharing initiatives since 2013.

Governors and state officials in New York, South Carolina, Indiana, Illinois, Michigan, Missouri and Arkansas are talking about data sharing, too. This year, Virginia Gov. Terry McAuliffe encouraged state CIOs to treat his state’s data policies and best practices as a beacon to follow.

In her paper, Lively explained that data sharing is important because it generates better outcomes for the citizens consuming state services, equips policy makers with more accurate and newer information and reduces costs for government. But these things are easier said than done.

Challenges

Before interagency data-sharing can occur, agencies first must overcome barriers such as legislature, privacy concerns, and technical issues, the report states.

The data needed to improve policy and decision making is usually spread across systems managed by different state agencies. This makes it nearly impossible for analysts to access the data needed to generate a clear picture of their constituents issues. These government silos have left agencies unable to identify individual constituents across their systems. Lively provides the example of Medicaid providers who aren’t informed that an enrollee has recently gone to a state-run mental health facility or called a state-run crisis line.

Lively said that even if agencies did share this data, there are still challenges posed by duplicate data. For example, in one system a constituent’s name might be J. Smith but in another it could be John Smith, or one agency might use a term like “Caucasian” while another uses “white.” These kinds of inconsistencies making it extremely difficult to generate accurate reports and analytics.

Drafting legislature for these new policies may also be difficult, the report found. However, states like Illinois have passed bills allowing for the creation of open data platforms and regulatory architecture.

Privacy is another issue facing data sharing. Lively said one way to decrease the risk of violating privacy restrictions could be removing names, age, race, or other person information from the data they share with other agencies. However, this may not always be possible as this information is sometimes necessary for data sharing.

A large part of addressing privacy concerns is “establishing policies,” Lively explained. This includes staff training, drafting policies for gaining access to data such as completing training and ensuring the data is governed by administrators.

Solutions

Lively’s report includes a list of solutions for the challenges presented to data sharing. Before data integration occurs, she said, states should define outcomes they hope to achieve. Part of this process includes determining the estimated return on investment and the public’s opinion of programs in order to justify costs and efforts.  

It is important to gauge public opinion because data sharing could lead public outcry over privacy rights, Lively said. For example, new data sharing efforts could create conflict with existing privacy measures like those included in the Health Insurance Portability and Accountability Act (HIPAA) or the Code of Federal Regulations (CFR).

To avoid privacy conflicts like these, Lively suggested the creation of a “data governance board” and a “data stewardship” position, as a way to establish and enforce data sharing policies and regulations. A data governance board would need to be implemented at the state and agency level to set standards, policies, establish procedures for data sharing, ensure initiatives deliver returns on investment, lower costs, minimize fraud, provide better information on allocations of funds and address privacy concerns.

The next step, she said, is to select technologies that can implement the chosen initiatives. For example, an extract, transformation and load tool, also known as ETL, can help de-duplicate the aforementioned messy data and enable accurate analytics.

States should start off with two or three smaller projects and work their way up to bigger or more projects, the report recommended. By doing this, they can demonstrate the importance of data sharing and generate a proof of concept in their work. Data sharing transitions will take time, she said, but they will “ultimately lead to improved services for constituents and more-informed allocation of funding.”

Despite rising interest and activity in the data sharing arena, Lively noted that these state projects are mostly focused on criminal justice and public health, and that there is still much work to be done.

The ten-page paper — Data Sharing Across State Agencies — can be downloaded free from CapTech.