Continue to

Big data storage continues evolution in state, local government

Sponsored Content: Culture, volume and expectations are driving the continued evolution of the data storage landscape, but state and local leaders can find a solution.

StateScoop Staff
StateScoop Staff

The way state and local governments are thinking about data storage is changing.

Shaun Bierweiler, vice president of public sector for Hortonworks, says in an interview with FedScoop and StateScoop Radio that state and local governments are evolving that storage mentality due to culture change, volume of data and expectations around data.

“We like to say that every agency is a data agency, and that stems from the evolution and the significance that data has taken in the lives and in the missions of our customers," Bierweiler says. 

In the past, data in traditional data warehouses was used in a transactional way — but now, it's at the center of every decision, Bierweiler explains.

The structure of big data itself has has evolved, he says. 

“Previously, you knew what was going in and what was coming out," Bierweiler says in the interview. "Today, now you have data from an infinite number of sources. You have images, you have videos, you have data encrypted within those items. The data itself has become very much more complex in terms of structure.”

The volume of data, however, is the biggest change.

“Agencies are drowning in data because there’s so much of it,” Bierweiler says. “You have to be able to store it, you have to be able to process it. You have to be able to extrapolate the value from that data. And so that’s become much more complicated and complex.”

Expectations for the use of that data has changed drastically as well, Bierweiler explains.

“Not only do you have more data that has more information that varies much more greatly, but now users expect to do more with it," Bierweiler says. "And they not only expect to do valuable things with their data, but they expect to extrapolate information and sharing data from other users’ data. What used to be very traditionally stove-piped and siloed now is a mesh of data that’s expected to be shared.”

With such an array of data types, sizes and uses, Bierweiler advocates for state and local agencies to embrace enterprise open source platforms to address users’ many needs.

“If you look at a traditional proprietary technology, the lifecycle for them tends to be much longer, and the development cycle even longer,” Bierweiler says. “When you get a new release of a proprietary solution, it’s often with very old or antiquated solutions and it’s solving the problems that existed when the technology’s development model started.”  

An enterprise open source platform like Hortonworks harnesses “the development model of community — people that aren’t paid by Hortonworks. What you get then is a very open solution that not only solves what people are trying to address today, but problems they foresee for tomorrow,” Bierweiler tells FedScoop and StateScoop Radio.

Because there aren’t barriers or proprietary interfaces on open source platforms, “it lends itself to a true best-of-breed solution.” 

“Consider everything as possible,” he recommends to agencies and offices considering open source. “It’s often difficult to make that cultural shift from something that you’ve always done and you convince yourself that that’s the only way. Technology has come a very long way and there are creative ways to do things better, cheaper, faster, smarter. So oftentimes, the biggest challenge we have is not a technical hurdle — it’s a cultural shift.” 

See more about how Hortonworks’ open source solutions can help you manage your data.

This edition of StateScoop and FedScoop Radio was brought to you by Hortonworks. Learn more about how Hortonworks' open source solutions can help you manage your data.

Join the Conversation

Related Articles