The data storage landscape is continually changing and in the background, there are a few shifts driving that evolution.

One of those is culture, Shaun Bierweiler, vice president of public sector for Hortonworks, says in an interview with FedScoop Radio. We like to say that every agency is a data agency, and that stems from the evolution and the significance that data has taken in the lives and in the missions of our customers.

With traditional data warehouses in the past, Bierweiler explains, data was used in a very transactional way. But now its at the center of every decision, he says.

To start with, the structure of big data has evolved. Previously, you knew what was going in and what was coming out. Today, now you have data from an infinite number of sources. You have images, you have videos, you have data encrypted within those items, Bierweiler says in the interview. The data itself has become very much more complex in terms of structure.

The volume is, perhaps, the biggest change.

Agencies are drowning in data because theres so much of it, he says. You have to be able to store it, you have to be able to process it. You have to be able to extrapolate the value from that data. And so thats become much more complicated and complex.

Finally, to top that all off, expectations for the use of that data has changed drastically, Bierweiler explains.

Not only do you have more data that has more information that varies much more greatly, but now users expect to do more with it. And they not only expect to do valuable things with their data, but they expect to extrapolate information and sharing data from other users data. What used to be very traditionally stove-piped and siloed now is a mesh of data thats expected to be shared.

With such an array of data types, sizes and uses, Bierweiler advocates for enterprise open source platforms to address users many needs.

If you look at a traditional proprietary technology, the lifecycle for them tends to be much longer, and the development cycle even longer, Bierweiler says. When you get a new release of a proprietary solution, its often with very old or antiquated solutions and its solving the problems that existed when the technologys development model started.

Youre also locked-in to the vendors roadmap, he says.

An enterprise open source platform like Hortonworks harnesses the development model of community people that arent paid by Hortonworks. What you get then is a very open solution that not only solves what people are trying to address today, but problems they foresee for tomorrow, Bierweiler tells FedScoop Radio. And because there arent barriers or proprietary interfaces, it lends itself to a true best-of-breed solution.

Consider everything as possible, he recommends to agencies and offices considering open source. Its often difficult to make that cultural shift from something that youve always done and you convince yourself that thats the only way. Technology has come a very long way and there are creative ways to do things better, cheaper, faster, smarter. So oftentimes, the biggest challenge we have is not a technical hurdle its a cultural shift.

See more about how Hortonworks open source solutionscan help you manage your data.

Go here to read the rest:

The rapid evolution of big data storage - FedScoop