Snowflake CEO Slootman: ‘Is it the end of the road for Hadoop? Well what do you think?’ – www.computing.co.uk

The Hadoop elephant in happier times

Five years ago, Hadoop was touted as the end of data warehouses as we knew them. Now, one of the three top players, MapR, has gone bust, its assets sold to HPE, and the other two, Hortonworks and Cloudera, have merged under the Cloudera name.

Asked whether it was the end of the road for Hadoop, Snowflake CEO Frank Slootman responded: "Well, what do you think?"

"I've never seen anything sink as fast as Hadoop," Slootman said at a press briefing during the company's London summit in October. "We haven't seen that in technology, it's usually a gradual decline. This is a rapid decline. I mean Cloudera were really successful actually, and all of a sudden they've gone over the dam."

Well he would say that. After all Snowflake is looking to coax Hadoop users into its cloud datawarehousing fold, but most neutral observers would agree that Hadoop has failed to live up to its billing as a viable, infinitely scalable EDW alternative.

The writing has been on the wall for the distributed data platform for some time. First, the Hadoop distributors becoming 'data platform providers' eventually dropping the H-word altogether a couple of years back, with the Hadoop Summit becoming the DataWorks Summit. At the same time, there was much debate about the real nature of Hadoop. Was it the original distributed file system plus MapReduce and a couple of other core tools, or was it the ever-expanding Apache ecosystem of loosely related solutions including Spark and Kafka, and if so did that include non-Apache alternatives like Impala and MapR Streams?

According to Snowflake co-founder Benoit Dageville, Hadoop, meaning the HDFS filesystem and core tools, will always have its uses, but those applications will become increasingly niche. The main problem, he said, is that Hadoop is difficult to use precisely because of its distributed architecture. He compared the challenge of turning Hadoop into an easy-to-use datawarehouse with that of turning a Nokia 'dumb' phone into an iPhone.

"There are always developers and engineers that want to take Hadoop and build something on top of it, but how many engineers you know that can do that in the world? Most companies are not going to have the talent, and then it's a risky investment. Even if you have 50 engineers all working on it, even then it's not going to be a much better system than Snowflake."

Asked for comment, Cloudera directed us to a blog post entitled Hadoop is Dead, long live Hadoop by CEO Arun Murthy. In that piece Murthy describes Hadoop as a philosophy, an enterprise mindest that moves away from monolithic architectures to distributed data management across commodity infrastructure using modular open source software components.

"Cloudera is a data company. We empower people to turn data into clear and actionable insights. We do it by embracing the 'Hadoop Philosophy'. We built this market we are proud of our past, but aren't blinded by it." Murthy writes. "Hadoop as a philosophy to drive an ever-evolving ecosystem of open source technologies and open data standards that empower people to turn data into insights is alive and enduring."

But with even its main distributor failing to speak up for it as a technology, it seems like it really could be the end of the road for Hadoop.

Here is the original post:
Snowflake CEO Slootman: 'Is it the end of the road for Hadoop? Well what do you think?' - http://www.computing.co.uk