How big data and AI work together – The Enterprisers Project

Big data isnt quite the term de rigueur that it was a few years ago, but that doesnt mean it went anywhere. If anything, big data has just been getting bigger.

That once might have been considered a significant challenge. But now, its increasingly viewed as a desired state, specifically in organizations that are experimenting with and implementingmachine learningand other AI disciplines.

AI and ML are now giving us new opportunities to use the big data that we already had, as well as unleash a whole lot of new use cases with new data types, says Glenn Gruber, senior digital strategist atAnexinet. We now have much more usable data in the form of pictures, video, and voice [for example]. In the past, we may have tried to minimize the amount of this type of data that we captured because we couldnt do quite so much with it, yet [it] would incur great costs to store it.

[ Could AI solve that problem? Get real-world lessons learned from CIOs in thenew HBR Analytic Services report,An Executives Guide to Real-World AI.]

The more data we put through the machine learning models, the better they get. Its a virtuous cycle.

Theres a reciprocal relationship between big data and AI: The latter depends heavily on the former for success, while also helping organizations unlock the potential in their data stores in ways that were previously cumbersome or impossible.

Today, we want as much [data] as we can get not only to drive better insight into business problems were trying to solve, but because the more data we put through the machine learning models, the better they get, Gruber says. Its a virtuous cycle in that way.

Its not as if storage and other issues with big data and analytics have gone bye-bye. Gruber, for one, notes that the pairing of big data and AI creates new needs (or underscores existing ones) around infrastructure, data preparation, and governance, for example. But in some cases, AI and ML technologies might be a key part of how organizations address those operational complexities. (Again, theres a cyclical relationship here.)

[ Sort out the jargon jumble. Read:AI vs. machine learning: Whats the difference?]

About that better insight thing: How is AI and ML as its most prominent discipline in the business world at the moment helping IT leaders deliver that, whether now or in the future? Let us count some ways.

One of the fundamental business problems of big data could sometimes be summarized with a simple question: Now what? As in: Weve got all this stuff (thats the technical term for it) and plenty more of it coming so what do we do with it? In the once-deafening buzz around big data, it wasnt always easy to hear the answers to that question.

Moreover, answering that question or deriving insights from your data usually required a lot of manual effort. AI is creating new methods for doing so. In a sense, AI and ML are the new methods, broadly speaking.

Historically, when it comes to analyzing data, engineers have had to use a query or SQL (a list of queries). But as the importance of data continues to grow, a multitude of ways to get insights have emerged. AI is the next step to query/SQL, says Steven Mih, CEO atAlluxio. What used to be statistical models now has converged with computer science and has become AI and machine learning.

As a result, managing and analyzing data depends less on time-consuming manual effort than in the past. People still play a vital role in data management and analytics, but processes that might have taken days or weeks (or longer) are picking up speed thanks toAI.

AI and ML are tools that help a company analyze their data more quickly and efficiently than what could be done [solely] by employees, says Sue Clark, senior CTO architect atSungard AS.

Mathias Golombek, CTO atExasol, has observed a trend to a two-tier strategy when it comes to big data, as organizations contend with the massive scope of the information they must manage if theyre going to get any value from it: The storage layer and an operational analytics layer that sits on top of it. News flash: the operational analytics layer is the one the CEO cares about, even if it cant function without the storage layer.

For specific use cases, it revolutionizes the way you get rules, decisions, and predictions done.

Thats where insights are extracted out of data and data-driven decisions take place, Golombek says. AI is enhancing this analytics world with totally new capabilities to take semi-automatic decisions based on training data. Its not applicable for all questions you have for data, but for specific use cases, it revolutionizes the way you get rules, decisions, and predictions done without complex human know-how.

(In an upcoming post, well look at some use cases that illuminate how AI and big data combine forces, such as in predictive maintenance essentially predicting when a machine might fail, for example and other practical applications.)

In other words, insights and decisions can happen faster. Moreover, IT can apply similar principles using AI technologies to reduce manual, labor-intensive burdens and increase speed to the back-end stuff that, lets face it, few outside of IT want to hear about.

The real-time nature of data insights, coupled with the fact that it exists everywhere now siloed across different racks, regions, and clouds means that companies are having to evolve from the traditional methods of managing and analyzing [data], Mih from Alluxio says. Thats where AI comes in.Gone are the days of data engineers manually copying data around again and again, delivering datasets weeks after a data scientist requests it.

Like others, Elif Tutuk, associate VP ofQlik Research, sees AI and ML as powerful levers when it comes to big data.

AI and machine learning, among other emerging technologies, are critical to helping businesses have a more holistic view of all of that data, providing them with a way to make connections between key data sets, Tutuk says. But, she adds,its not a matter of cutting out human intelligence and insight.

Businesses need to combine the power of human intuition with machine intelligence to augment these technologies or augmented intelligence. More specifically, an AI system needs to learn from data, as well as from humans, in order to be able to fulfill its function, Tutuk says.

Businesses that successfully combined the power of human and technology are able to expand who has access to key insights from analytics beyond data scientists and business analystswhile saving time andreducing potential biasthat may result from business users interpreting data. This results in more efficient business operations, quicker insights gleaned from data and ultimately increased enterprise productivity.

Read the original:

How big data and AI work together - The Enterprisers Project