The future of big data and AI boils down to one thing
When I started my first business in the mid-90’s I did what most first-time entrepreneurs do — I ordered business cards.
Actually, I first had to get an address and order a phone. After all, I couldn’t order business cards without them. Then it was setting up an accounting system, doing the legal paperwork, building a website, and, of course, writing a really long business plan.
I did everything except the things I should have been doing: telling my story and selling my solution.
But as is so often the case, I got too caught up in the mechanics and lost sight of my purpose. It took me a while to set myself straight.
The recent Strata Data Conference, produced by O’Reilly and Cloudera, reminded me of this powerful learning experience.
The mechanics are important, of course. But they are not the reason that any of these disciplines exist. When it comes to big data, analytics, and AI the value does not come from collecting the data, or even from deriving some insight from it — value comes from just one thing: action.
Big data: Starting on the wrong foot?
The over-focus on the mechanics may have started at the very beginning. I can best sum up the ethos behind big data as: Collect it all. Sort it out later.
The focus was on building massive data lakes that collected every piece of data imaginable with the mindset that it would, at some point, be useful. But that approach is proving difficult to sustain.
“[This approach] is a mistake,” implored Satyendra Rana, Chief Technology Officer of cognitive decision-making platform diwo. “You can’t win that battle. Data keeps growing and growing, and you’ll sink in that lake. You can’t swim in it.”
Many organizations are coming to the same conclusion. Moreover, IT and business leaders are finding that they must change their mindset and focus on both operational and transformative outcomes to uncover the real value of their big data and AI initiatives.
“The mindset shift is essential,” explained David Judge, Vice President of SAP Leonardo. “There are two paths our customers have gone down. The first is an optimization path — automate and draw down the manual activity. Then, there are those that [have focused] on creating new business models [with data], which is substantially more transformative. The companies that have done the best have focused on both.”
The message is clear. Focusing on the mechanics is not sufficient when the real objective is to create value from all of this data. Which begs the question: how do you get value from data?
Operationalizing value through action
“Data has no value,” explained diwo’s Rana. “Value is created when data is used by someone in context. When the data is put to use, that’s where the value comes from. So the responsibility is not on the data creator, but on the value creator to determine how to leverage the data.”
On the surface, Rana’s statement may seem contrarian when many pundits are referring to data as the new oil or currency, imbued, it would appear, with an innate value. But as organizations get further down the big data, analytics, and AI roads, they are finding the truth in this statement.
“When we started with Big Data, we just wanted to do some quick and dirty analysis and get some insights,” explained Diwakar Goel, VP & Chief Data Officer of GE Digital. “The initial value was uncovering those insights. But then we realized that those insights weren’t making the business better. So we needed to operationalize them and get those insights to the point of action, and you want to serve it to the person that can actually act upon it.”
It is, in fact, this lack of action-oriented business focus that is the greatest challenge when it comes to the traditional data-first approach to big data.
“Data lakes are IT-oriented,” explained Asaf Somekh, founder and CEO of continuous data platform Iguazio. “They’re fulfilling a charter to build a platform to store all the organization’s data. They’re not about improving business outcomes and are not business initiatives.”
When searching for value, forget the technical context
When trying to operationalize for value, therefore, it’s important to see things from a business perspective, rather than a technical one.
That may be harder than it sounds.
When I left for this year’s Strata, I walked in with my own bias towards AI. I was convinced that AI would force the industry to re-center around business value — something that I felt has long been missing.
My focus on AI, however, was just another technical context. I, too, was failing to see the business perspective and was merely focusing on a sexier, new technology.
For those of us that have been in the tech industry for some time, it’s a hard habit to break.
And, in truth, there is a lot of business value that organizations can derive from their investments in big data, analytics, and AI in many forms. The trick is to stay focused on how to best enable those closest to the action to take action with it.
The application of both streaming analytics and time-series data are good examples of how organizations can realize this value long before a full AI implementation.
“Stream processing and streaming analytics are an essential part of operationalizing machine learning,” explained Steve Wilkes, co-founder and CTO of Striim. “If you can move the data scientists upstream and have them work with stream processing…then they can build the model and then inject that model into the data stream…and make real-time predictions and perform real-time analytics.”
As organizations go down this road toward AI, it’s also vital that they don’t miss valuable opportunities along the way that can enable action.
“Step 3 [in the evolution] is the AI and machine learning world where you can predict everything that’s going to happen,” explained Ajay Kulkami, co-founder and CEO of time-series database company Timescale. “Step 1 is collecting data, but there’s a step in between which is just using the data to monitor what’s going on…and then go from monitoring to observability. That’s where we need to get to first, to be able to see in real-time what’s going on in your business.”
Make your data make sense
The challenge with going from the historical, retrospective analysis value proposition of big data to one centered on action, however, is that it raises the stakes. And, the more real-time those actions, the greater both the risk and the reward.
In this data-driving-action world, the veracity of the data and understanding how you will use it to make a decision or take action becomes a strategic imperative.
“Decision making involves the people that are making the decision as well as the data [used to make those decisions],” explained diwo’s Rana. “So a cognitive system needs to model both — not just the data.”
With the stakes raised, the need to understand the data itself becomes a critical capability and pathway to realizing value from it.
“When you ingest massive loads of data, you’re creating huge amounts of ‘dark data’ — data you don’t know about,” shared GE’s Goel. “This is where companies like Io-Tahoe come into play. They provide insights on data. Before you can get insights based on the analysis of the data, you need insights on the data itself.”
More importantly, however, this need to understand data goes beyond things like data lineage and governance. What becomes essential — particularly when you are taking actions based on this data — is to understand your data in context and in relationship to other data.
“Data ingestion is fundamentally destructive,” expounded Goel. “When you bring data into a data lake, you lose the relationships between the datasets. The value of the dataset is less in the data and more in the relationships. This is where [tools] can help. They help you reconstruct the relationships that used to exist, and help you discover relationships between data in different datasets.”
The future of data and AI
My big take away from Strata is that the data industry is maturing. While some of the technology companies in the space are sticking to the traditional big data ethos and remain focused on the mechanics and technical nuances, many more are recognizing that it is only outcomes and the ability to take action on the data that matter.
The continuing evolution of AI will undoubtedly play a significant role in this maturation — and will likely cause the industry to transform itself again as AI takes root in earnest over the next several years.
Kicking off their coverage of the conference, theCUBE, a live enterprise technology interview show produced by SiliconANGLE Media, hosted an event called The Future of AI. During it, SiliconANGLE’s head of research, Peter Burris, summed up the future of AI this way: “The goal…is to put more data to work.”
He went on to explain that doing so involves capturing data more effectively, turning it into value — and then using it. As we increasingly turn to AI to use our newly valuable data, he explained, we will need to grapple with its ramifications.
“Because of this notion of action, it forces us to think about a new class of system,” Burris explained. “That new class of system will be called Systems of Agency.”
The idea of systems taking data and acting upon it as the agents of organizations is something that is only now beginning to become a reality. There seems little doubt, however, that this is where all roads are leading.
As enterprise leaders go down these roads, therefore, it will be essential that they remain steadfastly focused on the value of their data as expressed by their ability to take action with it.
Diwo’s Rana summed it up best, telling me as we wrapped up our conversation, “This is a data conference. But it should be a business value conference.”