Taking the value of data lakes beyond storage

With terabytes of data pouring in from billions of devices, companies have more information at their disposal than ever before, but most are still figuring out how to make their data work for them. In a recent interview with VMblog, I discussed how companies need to extract more value from these massive amounts of raw data.

The primary challenge many companies are facing is sifting through this data and finding the concrete recommendations that improve business value. AI has made a tremendous difference in this regard as data scientists use AI to help comb through all of this data and uncover trends that may otherwise be overlooked. As the AI industry matures further, I expect we’ll quickly see the widespread use of advanced mining algorithms that will be able to find those trends at unprecedented speeds and produce better results for the business.

However, these capabilities have to make the leap from powerful technologies to powerful and affordable technologies to make a difference for most companies. The major players like Google and Amazon have already figured out how to get the most out of their data, and they have the scale behind them to be able to develop and introduce new ways of processing their data before most other companies. Most others are trying to figure out how to catch up and compete by building out data engineering pipelines and using machine learning to develop actionable insights.

I think we’ll see companies start to catch up to the major players next year, as they begin to see many of the benefits of their investment come to fruition. Continuing to adopt cloud computing, machine learning architecture, and better analytics will be critical for these companies as they make workflows more streamlined and efficient and optimize business processes.