Why we can no longer afford to pit Big Data against traditional BI - GoodData

Why we can no longer afford to pit Big Data against traditional BI

While some still consider Big Data a tool confined to behemoths like Google and Amazon, an ever-increasing number of B2B organizations of all sizes are moving beyond the constraints of traditional business intelligence by taking on the challenge of harnessing Big Data. As interest in Big Data increases, so do the number of tools available to address its demands. But looking for answers outside of your organization presents new challenges, temptations, and roadblocks.

Big Data is Eclipsing Traditional BI
Big Data offers major improvements over its predecessor in analytics, traditional business intelligence (BI). Take the fact that BI has always been top-down, putting data in the hands of executives and managers who are looking to track their businesses on the big-picture level. Big Data, on the other hand, is bottom-up. When properly harnessed, it empowers business end-users in the trenches, enabling them to carry out in-depth analysis to inform real-time decision-making.

While the scope of traditional BI is limited to structured data that can be stuffed into columns and rows on a data warehouse, the fact is that over 90% of today’s data is unstructured (as our VP of Marketing, Mike Smitheman discussed in his recent post). BI could never have anticipated the multitude of images, MP3 files, videos and social media snippets that companies would contend with in the Big Data era, but that’s the reality of business today. Traditional BI is stuck in a rut, left behind by forward-looking businesses that are desperate to tame and gain competitive advantage from unstructured business data floating around within and beyond their enterprise.

The Challenges of Big Data
Despite its advantages in today’s climate, Big Data can also present big challenges. Before reaping any benefits, a company must first hire data scientists to sift through huge volumes of data from a variety of sources. Even then, the questions remain: What valuable insights can be gleaned from the data? Who would benefit most from these insights? How can the selections of valuable data be culled, displayed, and shared with the right people?

Even beyond the implementation stage, Big Data solutions are difficult to carry out due to the demands of velocity, volume, variability: Big Data requires rapid processing of massive volumes of widely variable data types. Current solutions that purport to be magic bullet remedies lack the sophistication and scale to address these challenges.

Bypassing the Roadblocks
In attempting to overcome the three V’s, many of the early innovators in Big Data technologies focused on the areas of computing and storage. But at some point along the way, the progression of Big Data hit a roadblock when many organizations became sidetracked with Hadoop infrastructures and other low-level technologies. They’re stalled to this very day, midway along an endeavor that should have been about delivering true business value from data. Doing so is akin to building half of a bridge, only to continuously re-paint and upgrade the pre-existing suspension cables after losing sight of the original project goals.

Despite the temptation, businesses must resist the urge to follow the rest of the herd by hurling resources into a grand new Hadoop system or some other interim solution. Those who spend millions on merely manipulating their data with such insufficient solutions have lost sight of the ultimate end: profiting from concrete, actionable insights derived from business data. While these shortsighted businesses continue to toil over improving their half-built bridge, a select few have forged ahead to find new ways of spanning the waterway, all in the comfort of the cloud. These innovators have avoided the temptations and roadblocks by sharing a steadfast focus on the ultimate goal of profiting from their business data. They call this BizData Monetization.

The New Era of BizData Monetization
BizData Monetization borrows the best aspects from traditional in-house and cloud BI and Big Data solutions, drawing on structured data to optimize business strategy, but also employing massive volumes of unstructured data to get the real-time operational edge. The result is a fresh approach to extracting business value from data at a whole new scale. Business mashups, or Bashes, can now deliver executive-level views from intuitive, customizable dashboards that would stupefy a BI-enthusiast of yesteryear. Bashes breach Big Data’s obstacles by combining massive swaths of operational data from a wide variety of sources, including CRM and ERP systems and external social networks, revealing insights available on-demand, one mouse-click away.

In the era of BizData Monetization, businesses that seek to strategically differentiate themselves can no longer afford to stall out by hurling their resources into inadequate makeshift solutions, nor can they pit Big Data against traditional BI. To stay ahead of the pack, they must leverage a combination of the two to harness their data in order to profit from it. Only then will they achieve BizData Monetization.

4 Comments

Am I missing something? Big Data = Infrastructure BI = Application You wouldn't say "Why we can no longer afford to pit data warehouses against traditional BI." They're complimentary technologies. The same goes for Big Data and BI. Dashboards are as important as ever, and Big Data only helps that story.
Interesting post. The implication seems to be, if I understand correctly, that by simply moving into the cloud, businesses can leapfrog traditional BI and find actionable insight nirvana without hassle or pain. And, while this approach may indeed negate the need for Hadoop and other infrastructure investments, it still fails to address a fundamental shortcoming in many organizations, which is the lack of skilled analytical talent with business knowledge and data-aware managers and business leaders to develop a cogent strategy and design the programs. I touched on this in a recent blog post myself earlier this week here and it is also the subject of this excellent Mc Kinsey report that concludes that the US alone faces a shortage of 140,000 to 190,000 people with analytical expertise and 1.5 million managers and analysts with the skills to understand and make decisions based on the analysis of big data. Tools alone, no matter how sophisticated and whether home-spun or purchased, hosted in-house or in the cloud, will not solve this challenge.
Thanks for your comment. I was referring to Big Data and BI as full technology stacks ranging from data storage to analytical applications. At GoodData, we combine the best of both these technology stacks in our platform. I agree that if you consider Big Data as infrastructure alone and BI as applications alone, they can be complementary.
Thanks for your comment. I agree there is an urgent need for data-driven thinking and there is a dearth of people with such skills. That said, the best platforms and tools help users apply their analytical skills at the business level as opposed to having users meander through the intricacies of the tools. When users can start applying their analytical skills at the business level, fewer specialists are needed and greater proportion of organizations can make data-driven decisions.

Leave a comment

(required)