Written by GoodData Author |
Sometime after the 27th joke about the use (and misuse) of the buzzwords “big data” and “Hadoop”, the core takeaway of this year’s DataBeat conference became clear. We as an industry, as a society, have vague notions of what the future brings – the leveraging of massive datasets with advanced algorithms and machine learning that will embed intelligence into every facet of our organizations. But despite this healthy anticipation of the future, most still struggle with the basics of running a data driven organization. Hence the eye-rolls and snickers at the grandiose talk of big data from those who would seek to evangelize calculus before passing algebra. Organizations of all sizes must first master the basics, with the help of new technologies that will help them do so.
To launch themselves on a new, data-driven trajectory, organizations should consider the following nuggets of wisdom from this year’s conference:
1. Top-down integration will fail
Tom Davenport, author of the 2014 book Big Data @ Work: Dispelling the Myths, Uncovering the Opportunities, explained how top-down data management will give way to more flexible and subtle systems for managing integrations and modeling data, rather than one-sized fits all systems mandated by IT. This sentiment was echoed by subsequent speakers like Andy Palmer, who called for intelligent systems to help integrate more data sources than traditional master data management systems have been able to accommodate.
2. Organizations must assume a data driven culture
Throughout the conference, VPs, directors, and heads of data science and analytics from the likes of AirBnB, LinkedIn, Pinterest, and Intuit championed the importance of developing a data-driven culture. We learned how AirBnB used a data-driven feedback loop to inspire product developments that reduced support ticket submissions by 75%. Pinterest built systems to make data available to many more employees, leading to an in-product tweak that increased social sharing by 150%. Meanwhile, Intuit used application usage data to make it easier for product registrants to retrieve their company’s tax code ID – leading to an immediate 12% spike in conversions. Their advice? Focus on your business problems, identify key metrics that relate to those problems, and the right analytics tool for the job will become obvious.
3. Data promotes creativity
Data promotes creativity. Managing data does not. So why do companies hire really smart data scientists with degrees like PhDs in statistics and machine learning, only to ask them to engage in low level “data plumbing”? Rob High from IBM Watson and a host of others explored this topic over the course of Databeat, concluding that applications replacing rote human tasks will continue to free “data plumbers” to engage in more high-valued-added data science activities in years ahead. One way this might work is by implementing probabilistic applications that guess how data should be modeled or integrated, which could then be curated by human overseers. As companies begin to specialize in developing AI applications to help free up human time and energy to focus on more interesting problems in analytics, BI vendors with open platforms will be best suited to taking advantage of the latest technological advances.
There remains a surprising degree of disagreement on the methodologies and technologies that will pave the way in the years ahead (traditional BI stacks or “data lakes”? Hive or MapReduce? Hadoop or NoSQL?). But at a high level, Databeat speakers showed broad agreement on the fact that organizations of all sizes will continue to benefit from adopting tools that permit flexible data integration, that empower users across each organization to join in on data-driven initiatives, and that effectively outsource rote tasks of data management to new technologies and third parties. A new era of analytics-inspired creativity awaits.
Written by GoodData Author |