How to keep big data projects from failing

April 06, 2018
Roman Stanek's picture
Founder & CEO
Roman Stanek is a passionate entrepreneur and industry thought leader with over 20 years of high-tech experience. His latest venture, GoodData, was founded in 2007 with the mission to disrupt the business intelligence space and monetize big data. Prior to GoodData, Roman was Founder and CEO of NetBeans, the leading Java development environment (acquired by Sun Microsystems in 1999) and Systinet, a leading SOA governance platform (acquired by Mercury Interactive, later Hewlett Packard, in 2006).

Big data projects fail for a number of reasons,  most of which stem from not sufficiently understanding or preparing for the complexities of a new system. In a recent Inc.com article, I spoke with Greg Satell about the top four ways companies can keep a data project on track and ensure a successful outcome.  

In the article, we specifically cite building for purpose, automating the most tedious tasks first, focusing on the right data (instead of all data), and starting small. I think those four things can be summed up by the idea of developing a manageable implementation plan, rather than a comprehensive implementation plan. Too often I see companies that are understandably excited about all that automation and big data projects can offer, so they want to jump in with a major overhaul right away. While they sound promising, these kinds of implementation plans are just not manageable, and the backlash when things don’t go according to plan can make it harder for future efforts to succeed.  

Instead, companies should start with a task that is tedious and easy to automate. In this way, there’s a measurable, positive difference in an employee’s work experience, and it’s less risky than automating numerous disparate tasks. This kind of first encounter with a data or automation project will help future projects succeed, and picking something small to start with also enables companies to collect feedback and more quickly and easily make any necessary adjustments.

Companies shouldn’t implement data projects just to implement one. Instead, companies should implement data projects that have a goal of solving a business problem or improving a process. If a project doesn’t deliver positive change to employees or the business, it won’t be successfully integrated or used, and it won’t deliver the kinds of outcomes that have been promised. Using a small step as a learning opportunity for other, more complex automation processes sets companies up for success and ensures that employees are on board.