How The Election Shined a Spotlight on the Limitations of Predictive Analytics Technology

The results of this week’s presidential election came as a shock to many people. But as I sat glued to CNN, watching as more and more states flipped from Blue to Red, all I could think of was how this election stands as a stark reminder of the risks that come with predictive analysis technology.

Politics, just like every other industry, has become enamored with the power of big data. And as political analysts crunched the numbers and ran predictive data models over the past months a Hillary Clinton victory seemed almost assured, with the major vote forecasters placing her chances of winning between 70 and 99 percent. Of course we all now know how wrong those predictions were.

The promise of predictive analytics is an intoxicating one, and it’s easy to see why. Traditional analytics technology is primarily retrospective, but radical advances in AI and computing power have promised to shift this paradigm to a forward looking one. Using machine learning and predictive analysis, it’s now possible to come up with reasonably accurate predictions about what will happen in the future. But this new wave of technology comes with heightened stakes and greater risks, and the near total failure by political experts to foresee Donald Trump’s victory perfectly illustrates the pitfalls of flawed assumptions, wide margins of error and lack of context when it comes to predictive analysis.

The ability to use technology to gaze into huge amounts of data to discern upcoming trends and inform future actions can be a massive advantage, and I do believe that AI will transform the business landscape especially as far as business intelligence and predictive analytics are concerned. But this technology is young and still developing, and predictive algorithms are not magic. They rely on data models that can be based on flawed assumptions, and if those models are wrong the results can be disastrous. Just look at the event known as Knightmare, where the Knight Capital Group lost $440 million in 30 minutes when their trading algorithm went haywire.

My point is that predictive analytics, while potentially game-changing, have limitations. And as professionals from every industry become ever more reliant on these tools to make increasingly important decisions, it’s more essential than ever to appreciate and account for these limitations. Predictive technology gives probabilities, not guarantees. That is why there will always be a need for experienced, insightful experts to plug the gaps left by this type of technology. Only by applying years of experience and expertise can one take those probabilities and apply the intuition, creative thinking and context needed to make truly informed decisions. As Steve Lohr wrote in a recent New York Times article, “data is the fuel, and algorithms borrowed from the tool kit of artificial intelligence, notably machine learning, are the engine.”

But we still need to be in the driver’s seat, remembering to never take our eyes off the road.