Why Predictions Are Just Extensions of History

It’s only August, but the Minnesota State Fair is knocking on the door and for some, budget season is beginning. Of course, you don’t want just to look at the last couple years by customer and product; you want to look forward in ways that support and frankly doublecheck your sales and development teams.

In a world that seems obsessed with AI - affecting everything from human relationships to market trends to financial management - the term “prediction” can conjure thoughts of behind the curtain calculations, even magic. But it’s not that fancy: what we often call a “prediction” is little more than a sophisticated extrapolation of historical patterns.

Of course, analytics models are providing awesome help speeding up analysis of trends, of course based on a variety of factors: history, macroeconomic movements, sentiment, and others, depending on what we want to learn. And the toolsets are rapidly expanding, to the point that it can be confusing: do we want to use AI platforms from SAP or Salesforce or Microsoft, or standalone products like Alteryx or Qlik, or program something ourselves with Python/R/NumPy?

Of course, the solutions choices are often defined by our organization’s strategy, and there are many good options.

The far more important topic is: How do we make relatively accurate and helpful forecasts and predictions? Ones that get better and better as our internal capability grows and improves?

The answer, naturally, is in the data. We can simplify the keys to success using three steps. 

1. Know what you want to know.

This sounds so simple, and can be very difficult. Think about weaknesses or pain points. Do you want to do better at Gross Margin analysis down to individual product performance? Sales region performance? Cash flow management? Real estate location analysis?

Ultimately, many times we want to know what’s going well (and conversely, poorly) so we can do MORE of the good stuff.

But it cannot be overstressed: write down the requirements for analytics at the start and have clear and achievable goals.

2. Realize that good forecasting is simply using great tools to extend historical performance.

That’s all forecasting is. Thanks to good old Machine Learning, our tools can analyze factors and grade them by importance, but start by defining your key market factors. Usually you know what they are.

Then get your data. Three years is always nice so your tools can learn enough about seasonality, sentiment, customer quirks, and salespeople strengths, for example.

Define where the data resides: ERP? CRM? Excel files? Doesn’t matter. Locate your data, and define a process for getting it somewhere it can be staged and brought together: could be the cloud, could be sharepoint.

3. Clean your data.

This is the hard part and worth a lot of thought as well as work. We don’t just need solid history, we need comparable and clear history. Are your part numbers and sales region codes unified? Are the columns titles clear? Are there no blank cells?

And please remember, no one – at least no one at companies we run into – has perfect data. This is a process, and worth every ounce of the effort. But it’s hard and can take a while.

Of course next, having chosen your tools, run the data and start training your model. Get engaged and understand this, too, is a lengthy process. Progress not perfection is the watchword.

And then, after all this, presentation is the easy part. Really.