How to do Predictive Analytics - Part 2

Step 1 - Business Understanding
It sounds obvious but – in data analysis as in life - it usually helps to really work out what you are trying to do before you try and do it. Sometimes even those of who us who have been doing it for a while forget to do this with as much diligence as we should.

Having had preliminary discussions, and having agreed there is some potential to apply analytics to predict future outcomes, this is the phase where we decide whether there is a predictive effort worth undertaking. At the end of it we might conclude that it is either too risky or that the costs will probably outweigh the benefits. In our experience this rarely happens but the actual objectives can change significantly through the understanding phase.

From my perspective there are 4 crucial aspects to a successful “understanding” phase (and hence a succesful analytical task, project or programme):

1. The importance of domain (subject matter) expertise
Domain expertise is the essential ingredient that the eventual success of the whole effort relies on.

We can think of analytical expertise and domain expertise as the flip sides of the analytical coin. In some instances one person may fulfil both roles (e.g. a product manager may produce demand forecasts for his/her own brand) but in more complex situations different stakeholders typically contribute these elements.

The level of each component required clearly depends on the context. For example an external consultant who is an analytical expert may be engaged into a business area that s/he knows quite a lot about - let’s say it is consumer marketing in the retail sector - even then there are probably intricacies in the specific business context which the consultant needs to learn about from those more involved in the specific business. Strategy, current priorities and specific knowledge on this company’s customers, products, services and resources are typical examples.

Hence the business understanding phase is typically the point at which the domain expert(s) and analytical expert(s) engage the most to frame the project through the rest of its lifecycle.

The partnership between the business and its analysts also underlines a key point about how predictive analytics succeeds, or fails. There is a common, and understandable, misconception that predictive analytics (and its compatriot “data mining�) are comprised of, almost magical, techniques that can be whimsically applied to data (by someone who knows how to apply them) and gems of hitherto unknown insight will flow from through some special software. The truth is rather more mundane. It is almost always through the definition of more specific business-related goals, and the translation of those goals into achievable modelling activities, that true benefit can be derived.

From the analyst’s perspective the worst setup is usually the business challenge “here’s some data, tell me something interesting�. That’s not to say that s/he couldn’t get to something that way but if the question is a more specific, and relevant, one e.g. “can we identify which, if any, of these customer transactions are likely to be fraudulent?� then the chances of success are far greater.

We don’t talk anymore
Moreover, our experience teaches us that the more interaction there is between the business expertise and the analytical expertise throughout the project the more likely it is that a project will succeed. If I was to highlight the main reason why projects fail to live up to expectations it is probably because of the lack of communication, and hence understanding, between the two sides.

In a consulting context this is often harder to achieve because of traditional client/supplier relationships and the time constraints on the client side. However this is something that needs to be addressed in the business understanding phase. From the consulting perspective it is better to agree not to proceed with a project if one of the most critical resources; domain expertise from the business, can’t be supplied in sufficient measures.

2. Evaluate all relevant business elements
CRISP-DM has a good checklist for this but it is effectively an audit of all the people, systems, tools, financial and other operational factors which may impact the analytical effort. Clearly the level of investigation that needs to take place is dependent on the perceived size and complexity of the programme/project.

In our spreadsheet example in my last blog, though the modelling may only take a short time, if successful we might have to figure out how we deploy additional marketing effort to increase sales (if the model tells us that is what we need to do).

3. What is the risk, the cost, and the potential benefit?
Probably the key criteria when assessing risk is the familiar one of precedence. Has anyone involved done anything like this before and did they succeed? Is there any other material in the public domain that indicates if/how something similar has been achieved before?
Modelling customers who are likely to churn is one of the most common applications in predictive analytics. Mobile service companies and other providers in countries with de-regulated Telecommunications sectors are particularly active in the development and application of predictive models to achieve this objective and there is considerable evidence of success.

So if I am a mobile operator in a similar market I am more likely to conclude that there is less risk in any similar project that I was to undertake.

Sometimes the applications are slight variations on common themes. Also in telco we recently worked with one of the de-regulated suppliers to identify when it was best to try to reactivate churned customers (as opposed to the old favourite “how can we tell whether someone is about to churn in time to stop them?�). Though this was a slightly different objective it used similar analytical methods to the original. Going in we couldn’t say for certain that it would work but we had a strong suspicion - partly driven by some recent bespoke customer research, and partly because we knew that the original approach works - that it probably would.

There’s a first time for everything
Just because no-one appears to have used predictive approaches for a particular application that doesn’t necessarily mean that they can’t be applied in that way. The greatest rewards can often come from more innovative applications (as long as we are not blind to the potential pitfalls).

In the past we’ve worked with a leading global CPG company to model the best combinations of chemical ingredients as part of their product development research. We are currently working with a European partner to predict incidents of illicit, trans-national, trade. To our knowledge no-one had applied predictive analytic methodology in this way before but we are adapting techniques we’ve applied in similar areas in other domains and working closely with subject matter experts in, hitherto, unfamiliar domains.

Show me the money
This is also the phase where we need to figure out what the potential costs and benefits look like. These are usually financial and it ought to be possible at this stage to estimate what the likely levels should be. The consensual success criteria, discussed below, are usually the scenarios we use to evaluate the financial, or other, improvements that we are aiming for.

One of the great things about this whole area is that it is explicitly about modelling and predicting metrics, like ROI, with some scientific rigour. Once we get into the data we should getter a better idea of what is achievable and be able to more accurately evaluate these numbers.

4. So what do I do with it?
We’ll come on to the final phase, deployment, in more detail in a later blog but in the first phase it is important to have a clear idea of how, in the end, we expect the predictive work to be applied to achieve the original goals defined in this business understanding phase.

As this can often involve a degree of business process change everyone involved, needs to understand what needs to happen and be comfortable that it can happen. For example some of our deployments have involved daily data processing steps e.g. loading data to produce demand forecasts. Often these are automated processes requiring minimal, but regular, manual inputs.

In some instances the change can be more involved. For example we’ve often found it necessary to help train/recruit staff into model development roles. In those places where we’ve seen predictive analytics succeed the most then there tends to be resource dedicated to generating, distributing and maintaining models throughout the enterprise. More on that later.

Agreeing clear, and achievable, success criteria at this point is an essential means to clarify the ultimate deliverables using KPIs for the output goals. Some example criteria are:

  • Reduce customer churn by 20% in 6 months
  • Increase the average customer satisfaction score to 8.5
  • Reduce click fraud by 50%

For this, and the other phases, CRISP-DM formalises all of the above and details specific outputs and deliverables. From our perspective though the important thing is that everyone involved clearly sees and understands what is going to happen. The setting of success criteria is very helpful here but also that the means to that end is understood.
As I mentioned in my last blog the size of the task can vary and hence the amount of time you spend in business understanding can flex accordingly. Generally speaking the bigger the project the more critical this step is to really evaluate whether it is worth proceeding and to clearly establish the goals.

Having nailed all of the above (you never have completely but let’s say we’ve done our best) we will have decided that we probably have enough of the right kind of data to meet out goals. The next step is to delve deeper into that data to test that assumption further.

More from Applied Insights

See more: Applied Insights Blog
See more: Predictive analytics
See more: Data mining

Learn about our web analytics consulting Discover our innovative marketing applications Get to know Applied Insights