Selecting analysis systems

There was a bit of research publicized recently about organizations’ satisfaction with web analytics systems. It reported that 62% of the respondents from these large companies were happy enough with their web analytics package that they would recommend it to others. 62% seems like a pretty good number to me but what about the other 38%? Are they not getting what they need from their package and if not, why not?

I’ve worked with a number of companies over the years helping them to select and implement analytical software including web analytics packages. Indeed I went through the painful process myself, implementing a tag based ASP solution across a dozen sites across Europe and integrating it in with other information and operational systems. As a result of the work I do with clients, I have heard lots of stories about why they want to move from their existing package to another one.

Some of the common themes that I come across are:

The system doesn’t deliver any value to the business…
The system sits in isolation and it’s difficult to get it to integrate with other data…
The system doesn’t appear to work properly and deliver what’s needed…
Nobody really knows how to use it…

These themes are usually the symptoms of a poorly thought through selection and implementation process for the existing system and may not always be a problem with the system itself. Often these issues stem from a lack of clarity about what is really needed in the business and also an underestimation of the associated effort required.

There can be a tendency in the process of selecting analytical systems to be led by the technology and the feature set. As I mentioned in the last article, funky overlays and pretty dashboards might make the data look good, but you also need to ensure that you can get at the right sort of data you need in the right sort of way.

Here are some thoughts about how to minimize the risk of being one of the “38%”.

  • Be clear about your goals
  • Be very clear about what you are trying to do online and the reason for the investment in the web channel. From this all else flows.
  • Define your KPIs and main tracking metrics

Once the goals are visible, it’s possible to identify what the channel Key Performance Indicators (KPIs) are. In addition, there will also be a number of other important tracking metrics as well, which are not as strategically important as KPIs. It’s important to recognize that KPIs are not always metrics that come out of a web analytics package. In fact, in my experience they rarely are.

Define the business processes

As well as defining the important metrics, it’s important to map out and define the important businesses processes. How are campaigns managed? How is the site development process managed? The tools should help the business processes and you shouldn’t need to change your business processes to fit the tool, unless it is clearly a much better way of doing it.

Write the business requirements document

This is the document that goes out to potential vendors. The point of the document is to clearly set out your needs and to invite the potential supplier to articulate how their system can meet them. In my view this document should enable the best potential suppliers to shine through, so don’t be too prescriptive in approach. The kind of huge document that requires huge amounts of detail input on every single aspect of the system does not help at this stage. You hate writing them, the vendors hate responding to them and then you hate reading them. And even then, there is no guarantee that you will be able to see the wood from the trees anyway.

Keep the requirements document concise, clear and open. Invite vendors to make an effort to differentiate themselves. A vendor once told me that the best brief they had had from a prospect was “This was what we are trying to do, tell me how you can help us do it better” and that was it.

Get it down to two or three

From the various responses that you get, you need to whittle it down to two or three potential vendors to come in and pitch for the business. Don’t invite them all. Look for quality not quantity in the response documents that you get back from them. First of all, have they answered the brief? Have they thought about what you asked them to think about? Or have they merely cranked the document out of the proposal machine? Do they demonstrate an understanding of the business and how they can help? Does it look like they want your business more than the next person?

Test it out if you can

In an ideal world you should test at least your preferred system, or preferably two systems side by side to see them in action. Analysis and reporting systems are “experience products”, you only really know what you are going to get when you’ve already got it. Having real hands on experience will be invaluable in helping you decide whether this is the one from you and you are going to be one of the “62%” of organizations.

Selection though is only half the battle. The other half is getting the system in, working properly and being used to its potential. Next time, I’ll being sharing some thoughts on successful implementations. Till then…

More from Applied Insights

See more: Articles
See more: Analytics strategy
See more: Web analytics
See more: KPIs

Learn about our web analytics consulting Discover our innovative marketing applications Get to know Applied Insights