A word about averages

A lot has been written recently in this publication and others about the development of Key Performance Indicators (KPIs). Increasingly businesses are doing the right thing by aligning metrics to the web channels goals and objectives and ensuring that they are “measuring the right things in the right way”.

When you are developing your KPIs or when you are looking at other measures of performance, it’s possible that you may consider metrics such as:

  • Average time spent on site per visit
  • Average order value
  • Average number of pages viewed per visit
  • Average number of visits per visitor and so on

These metrics are typically found in the reports of most web analytic systems but the trouble is that they are also potentially highly misleading. The reason for that is that the majority of web analytic systems will report on these metrics using what is know as the arithmetic mean which is inappropriate for the type of behaviour we observe on most sites.

For example, the average number of pages viewed per visit will be calculated as the total number of pages viewed on the site divided by the number of visits made to the site. The trouble is that the underlying assumption about the use of the arithmetic mean is the data has a “normal” distribution to it, ie the famous bell shaped curve that we all learned about at school or college.

If the distribution of the number of pages viewed per visit is “normal” then you would expect around half of visits to be less than the average and half to be more. In addition, you would also expect “most visits” to be around the average value.

The reality is that user behaviour on the web is not “normal” in the statistical sense of the word. It is usually highly abnormal and highly skewed. In fact, most visitors on most sites don’t do very much of any value. You can see what I mean in the example below.

The chart shows a fairly typical example of what you find on most sites. In this case we are looking at the distribution of the number of visits by the duration of each visit on the site. You will also find a similar pattern when you look at the distribution of the number of pages viewed per visit, the number of visits per visitor, the number of orders placed per customer and so on.

In this case the reported average time spent on site is 6 ½ minutes. But you can see that the distribution of visits is highly skewed. There are a very large number of visitors who spend a very short time on the site and a small (but significant) number of visitors who spend an extremely long period of time on the site.

The net result is that it “averages” out to about 6 1/2 minutes. However, the reality is much different:

  • 50% of all visits actually last for 3 minutes or less (ie about half of the “average”)
  • 70% of all visits last less than 6½ minutes.

You can see that using these averages as metrics that you report around the business hides the true behaviour of visitors on the site. These averages also will tend to overestimate what the bulk of visitors are doing on the site in terms of time spent, pages viewed and so on. The mean is also sensitive to changes in the behaviour at the extreme right hand end of the scale.

So, what to do? Well, one alternative is to use the “median” which is the point in the distribution where 50% of people lie (in our example above it was about 3 minutes). Unfortunately most web analytic systems do not produce this metric and so you would have to calculate it manually. It would be great if web analytics vendors would start to produce this metric as standard. WebTrends does report on the median visit duration as standard and WebTrends users should take a look at how different that number is to the average that they probably typically report.

We also advise clients to use a threshold measurement instead of an average and to look at the proportion of visitors who spend more than a certain amount of time on the site, or who visit more than a certain number of pages. This threshold is usually easier to calculate that a median and is also perhaps conceptually easier to understand for most business users. What the thresholds should be is dependent on the type of site and should tied to the goals of the site. For example, if it would take a minimum of 3 pages to reach any valuable content on a corporate website, then an appropriate metric might be the number of visitors looking at 3 or more pages.

So, beware of averages and look at the underlying patterns of behaviour on your site to get a better understanding of what is really going on.

More from Applied Insights

See more: Articles
See more: Web analytics

Learn about our web analytics consulting Discover our innovative marketing applications Get to know Applied Insights