Build a Measurement Framework before you do any Analytics to avoid failure

I’m repeatedly amazed by how often people come to me to tell me about their struggles in analytics and it stems from failing to do one basic step right at the beginning. I say ‘basic’, where of course I should say that this is hard. If it was easy, everyone would be doing it and not starting with this blog to give them ideas on how to do it.

The first step you should do when starting with any analytics is a Measurement Framework.

A measurement framework is something that allows you to work through from your business objectives to show the metrics that you should be showing to your boss to show how well the website is doing.

If you’ve chosen your metrics before you’ve gone through this process, the metrics that you use to persuade your management that your website is doing well will either be misleading, or they will lack the conviction that the metrics that older parts of your business use. Either way you will end up undermining your insight and recommendations. That means more work for you and less money for your boss.

I tend to use a bastardised version of the measurement framework that Avinash uses (although most of the rest of my organisation don’t – they make it up as they go along depending on what the client wants, likes to see or what day of the week it is). I like this because I think it is a very visual way of describing the framework. I think my favourite phrase as the moment is that you can hang this up on the wall in your office and point to it any time anyone asks why you are using that metric (I’ve yet to meet anyone who has ever hung one of these up on the wall).

Business Objectives


Working out your business objectives is the hardest part of the process.

Avinash says they should be ‘DUMB’ (Doable, understandable, manageable, beneficial) – which is the very start of the process.

This is a strategy question – you should spend time and effort working out what it is that you are trying to do with your business before working out whether your metrics are showing it or not.

Working out your business objectives is something that companies spend a lot of money hiring consultancies (including mine) to do for them. If you want to do this on your own, you will find it very beneficial, but you do a measurement framework without it (a framework from a consultancy will cost you more, but will get more buy in).

I quite often do this with clients who don’t want to (or don’t have the funds to) run proper strategies. This isn’t an easy task. You need to ask the right questions and infer an awful lot. You may make an unpopular choice that people disagree with – but stick to your guns.

Good Luck.

Website Goal

The goals of the website (or whatever digital media you have) are how you enact those business objectives. Some organisations will call them critical success factors, some will just not name them at all. Generally they are content and marketing describing what you want your users to do.

These goals may be qualified or unqualified. They may be things that you know will meet your business objectives, they may be things that you want to prove later cause your business objectives to be true (through surveys or econometrics or ‘big data’) or they may be things that you just assume will meet your business objectives.

I have had several arguments with people in my office about whether you will ever have more than one goal per objective. I think you can possibly end up with very specific objectives, where objectives overlap where you are unlikely to have more than one goal per objective.

Your goals are the sorts of things that you should be able to spot relatively quickly from an understanding of the website (or digital activity).

KPIs

Your Key Performance Indicators (KPIs) are the things that show whether the users are doing the things that you set out in your goals. Your goals, by now, should be close enough that picking your KPIs is a relatively simple job.

KPIs are all metrics. Not all metrics are KPIs. Bounce rate shouldn’t be a KPI for your website. Bounce rate is something that describes whether you are going to meet your KPIs or not.

Visits to the site may be a KPI, but I would suggest that if one of your goals is just to get people to the website then you haven’t got your business objectives quite right.

KPIs are the name of the metric that you are using to show your objectives. You should be able to say X number of visits (or people or whatever) did my KPI Y.

Targets

Targets are the things that say whether the value that you get for KPI Y is good or not. Knowing that you got 200 downloads is all well and good, but if your target is 500 then that represents a poor return, whereas if your target is 50 then it represents a good return.

Creating targets, however, isn’t as easy as it sounds. It may be relatively simple as looking at the profit that you would get from doing one of Y, but that only works if you’ve got the most obvious sales line possible and only one product.

What you can do is look at historical performance to help you build targets. You can look at your conversion rates based on the spend that you are running. You can put a finger in the air and then change it after three months when you realise that you aren’t going to make what you wanted.

Just like your work life, you should create targets that are SMART, particularly focussing on the attainable and time bound bits.

Segments

When it really comes down to it, what you really want to know is what you can change to make it so that you can hit your targets for your KPIs.

This is where your segments come in. These are the things that you report that breakdown your KPIs into smaller groups. For example Marketing A drove z of KPI Y, Marketing B drove w of KPI Y.

If you can identify these things up front, then it makes your job as an analyst much easier, because you know where you can start your analysis. It means you can ignore stupid things like what browser the person was using or what version of JavaScript they have on their computer.

Web Analytics Tools

Of course I am biased, but one of the great things about the Measurement Framework is that I can use the thing to set up my analytics system. You make sure that your goals, or custom events or custom metrics (or whatever your tool calls custom metrics) are set to be your KPIs. You make sure your segments are set up as custom dimensions, custom conversion variables, custom variables (or whatever your tool calls them).

Now when someone says I want so and so report from the tool, you can ask them why and what they are going to do with it. If it is something simple then you can say “Yeah, the tool does it out of the box – here’s how you find it.” If it is something complex then you can challenge the person as to why they really want this.

Different Industries and Business Models

Every industry thinks that they do this well or badly. The truth is that the number of times I’ve seen this done well I could count on the fingers of one hand regardless of industry and business model.

Those in industries where you sell something then this seems like it is an easier job, because you can talk about sales. But that isn’t the only point of your business and the hard sell can sometimes compromise things like persuading people to come back in the future, leaving comments about products, talking to their friends about the company, etc.

Those in industries where they don’t sell anything find this very difficult because of the lack of direct link between website activity and sales. But that sometimes mean that they have a much tighter control over what they are trying to do on their site.

Posted in Business model, Web Analytics
0 comments on “Build a Measurement Framework before you do any Analytics to avoid failure
1 Pings/Trackbacks for "Build a Measurement Framework before you do any Analytics to avoid failure"
  1. […] have difficulty proving return on investment. Part of our job at Blue Latitude is working with them to extract Key Performance Indicators (KPIs) from their Business Objectives; and helping them to ena…. KPIs are often a leap of faith where we think that a user doing something makes them more likely […]

Leave a Reply

Your email address will not be published. Required fields are marked *

*