Is Engagement a metric?

Is Engagement a metric?

I was about to start this post with “Ok, I’ve talked about engagement before…”, but when searching for those posts, I realised that I’d started a post like that before, so I had to come up with something different.  I couldn’t, so I decided that I’d just tell you that I was going to start like that as a way of broaching the subject.  Needless to say, you’re probably very hoping that I get to the point at some point, so I’ll go straight there.  Eric T Peterson recently wrote a white paper on Engagement and in particular a metric that measures visitor engagement.  He then followed that up a couple of weeks later with a follow up on Audience Engagement with some examples included.

Hang on a second, I can hear you saying.  You said a while ago that you couldn’t measure engagement – you could only measure things that indicated engagement.  In fact, back in June I argued that Engagement was conversation where I talked about how the user experience was the brand experience and hence engagement was the conversation you were having with the user.  Or the conversation that the user was having with you – it should work both ways.  Eric and Joseph Carrabis in their white paper argue something slightly different.  They use the definition of engagement to come up with the following:

Engagement is the demonstration of Attention via psychomotor activity that serves to focus an individual’s Attention.
Attention is a behavior that demonstrates that specific neural activity is taking place.

Which they’ve then broken down into a more specific website version of the above quote:

Engagement is an estimate of the depth of visitor interaction on the site against a clearly defined set of goals.

But what does that mean?  I think the second one of these two definitions is a bit more meaningful, but I don’t know how you could measure it specifically.  How do you measure the depth of visitor interaction on a site and if it is only an estimate what use is it?

Well Eric thinks you can measure it, as proved by the fact that it is against a set of clearly defined goals.  So this is what he’s come up with:

   Σ(Ci + Di + Ri + Li + Bi + Fi + Ii)

Which looks a little confusing, but it is nowhere near as confusing as the long version of the formula:

 

What are they saying?  Well basically they are saying that there are seven measures of engagement and they have normalised them all so you can then add them together.  This means that at some point you could get a site with engagement of 7 and that would be perfect, but you could also have a site with engagement 0 and that would be rubbish.

I won’t go into too much detail on what each of the seven metrics is and how it is measured (you can read it in the white paper), but the crux of it is that you take the standard metrics we always look at (page views per visit, visit duration, recency and loyalty) and mix them in with some other non standard metrics (Brand awareness, providing feedback or from feedback surveys and your interaction index which are your conversions).

My suggestion is you pick up the whitepaper and try running it for your site.  See what you come up with.  There are a few things that may be a bit difficult to do, but I reckon for your average site, using Google Analytics you could do it.  Here below for example are the page viewed per visit:

 

Here above we have depth of visit.  I think that in the document Eric and Joseph suggest that you take your percentage of visits who viewed more than four pages.  That sounds a bit optimistic for my blog – most people only stay for one page.  I would have said that was similar for most media sites (ok, maybe not most of them, but some of them).  The other metrics that you’ll need from Google Analytics are there also.  In fact they are in that one section of the report right there.

I think the most important question here though, is one that was debated on the web analytics forum and on the bottom of Eric’s articles.  I like the quote Eric gives in the comments at the bottom of the article:

Engagement and conversion are different measures. I’ll say it again: ENGAGEMENT AND CONVERSION ARE DIFFERENT MEASURES. Same for satisfaction … ENGAGEMENT AND SATISFACTION ARE DIFFERENT MEASURES.

This is something important, I think.  You have your conversions – in the retail world these are the things that give you  money.  These are the most important things on your bottom line.  If you are only concerned with your bottom line, you should be looking at your conversions.

If you are more concerned about whether you are going to get your users to come back, then you really need to be concerned with the Satisfaction of your users.  This Satisfaction works in two ways, remember.  You need the satisfaction of your website users – will they come back, do they like the experience, how happy are they.  You also have your satisfaction of your customers, this is all down to the quality of the product, rather than the website experience.

What I don’t understand then, is why in the Engagement metric we are including conversions and satisfaction.  Engagement now is a combination of conversion and satisfaction.

This brings us on to the Audience Engagement metric.  I think that I get this one much more.  This is something that Comscore wants to use, because people are starting to question their metrics (Comscore uses a sample which it then extrapolates to the whole world, read Avinash’s great review).  So lets take the bits of the Visitor Engagement metric that we can measure from Comscore and put them into a new metric – Audience Engagement.

So if we’re not confident that their data extrapolates that well, how do we get around this problem?  Well we use the information that they do have to come up with an Audience engagement metric.  This doesn’t need to be extrapolated, because it doesn’t grow with the size of the audience, it just gets more accurate.  This means that Audience engagement is a great measure for those that use Comscore.  It will tell you which websites have users that go deep into the site. 

Is this good for an advertiser that is planning to put adverts on those pages?  Not quite so sure.  Display advertising has long stopped being about click through rates (they are rubbish and not worth it on the whole) and started being about brand advertising.  For your brand is it better that users are seeing more pages on a site and coming back more frequently?  I don’t know.  If they are more engaged with the site, they might not notice the ads around the side any more.  So do you want your site more or less engaging?  It’s a tough one to call.

3 Comments on “Is Engagement a metric?

  1. I am not convinced measuring engagement like a mathematic equation is really the whole story. If you have read this http://www.digitaldesignblog.com/uploads/2008/11/meet-the-connected-consumer.pdf you may agree on that. Human beings are weird in their behaviour and I think the reasons to be engaged are so vastly different between individual users that to quantify it as a score is potentially misleading. I can understand wanting to come up with a way to do it and it certainly is a holy grail type of activity for an analyst but surely there is more to it than 7 elements…what do you think?

  2. I have thought about this quite a bit actually James and I don’t think the question should be ‘is engagement a metric?’. It is. We’ve just come up with one. We’re not suggesting that this is a perfect metric for measuring engagement, but then that isn’t the point of the metric – it’s meant to give an indication.

    The question is – what do you do with it? What action do you take because of it? Does it mean that your website is good? Will it sell more if goes up? Probably not – so how are we going to use it? I don’t have that answer yet.

  3. Hi Alec,

    I think what seems most unnatural is to mix factors that are quite different into one metric, also the fact that Eric Peterson’s engagement metrics factors in conversion.

    I see the value of an unified index within the comScore metric offer. But at the same time I doubt if this is the way to go as analysts for sites with individual needs.

    I posted an approach that keeps the different flavors of engagement separately and also allow to view the contribution of the different audiences to revenue.

    http://www.webanalytics.cl/en/11/2008/aware-amused-addicted-aaa-analysis-a-spin-off-to-eric-peterson%E2%80%99s-engagement-metric/

Leave a Reply

Your email address will not be published. Required fields are marked *

*