What WebTrends should do for Analytics version 12

I've been in my current job for about a year now and I've been working quite a lot with WebTrends (as well as Google Analytics and all sorts), but this week I've been working on SiteCatalyst with a client again. It reminded me of a post that I wrote back at the end of 2010 on what SiteCatalyst should do for version 15, so I thought that I would write one for WebTrends as well.

WebTrends is currently on version 10.6, with the last major release (version 10) being in April 2011. Version 10 is a superficial improvement in the user interface over version 9 ('the admin interface'). This lack of update in the last couple of years to their flagship product (if it even is any more) means that WebTrends is falling behind SiteCatalyst (see who would win in a fight between SiteCatalyst and WebTrends) and Google Analytics. Therefore the point of this post is things that WebtTrends could do better to encourage people to use their Analytics tools again.

1. Live Segmentation

WebTrends can do segmentation and it can do it really easily. You can apply filters to individual reports in the backend of the system and you can apply whole filters to entire profiles.

This is very powerful.

But it is misused. At the moment it is used to create reports that can be used on a regular basis. This is reporting.

Segments should be used to allow analysts to create analysis for insights and recommendations. This is one of the pillars of the analytics maturity model for organisations. At the base level you are producing reports, at the more advanced levels you are doing analysis. If you have to know in advance of your users doing things that you need the reports then your not doing analysis, you are doing reporting.

The trouble WebTrends has is that the people using Analytics tools have changed in the last 3 years and WebTrends hasn't. You need to be one step ahead of your audience.

Google Analytics does it, SiteCatalyst version 15 does it, WebTrends should do it.

2. Standard Custom Measures in any report

Setting up measures in WebTrends is one of the most annoying things ever.

I want to be able to code on the page WT.metric1=true and metric1 will appear in any report that I want it to.

I don't want to have to go into custom measures and create a new one. 

I don't want to have to add it to every report in the back end. 

I just want it to be an option that appears in all reports when a user wants it to. It may not make sense with the visits and the page view based measures, but I'll let the people looking at the reports work that one out.

Currently the barrier to adding measures to reports means that I have to spend too much time up front working out which measures work in which report. I have to set up measures that are visit based or page view based depending on the situation and if I have too many of them then it ist5 annoying to have to create two reports. Just let the users add them as they want.

3. Second dimensions on the fly

SiteCatalyst does it, Google Analytics does it, even educated fleas does it.

You have two reports. One has, say, the keywords that you are bidding on in your search engine. You have another report that shows the search terms that you pick up from the referrer field when a user clicks through. 

Want to break one down by the other? 

Tough, unless you set it up in advance.

However, it turns out that you want one dimension broken down by five different ones in various different ways. That shouldn't need five individual reports, it should need one report that you can add a second dimension to on the fly.

Google Analytics does this in a way by allowing you to break each report down by a subset of the other reports, depending on which one it is. This would seem like the safest solution for WebTrends, but WebTrends has lots of custom reports so a SiteCatalyst style breakdown might work better in practice.

4. REST user interface

Rest is a great idea. But it feels like someone has implemented it in a real half arsed way.

It's great that you can just download data from reports by using the API so that you can create your own interface. You can put the data in your third party systems much more efficiently and you can put data into Excel however you want.

What Rest really needs is a GUI for people like me who don't want to have to bother with coding stuff on pages. What Rest needs is a Report Builder.

Why does Rest need to do this? Because Rest has been lost in a past that no longer exists in most companies where the analytics tool is looked after by the IT team. The Analytics tool should be looked after by a Marketing team (or the end users of the reports, who are usually Marketing). Because of this the way of getting data into excel should be in the most simple fashion possible.

5. Improved Dashboards

Dashboards in WebTrends are frankly quite poor and probably unusable.

I wouldn't recommend any of my clients use them and instead export the data themselves into Excel to manipulate there. I think it would be rare that you wouldn't advise even the basic user to manipulate some of the data in the tool itself.

This is probably a symptom, slightly, of some of the other shortcomings of the tool rather than a symptom of the dashboards themselves.

I could probably write a whole blog post on things that could be improved in the dashboards, but starting with:

• Simple filters of the reports
• Adding more than one measure to reportlets
• Having more than five lines in the reportlets
• Sharing the dashboards with other users
• Adding in custom stuff (like a box for normal text to explain stuff)

6. Targets

I've spent the last twelve months telling clients that they need to set KPIs that they know reflect their business objectives to build confidence in what they are doing.

Having set KPIs they need to give targets to those KPIs so that they can tell whether they are performing well or not. 

The favourite question is "What does good look like?" Good looks like hitting your targets for your KPIs. How do you know whether you've hit them or not? You have uploaded them into your analytics tool and the reports that you get in your newly created dashboards show how you are performing against them.

What to do with the demographics of your twitter followers with demographicspro

When I got an email in my inbox from Schmap the other day about a new tool they had released I thought I was going to leave it sitting in my junk box. In reality I sent it around the office to everyone telling them about it and asking if they had any clients who might want to use it. We probably won't do, because of the nature of most of our clients (they work in regulated markets with few twitter accounts). But you guys could be interested in it, if I tell you what it is and how it works.

What is this new tool?

Demographicspro is the name of a new tool and was launched last week. The tool costs money, but not very much - if you've got a tiny account like mine then it could cost about $20 for a one off analysis, in fact if you've got anything up to 10,000 followers it will cost as little as $80 for a one off analysis. You can, of course, buy a plan for regular reporting to show how your demographics change over time.

How does it work?

The company says it uses algorithms to crunch proprietary data across four general focus areas -- network, consumption, language and physical appearance -- though it doesn't say where it comes from. At any rate, it's better than what Twitter offers brands today, and it spits all of this information out into your choice of an Adobe PDF report or Microsoft Excel spreadsheet.

says Andrew Nusca over at ZDNet. Translating this into English it means that they use meta data about tweets (if you say where you are or what tool you are using, for example), information in the tweets themselves and any information entered into the profile of the twitterer.

My guess is that there are a number of standard things that if you put in a tweet relates you to groups of demographic information - use of hashtags, use of language in tweets, etc, etc.

What data do you get?

I'll give you some information about my Twitter followers (@whencanistop, in case you didn't know!), just to get you started.

Here you can see some information about my followers that shows their gender, marital status, age and income. In the middle it shows what percentage of my followers fall into each of the demographics and on the right hand side it shows the entire twitterati.

The most important bit of information on the chart is that little black dot on the right hand graph. It shows you how you compare to the entire world. The further right the dot is (in the dark red) the higher the representation that group presents of your followers compared to the average. anything that is on the far left in the yellow is much less represented.

My followers are more likely to be male (this is hardly surprising in this industry), more likely to be married and/or have children than average, in their late 30s and high earners. If that describes you and you are a follower, well done.

To give you a quick flavour of some of the other bits of information you'll get, above we have some information on the location of the followers, what the interests of followers (77% are interested in technology - I think that someone could do with breaking that large group down a bit in future developments) and what occupation the followers have:

Is it any surprise that so many are into 'Sales/Marketing' (does 'Sales' as a job even exist any more? And if it does, these people are surely really different from Marketing people that it isn't worth grouping them). It is nice to know that I'm also connected to a lot of Senior Managers - that should make it easier to sell my services over Twitter (although I do use my twitter solely for professional purposes, so maybe it isn't so much of a surprise).

We also have information about which people your followers are likely to follow too (Pete Cashmore appears at the top of the list - he must be very popular around my groups!). We also have information on the sorts of Brands that my users are interested in:

These bits of information are probably found from various bits of data about check ins, tweets and generally linking demographics of people to the demographics of the sorts of brands they like. Here it is more important than ever to look at the dot to see where you sit compared to the global audience. KFC appears high on my list, but my users are far less likely to like KFC than the whole world. Me tweeting "@KFC, isn't it great! #chickenforlunch" probably wouldn't go down so well.

Finally we have some information about how long these people have been on twitter, how many followers they have, how many they follow and their daily activity. Knowing how often my followers tweet gives me some insight into how often I should be tweeting too (my first tweet was just over 5 years ago and I've tweeted almost 1,500 times, so once a day makes sense).

How can a Brand use this data?

I see there being five ways to use this data:

1. Tone - you can use information about your followers to dictate the tone of your tweets. You may think you know about them, but you may not necessarily have the level of detail that allows you to decide what tone of voice you are going to use.
2. Compare to peers and competitors - use the information that you gain about your brand and compare to peers and competitors by running the analysis on them too. You can gain insight from comparing your tactics to theirs and the results of the followers.
3. Work out what brands you might want to partner with - knowing other brands that your users are interested in may allow you to build strategic partnerships with 'upstream' or 'downstream' brands - things that are related to your product, but aren't direct competitors (eg if you sell socks then you might want to partner with a shoe company).
4. Use the general interests of your users to create personal tweets - if you know what users are interested in then you'll know whether to make tweets about events (eg my followers are universally uninterested in music, tv or film - so I know not to tweet about these things, but tweeting about new technology or social media is encouraged).
5. Frequency and timing of tweets - knowing where your users are and how often they tweet will let you know when you should be tweeting for maximum visibility and how often to get the users attention

Remember though that your overall twitter strategy should integrate with your overall strategy and should drive real world outcomes. You get data from this tool and you need to analyse it to get insight that you can use to change stuff. You then need to use your standard measures to show the impact of these changes. You can start using this data as part of your metrics, of course, and run this analysis on a regular basis.

6 steps to get the most out of AB or Multivariate testing

With the risk of teaching grandmothers to suck eggs, I'm going to spend a bit of time today talking about A/B testing and multivariate testing. Last time I wrote about this was five years ago when I wrote about how you could do A/B testing to improve engagement on sites. In five years lots has changed: I have grey hairs and A/B testing is a lot easier to do than it was back then. You have free tools and you have cheap tools that are readily available to you that you just didn't have back then. Now is the time to make use of them.

However the processes in those last five years haven't changed.

1. Identify the Need

First step on any part of your project should be to look at the data on what needs improving. This means doing the part of the process that looks at the analytics. Working out where the problems are arising based on your data.

That means looking at conversion funnels (in Google Analytics or Omniture or whatever) and working out where the pain points are. By going from your macro analysis to your micro analysis you should be able to find places where changes would improve things.

Of course (and typically) many times identifying the need is based around Hippos. This is where someone high up in the organisation says "I think this would work better, let's put it on." Or where there are disagreements between stakeholders on the best way to put something forward.

In these cases you can definitely step in and get your users to decide which version works better.

2. Decide what sort of test you want to do

Effectively you can do three different sorts of tests and you need to choose at around about this point which one is going to be the best option for you.

A/B/X test. These tests involve you putting forward one or more versions of a part of a page to test to see which one works. This might be a button, it might be a banner or it might be three quarters of the page.

Multivariate test. These tests involve you putting forward one or more versions of more than one part of a page to test to see which combination works best. This can be buttons, banners, headlines, copy, etc.

Split URL test. These tests are somewhat similar to an A/B/X test, but usually you have radically different pages, rather than just a different element within a page.

3. Design decisions

Doing A/B or Multivariate testing isn't an excuse to cut back on design work.

You should do all the usual things that you would normally do when designing a page (or a journey). That means doing wireframes, looking at branding, making sure the colours fit with each other, the text is consistent you've thought about scent and your persuasion architecture.

What you should definitely do, if you have the budget available to you, is to test your designs with real users. I remember being at a conference and hearing someone (whose face escapes me at the moment) who suggested that at worst you should give it to your mum to have a look at.

4. Choose your conversion point(s)

Arguably you should do this before you start designing, because you would use it in your design decisions, but having come up with your designs then you need to work out what it is you want your users to do.

This may come back to your conversion funnel, but it is an important stage in the process. Too often the you see the conversion metric being click through to the next page, when in reality you want people to continue through to a sales page or sign up to a newsletter or whatever. If your metric is click through rate then the best solutions will be "Free Money" in large letters.

Having come up with your key conversion points you might want to rank them so that you can decide if one of them is increased and another is decreased. Or you could apply values to each one so that when you add them up you come up with a good statistical model. This is always easier when there is a real monetary value associated with the conversion, but it doesn't mean that you can't give arbitrary values (eg 5 for a Newsletter sign up and 10 for a responding to a classified).

5. Choose Your Tool

There are literally hundreds and you could pay anything from nothing to a fortune.

Choosing a tool depends on a whole host of things, including which Analytics tool you use (you want the integration to work), how complex the changes to the site are, how many tests you think you are going to do and how many people you want to be included in the test.

As a good starting point you have two cheap tools:

Google Analytics Experiments (nee Google Website Optimiser) - this is a great little tool if you have a simple test and only one conversion point. You put the two pages live and the JavaScript on the original page and off you go. Great for simple sites, but anything with more than one conversion is going to be scuppered.

Visual Website Optimiser - this tool is what we usually use as the entry point for a site with smaller volumes of visitors. You can do all the testing types as above, you can do it to certain user types and you can have more than one conversion point set up. We've also set it up to send which version is being shown into Omniture as well as Google Analytics (and you can probably do it with WebTrends and all the other analytics tools too).

Maxymiser or Autonomy (nee Optimost) - these are more than just tools. They tend to be managed services that help with all of the processes above. They're good for companies that know they need to do something, but don't have the analysis or design skills.

Test and Target or WebTrends Optimize - these are the enterprise level versions of the tool that integrate seamlessly. You use these if you are spending money with those companies already, have large numbers of page views. They can of course both do targeting as well as testing as a primary function of the tool. These are designed for companies who want to do optimisation on a long term plan, rather than a one off project.

6. Test!

The last step is for you to just start testing! Get enough people through your pages that you can statistically say which gives the best conversion rates. Then iterate and do it all over again.

What should managers do with their Analytics function?

Whilst at Measure Camp a couple of weeks ago I attended one session entitled 'Does anyone outside this room care about Web Analytics?' At it, I posited that many people's dissatisfaction with organisations was just a misinterpretation of a maturity model put forward by Stephane Hamel: they were actually just seeing the impact of deficiencies in other areas and that managers were getting the brunt of it. I was, however, rebuffed on this somewhat. Whilst discussing this in the office a couple of days later I realised the reason why. Managers would not be doing their job properly if they were the most advanced of all the disciplines.

So here we have the big guide to how you should manage analytics if it is your responsibility. Previously when talking about the maturity model, I did a post on the sorts of things that I thought you should know if you were a Web Analyst.

This time I'm not going to assume you know anything about analytics, but I'm going to assume you can work out where you are as an organisation by putting into place the model. But before we do that, here are the basics of the maturity model:

The model takes six facets of analytics that would lead an organisation to be able to optimise in the most efficient manner. You then work out how mature you are in each area, on a scale of 0 - 5, and it leads you into a little graph like the one above.

If you are equal in all areas then you are optimising in an efficient manner, but you could be getting better value for your money by being more mature. If one of your arms is more mature than the other then you will be wasting money by not being able to use it to its full potential.

The classic examples are: buying a tool that is all singing and all dancing whilst not having anyone in place to look at the data; or not having the processes in place to enact the recommendations that your tool and experts recommend because you need to wait 12 months for the next development cycle.

So what are the six facets?

1. Are you optimising with respect to your business objectives? Whilst this seems like a yes/no question, rather than a 0 - 5 scale, but really it isn't. You can go from nothing to being simple channel objectives, or overall organisational objectives.
2. Do you have the processes in place to make the recommendations that the analytics suggests a reality? This is a a scale from a large waterfall process through Agile to multivariate testing on the fly (and it doesn't just have to refer to the technical side, but also to your marketing or organisational processes)
3. Do you have tools in place to give you the right levels of data? We quite often call this 'Data and Tools' because you can have a great tool that gives you crap data. The scale here goes from nothing through basic tools, basic reports through to fully integrated report sets for all levels of the organisation.
4. Do you have people on the ground to do the analytics? Avinash always recommends the 90/10 rule (for every £10 you spend on technology, you should spend £90 on people doing stuff with the data). The scale runs from nobody, through part time work, through multidisciplinary teams to experienced and empowered individuals.
5. What are the things that you do analytics on? This can start as something as simple as a single tactic (eg email) through to the whole web presence and finally all the way up into multichannel.
6. Finally, where does the responsibility for this sits in a management team? Does it sit with a project manager or does it go all the way up to the top of the organisation.

You as a manager should work out where you are in each of these sections and then evaluate your own presence too.

You should always try and be even in all areas and you as a manager's job is to ensure that you bring each of the other areas up to the appropriate level.

But equally as a manager you should know that you shouldn't waste your time fighting battles that you're not going to win and transformation frequently is one of those. Therefore as a management team you should ensure that you are equal lowest of all of those.

I'll repeat that - to be a good manager, if you have hold of analytics you should be equal worst in this list.

This is the reason that so many analytics people end up complaining about management. It isn't so much that management are holding things back, it's that they know if it wasn't them, then something else would be the road block in an almost identical fashion.

So you're a manager and you want to work out where everything is on that scale. How do you do it?

1. It would be remiss of me, as the head of a function at a consultancy that I didn't say 'Hire a consultancy'. But I think this is a route that only people who are very serious about implementing recommendations that a consultancy make. These changes are likely to be organisational and they are likely to impact your job. The price won't be cheap. What you will get is a company that will 'interview' various members of your team and those around you to find out what you do on a day to day basis. They will look at the implementation of your tools. They'll look at your reports. They'll look at the decisions that you make from those reports. Then they'll workshop what you should do to bring yourself up to a level playing field. It will have credibility because all teams will have been involved in setting up a series of actions and a timeline of when you want them to be in place.
2. You can do the above in an internal manner. You'll get a biased view and you may lose credibility of the results because they'll seem to be sponsored by one part of the organisation.
3. You can do a simplified version of the above. I quite often do this with teams that I work with - we'll sit in a room with a couple of people who work with a client, I'll go through the levels and we'll work out where we think an organisation is. Then we'll come up with a couple of simple ideas that will push the organisation up in an area we think they're deficient in. It stops us just repeatedly offering the same service over and over where it might not be applicable.
4. You can listen to snake oil salesmen whose job it is to sell you their product (I jest, but quite often you'll find companies try and push a product, particularly tools, when the reality is that isn't the reason that you aren't taking advantage of what you currently have).

Is the ball in your court? Yes, because you're the one in charge of analytics. Don't expect it just to happen.

Which tag management solution should you buy?

A couple of weeks ago we had a Web Analytics Wednesday in London about tag management (don't forget that I presented one once!). The highlight of this for me (I was coming down from a nice two week holiday on the slopes and the bars of Andorra, so I wasn't my usual self!) was a quote by the Tagman representative that most of the tools were remarkably similar, but they all had different outlooks on where they thought the industry was going to be in five years time. What you should do, is go to their websites and work out if where they are going matches where you are going as a business.

You can tell where this is going, can't you? I'm going to help you with this by going to the websites of all six panellists (and maybe Google Tag Manager too) and write it out for you. All you have to do is work out where your business is going to be in five years time.

Tag Management isn't dead according to the panel, despite Google entering into the arena. They think that Google will stick to Google products (we'll come back to that later). But, they do think that customers need tag management tools to make it easier to change and add tags to their website. Given that, you're probably going to hang on to a tag management tool longer than an analytics tool (or a remarketing tool, or a affiliate tool, etc), so they have a slightly longer term focus than the Analytics tools that you use. Working out where you are going to be in 5 years could be difficult though (5 years ago the world was very different) - but this is a good indication. I'd still recommend talking to many vendors if you are going to decide which to go with though.

Tealium

Tealium come from a web analytics background. The two founders were WebSideStory guys originally (if you don't know who they are you should check out my post on where is your analytics tool now) and they have a large backing from venture capitalists in the US. Generally they are considered as the market leader - they have their own conference and a 'university' to teach people how to use the tool.

I'm sure that I'll get shot down for saying this, but Tealium is aimed at people who have problems with their Web Analytics tools. Do they do video and mobile? Yes, but that is just a side affect of doing web analytics. Are they interested in privacy? Yes - probably more so than your average Web Analytics provider. Do they think it will improve technical performance of page load time? Yes - that is a big benefit. They view themselves as being inclusive of all tags, meaning that you aren't limited to who you can swap for. And finally they allow you to do server side analytics if your site (or whatever it is you are tracking) doesn't allow JavaScript.

BrightTag

BrightTag go for the data approach, but their background is ad tags. The annoyance of adding yet another affiliate tag to the site when you strike up a deal means that a tag management solution is ideal.

This data approach is interesting though - rather than have an approach of trying to work out what you want to collect and then send it to a particular tool, they do it the other way around. They tell you everything you have and then you choose where you want to send it. This alternative approach means that instead of adding a new tag to a site, you are effective providing the old data to a new company. It's an abstract way of looking at the same thing.

The impact of all this is that rather than keeping a handle on which tags you are putting in, you need to keep a handle on which data you have. This is great if your site is simple, but if your site is more complicated and changes in structure frequently over time then it could cause issues.

Ensighten

Ensighten CEO Josh Manion came from a web analytics consultancy background, so you can see why he had the focus he did when he set up Ensighten. Early tag management solutions were slow and caused lag in page load times or didn't capture the right data.

Ensighten is all about speed. It speeds up the loading of the tags on the pages so that your users don't get pissed off. This might sound like something out of the 14th century, that isn't the case with things like mobile networks where it can still be important to ensure that your tags work before the user leaves the page and if necessary the tags put the advertising up too.

Tagman

Tagman's current raison d'etre is about attribution (or at least, that is what is slightly different about them as opposed to the others).

Seven years ago whilst I was working for a famous insurance company, we tried to implement an attribution system that would allow us to pay affiliates and so on, only if they were one of the last five referrers. That meant coming up with a solution that would only send off the affiliate tags on payment page if they should be.

This is the sort of thing that is going to be increasingly important in the future for sites that have frequent turn arounds on visitors. You don't want an affiliate to be allocated money for sales number two, three and four in a month by a person if your own remarketing has persuaded the user.

TagCommander

If Ensighten is about speed of pages, TagCommander is about speed and ease of deployment. It is also a European solution, which is often preferable for European brands.

When I say ease of deployment, the container in all these cases is easy to deploy in equal parts. The difficult bit is the interface and putting the tags on the pages. TagCommander goes for a four click process for getting a new tag on the page. If you've dealt with Google Tag Manager (why should I need to know regex to make a tag live?) then you'll know why having a simple solution makes the most sense.

DC Storm

DC Storm are a football team from Washington. Not really, they're a tag management solution from Brighton.

DC Storm look at the world from a data point of view as well. Where BrighTag could probably be called a taxonomy of your site that allows you to point data at will, DC Storm take the data into their systems first and then you work out how to disseminate it from there.

This is an approach that some will feel will allow you to be all inclusive of all sorts of channels ('Big Data') and not just traditional web based data. This allows you to lots of attribution and so on outside of your traditional reporting mechanisms.

Bonus: Google Tag Manager

It's USP, unlike the others, is that it is free.

Tag Management is a relatively new industry. Many of these organisations that you see above haven't been going that long, so you'd expect this space to change relatively rapidly. This isn't great for a company that you are going to want to stick with for five years.

Watch this space.