January 29th, 2007

Engagement is (not) made to measure

Measuring ‘engagement’ is like eating an elephant: it’s a big job and you’re not sure where to start.

Photo by Alice Creative Commons licenceI’m no exception, and my thinking on this topic still feels heavy and a little clumsy. So, please indulge me for a moment…

At the start of the year, I wrote:

The page view is dead, long live, err… something else! Hmm… web metrics just do not cut it (and just when you’d got to grips with it!). But what should we be looking at now? In 2007, the sector needs to identify new measures of ‘engagement’ online. This work is urgent, especially as charities need to show accountability for everything they do.

In one sense, this may seem a pointless exercise – preparing to get the tape measure out as the social web gets widgetised, atomised, and more distributed.

But engagement was a key theme explored at the Future of Information Summit ’07 presented by Experian recently. Last month, a Factiva roundtable reached to figure out how to measure social media the best way, and Robert Scoble (no less) had already added his call for a new metric for engagement.

I’m equally aware that some people do not care for the term, ‘engagement’ (possibly because of all this attention). Anyway, for want of anything better, I’m sticking with it for now. More importantly, a lot of people whom I listen to in the sector are using the e-word. So there.

So, why all the talk about social media measurement? Well, it’s one thing to have an engaged website, but more and more the action takes place in other places, in existing communities and social networks. Charities must turn from ‘owning’ their cause to enabling networks to run with the ball. Yet again, this was reinforced to me over the weekend after reading Robin’s Hamman’s post about BBC 2.0.

So what are we measuring? Influence? Reach? Audience…?

Brian Oberkirch helped me make some more sense of this conundrum, although he admitted it was tough: “Like nailing down a shadow”…

That’s why I have a bit of trepidation over the rush to quantify and reify ‘engagement’ as the baseline by which all social media work should be evaluated. JKO called these ‘the holy grail’ as part of the discussion, and that’s what is problematic. ‘Engagement’, like ‘conversation’ is one of those terms that feels like it means something, but really is mushy enough for anyone to bend it to their will.

Check out Brian’s excellent post for some things we might want to measure. This certainly goes beyond the standard (and not so standard) toolset on web metrics deployed and listed here by Beth Kanter. To pick out one snippet from Beth’s post:

Metrics alone are not very meaningful – they need to be put into some context. Context to me means outcomes, intent, and audience. No matter what type of metrics you trying to figure out … that’s a universal metric standard.

Outcomes. That’s it. Or “Return on Objectives” (ROO) as my friend Richard Sedley is justly keen on saying.

It takes two to tango

For me, the term ‘engagement’ suggests a two-way street – it implies not simply a ‘connection’, but a reciprocal action. As Mark Ghuneim et al say in their mini-essay on the Wiredset blog, Terms of Engagement: Measuring the Active Consumer

In the traditional sense, engagement is the period between proposal and marriage

True. Many (most?) people will rebuff your advances. Others may be content to donate cash, but not wish to be ‘engaged’ in anything. A few will get mobilised into taking some form of action for your cause.

These ‘degrees of engagement’ (is there a better way of saying this?) remind me of Dick Carlson’s comment on the aforementioned Scoble post.

Dick proposes a four-level model for measuring engagement:

1. Click – A reader arrived (current metric)
2. Consume – A reader read the content
3. Understood – A reader understood the content and remembers
4. Applied – A reader applies the content in another venue

Now, let’s put some meat on the bones – with thanks to Mark Ghuneim for allowing me to reproduce this terrific graphic (original here).

Now we’re getting somewhere.

Work with your buzz director to create milestones and targets for activity for each engagement ‘type’. Roll your findings up into monthly progress reports (which should get as wide a distribution as possible). And remember, ensure what you are measuring is aligned to your organisation’s strategic goals.

The ‘goalposts’ haven’t moved; it’s just that there are now many more pitches on which you must play (a bit like Hackney Marshes on a Sunday morning).

* * *

Technorati engagement, nptech, nptechuk, social media measurement, web metrics

AddThis Social Bookmark Button

6 Responses to “Engagement is (not) made to measure”

  1. DemocracyInAction Blog Says:

    Metrics Meretricious and Meritorious…

    Amazingly, even marketers under the straightforward whip of a dollars-and-cents ROI struggle to understand what an e-mail address is really worth….

  2. Ron Mader Says:

    Stunning essay. It makes me wonder … how do we measure ‘engagement’ that has a foot both in the natural and online worlds?

    Engagement matters more than traffic, indeed!

  3. philanthropyOz Blog » IT in the Nonprofit Sector Says:

    [...] ‘Engagement‘ is a catchphrase that seems most relevant to fundraising organisations seeking to ensure donors return frequently to their website, but the general gist of the term is just as relevant to philanthropic funders aiming to share information and encourage social change. Wiredset’s ‘Type of Engagement’ diagram is a great, more generic snapshot of just what having an ‘engaging website’ will mean for you; and nfp 2.0 analyses just what ‘engagement’ entails and suggests just where to start when considering it. [...]

  4. Craig Says:

    One of the difficulties with measuring engagement is attempting to do so from the outside – clicks and contributions – rather than from the inside – user or audience involvement. Engagement, or a similar type of term, is very important if you are looking at behavioral outcomes that occur AFTER people view/hear the content. My thinking right now is that it requires a combination of objective and subjective data to get at a truth about engagement. Some of the work I am doing was described here with more to come.

  5. Ian Wilker Says:

    Steve — missed this when you wrote it, but found it looking for answers to a client’s questions about ROI on social-web efforts. This is a tremendous post — thank you!

    I wish there was a list-serve for community managers, “buzz directors,” all of us working this side of the street. But then again, I guess there’s always feedreaders and comments… ;-)

    – Ian

  6. nfp 2.0 » Buzz Director: help me write a job description Says:

    [...] Engagement is (not) made to measure [...]