Tag Archives: feedback

AARRR! Pirate metrics vs survey fatigue – what’s a business to do?

18 Mar

Pirate Metrics Cat

I spent much of the past three weeks trying to convince people – friends, random strangers, even people in LinkedIn Groups (I know, I know) – to fill in a survey about their news consumption habits. I cajoled. I implored. I offered dinner, drinks, lunch, babysitting. I received more than one “will I win the new iPad?” query.

And I felt utterly hypocritical throughout.

Because I loathe surveys. I despise forms. I’ve abandoned sign-up processes because the data requirements were too onerous. I angry-click closed those cheery pop-up boxes asking for my feedback. I deeply resent every time I get an email from an online retailer asking, “How did we do?” [In a not unrelated development, I tend to fill in surveys only when I am really, really pissed off about the quality of service received.]

A plague of surveys. An outbreak of requests for feedback.

But why? The NY Times offered the following explanation for the survey epidemic in a piece on March 16:

One reason is that software companies like SurveyGizmo and QuestionPro have made it possible for small companies to create customer surveys at a fraction of the cost of traditional surveys done by established research companies. Businesses of all sizes, desperate to lock in customer loyalty, see surveys as a window into the emotional world of their customers and a database that will offer guidance on how to please them.

I could blame Eric Ries and the rise of the “lean startup” philosophy, one of the central tenets of which is eschewing so-called vanity metrics (like page views and followers) in favour of actionable metrics. I could also blame Dave McClure, who in 2007 unleashed pirate metrics onto the world.

Here’s more from one of McClure’s blog posts in 2007:

The basic concept is based on 5 types of measurements of user behavior:

A: Acquisition – where / what channels do users come from?
A: Activation – what % have a “happy” initial experience?
R: Retention – do they come back & re-visit over time?
R: Referral – do they like it enough to tell their friends?
R: Revenue – can you monetize any of this behavior?
(… otherwise known as “AARRR!”, and thus the “Pirate” designation 😉

But I shall save the blame for one Frederick F. Reicheld and his 2003 Harvard Business Review article, “One Number You Need to Grow“.

Cue dramatic irony:

Dramatic Irony

Reicheld argued that a single survey question – “would you recommend this company to a friend?” – was a company’s best predictor of revenue growth. This so-called “net promoter score” has been embraced by corporate behemoths (including Apple) and lean startups alike.

The third R of McClure’s model – “referral” – is just net promoter score with an eyepatch and a peg-leg.

Cue relevant anecdotal evidence:

Grub Hub hopes I like them (I do)

Companies love this kind of data. But the consumers who provide it? Not so much.

Here’s the NY Times again:

Consumer patience may be fraying under the onslaught. The constant nagging has led to a condition known as survey fatigue and declining response rates over the last decade.

The NY Times said companies are throwing money at the problem:

On their register receipts, stores like Walmart, Petco and Rite Aid include a Web address and an invitation to fill out a survey, with the chance to win a prize. At Staples, the prize is a $5,000 store card.At Staples, the prize [for filling out a survey] is a $5,000 store card.

From AARRR to aargh.

Would You Recommend Us? – Businessweek, 2006
What’s Wrong With the Net Promoter Score – ClickZ, 2009
What Do Companies Like Amazon, Virgin America, Apple and Trader Joe’s Have in Common? – Daily Disruption
The Order of AARRR – Market By Numbers

Elsewhere on Ent!
qualitative feedback is most effective when it’s overwhelmingly negative.
in-home research visits are key to Facebook’s continued success.


Pissed off users = progress

11 Feb

“If you create a platform on the internet, if your users start screaming in rage on Twitter or in their blogs about something you’ve done, rejoice. You matter. You are successful vs. your 50 potential competitors who no one cares about enough to complain. Now, listen. Respond. Fix it.”

Just because you can make money from something doesn’t mean you should, and other rules of the web | Rex Hammock’s RexBlog.com