Archive | March, 2012

AARRR! Pirate metrics vs survey fatigue – what’s a business to do?

18 Mar

Pirate Metrics Cat

I spent much of the past three weeks trying to convince people – friends, random strangers, even people in LinkedIn Groups (I know, I know) – to fill in a survey about their news consumption habits. I cajoled. I implored. I offered dinner, drinks, lunch, babysitting. I received more than one “will I win the new iPad?” query.

And I felt utterly hypocritical throughout.

Because I loathe surveys. I despise forms. I’ve abandoned sign-up processes because the data requirements were too onerous. I angry-click closed those cheery pop-up boxes asking for my feedback. I deeply resent every time I get an email from an online retailer asking, “How did we do?” [In a not unrelated development, I tend to fill in surveys only when I am really, really pissed off about the quality of service received.]

A plague of surveys. An outbreak of requests for feedback.

But why? The NY Times offered the following explanation for the survey epidemic in a piece on March 16:

One reason is that software companies like SurveyGizmo and QuestionPro have made it possible for small companies to create customer surveys at a fraction of the cost of traditional surveys done by established research companies. Businesses of all sizes, desperate to lock in customer loyalty, see surveys as a window into the emotional world of their customers and a database that will offer guidance on how to please them.

I could blame Eric Ries and the rise of the “lean startup” philosophy, one of the central tenets of which is eschewing so-called vanity metrics (like page views and followers) in favour of actionable metrics. I could also blame Dave McClure, who in 2007 unleashed pirate metrics onto the world.

Here’s more from one of McClure’s blog posts in 2007:

The basic concept is based on 5 types of measurements of user behavior:

A: Acquisition – where / what channels do users come from?
A: Activation – what % have a “happy” initial experience?
R: Retention – do they come back & re-visit over time?
R: Referral – do they like it enough to tell their friends?
R: Revenue – can you monetize any of this behavior?
(… otherwise known as “AARRR!”, and thus the “Pirate” designation 😉

But I shall save the blame for one Frederick F. Reicheld and his 2003 Harvard Business Review article, “One Number You Need to Grow“.

Cue dramatic irony:

Dramatic Irony

Reicheld argued that a single survey question – “would you recommend this company to a friend?” – was a company’s best predictor of revenue growth. This so-called “net promoter score” has been embraced by corporate behemoths (including Apple) and lean startups alike.

The third R of McClure’s model – “referral” – is just net promoter score with an eyepatch and a peg-leg.

Cue relevant anecdotal evidence:

Grub Hub hopes I like them (I do)

Companies love this kind of data. But the consumers who provide it? Not so much.

Here’s the NY Times again:

Consumer patience may be fraying under the onslaught. The constant nagging has led to a condition known as survey fatigue and declining response rates over the last decade.

The NY Times said companies are throwing money at the problem:

On their register receipts, stores like Walmart, Petco and Rite Aid include a Web address and an invitation to fill out a survey, with the chance to win a prize. At Staples, the prize is a $5,000 store card.At Staples, the prize [for filling out a survey] is a $5,000 store card.

From AARRR to aargh.

Related:
Would You Recommend Us? – Businessweek, 2006
What’s Wrong With the Net Promoter Score – ClickZ, 2009
What Do Companies Like Amazon, Virgin America, Apple and Trader Joe’s Have in Common? – Daily Disruption
The Order of AARRR – Market By Numbers

Elsewhere on Ent!
qualitative feedback is most effective when it’s overwhelmingly negative.
in-home research visits are key to Facebook’s continued success.

Advertisements
Aside

“qualitative feedback is most effective when it’s overwhelmingly negative. A strong negative signal…”

15 Mar

“qualitative feedback is most effective when it’s overwhelmingly negative. A strong negative signal indicates that your assumptions most likely won’t work and lets you quickly abandon or refine it. If 5 out 5 customers tell you they don’t have a problem, that’s pretty significant!
On the other hand, a strong positive signal doesn’t necessarily mean it will scale or that the customer isn’t lying. All it does is give you permission to move forward until that can be verified later through quantitative data.”

Build Your Lean Startup Through Conversations

Aside

‘Emotional attachment to a vision shouldn’t override the messages the market is sending you.’

13 Mar

‘It turns out that “never give up” is a dangerous lesson to take to heart with start-up companies. Emotional attachment to a vision shouldn’t override the messages the market is sending you. When the feedback from the market is not positive, when customers are not buying, when you have tried to pivot several times and you still haven’t gotten a foothold, it is wise to make a real and carefully considered evaluation and decide if your vision truly makes sense.’

Face It: You’re Clinging On to a Zombie Start-Up | WSJ

Aside

“in-home research visits are key to Facebook’s continued success. “A company can be very strong…”

12 Mar

“in-home research visits are key to Facebook’s continued success. “A company can be very strong technically,” she explains, “but without that connection to what people want, it’s difficult to build loyalty. In the end, it’s all about people. We don’t want to think about users. We want to think about people.””

Inside the Facebook Research Team’s In-Home Visits | Facebook

Aside

“the single most important thing product owners can do to add value to their team, their product and…”

12 Mar

“the single most important thing product owners can do to add value to their team, their product and their company is a user test”

The Most Important Thing

Elsewhere on Ent!
Takeaways: Jason Schwartz’s ‘How Product is Built’ class at GA

Takeaways: ‘Managing Startup Teams: Building Culture’ with Anne Libby at GA

12 Mar

Class: Anne Libby of Anne Libby Management Consulting on “Managing Startup Teams: Building Culture”, via GA

Date: March 6 2012, 18:00 – 19:30

When it comes to building teams – especially when those teams consist of people in far-flung offices around the world – I’m yet to be persuaded that anything is more important than culture*.

And by culture I mean the shared stories, the common values, that explicitly articulated and implicitly understood characteristics that inspire people to rally around that cause, that product, that service. The why-you-want-to-work-here-when-X-would-pay-you-twice-as-much. The-this-is-why-I-get-up-in-at-dawn-on-a-day-when-I’m-not-even-supposed-to-be-in-because-the-team-needs-me-and-hey-they-didn’t-even-have-to-ask factor.

I’ve worked on two projects that I may claim to have helped build and shape; both of these projects had quite distinct cultures.

But what they had in common was excellence über Alles. And yes, that included being excellent to each other (or put another way, jerks need not apply).

Two of my favourite blogs both recently featured culture-as-it-relates-to-startups as a theme, and I thought I’d highlight them as the lesson is relevant to the takeaway from the General Assembly class (which was, not surprisingly, all about culture). Continue reading

Takeaways: Jason Schwartz’s ‘How Product is Built’ class at GA

11 Mar

Class: Jason Schwartz of Matchbook on “Introduction to Product Management: How Product is Built”, via GA

Date: March 5 2012, 20:00-21:30

[This session was the third in a series of four; I missed the first two due to travel and schedule madness, but I’m attending the fourth on March 19]

Jason Schwartz, a product manager/UX specialist and the founder of Matchbook (a location-based app), offered a high-level and useful overview of the science of product management. Some highlights from the class are below.

Continue reading

Takeaways: Alex Taub’s ‘Intro II Business Development & Partnership for Startups’ Skillshare class

4 Mar

Class: Alex Taub of Aviary on “Intro II Business Development”, via Skillshare at Aviary HQ

Date: February 29 2012, 20:00-21:45

If you comes from a traditional journalism background (caveat: I don’t), “business development” (BD) is a dirty word, right up there with “sales”.

And in fairness, if you come from a traditional journalism background, you’re unlikely ever to have to engage in either.

But for all those newsies making the leap from working in a newsroom to working for themselves – and yes, that includes freelancers – or in a startup environment, understanding business development (what it is, and why it’s important) is crucial.

Alex Taub, who leads business development and partnerships at Aviary, defined BD thus:

BD is about building and maintaining relationships, developing your company, and finding a point of transaction and turn them into repeat occurrences.

(For the difference(s) between BD and sales, see this post at false precision, this pithy explanation by Lincoln Murphy and this Q&A at Quora)

Continue reading

Aside

“Under promise and over deliver TO YOUR TEAM. Worst thing you can do is let them down emotionally by…”

4 Mar

“Under promise and over deliver TO YOUR TEAM. Worst thing you can do is let them down emotionally by telling something was closed when it wasn’t. – Alexander Taub”

Notes From a Skillshare Class: Introduction to Business Development and Partnerships

Cannot stress this enough. Would put in both all caps and bold if I thought it would help. Seriously, having been on the other side of this (i.e the victim of an over-promise/under-deliver), it is awful, demoralizing and frankly, dishonest.

Aside

“you’ve probably asked a customer for a “ballpark price” at some point. Well, that’s just backwards….”

4 Mar

“you’ve probably asked a customer for a “ballpark price” at some point. Well, that’s just backwards. Think about it for a moment. There is no reasonable economic justification for a customer to offer anything but a low-ball figure. They may honestly not know and this question only makes them uncomfortable.”

Don’t Ask Customers What They’ll Pay. Tell Them.