Effective customer feedback programs need to consider the depth and breadth of the data they collect:

• Depth is the amount of data the survey collects on each individual participant. Depth is important for understanding what's driving the customer's opinions and making the feedback actionable.
• Breadth is the number of customers who participate in the feedback program. Breadth is important for generating statistically meaningful results and calculating accurate metrics.

Deeper surveys generally cost more than shallower ones, because getting the additional feedback from each participant usually requires a live interview instead of an automated survey. A program's budget will determine how much depth and breadth you can afford.

The most effective feedback programs provide a balance of both depth and breadth. At the extremes, focusing entirely on either depth or breadth often represents wasted effort:

• Narrow and deep feedback gives a very detailed look at the opinions of each participant, but the budget normally won't allow a statistically meaningful sample. For example, a focus group or usability test where you physically bring participants to a study center and interview them at great length will give fantastic insights and generate many ideas for improvement. But this can cost hundreds to thousands of dollars per participant, making it financially impossible to include the hundreds of participants necessary to gather meaningful statistics. So it's impossible to know with certainty whether the insights you get are problems common to all customers, or represent the quirks of the individuals who happened to participate.
• Broad but shallow feedback is the opposite. Automated surveys are nearly free so it's possible to collect feedback from thousands or even millions of customers. However, many consumers today have little patience for these surveys, so its not realistic to expect to get answers to more than a handful of questions (and response rates tend to be poor). Getting thousands of people to answer the same questions is fantastic for precisely calculating the statistical results for those specific questions, but everything else is pretty much speculation.

There's a common fallacy that you don't need depth if you have enough breadth, and vice-versa. This is mistaken: getting more detailed information from one person tells you nothing about the opinions of people you didn't talk to; and asking the same questions of more people doesn't give any information about the questions you didn't ask.

Balancing depth and breadth takes more thought than just doing a bunch of surveys, and it may require collecting feedback in more than one way. But it's the only way to make sure you're getting reliable and actionable results from a customer feedback program.

Apple is not the biggest company in the mobile phone industry, as measured by metrics like employees, customers, and revenue. But it is far and away the most profitable--to the extent that the company actually makes up the bulk of the profits of the entire mobile phone industry. According to today's Wall Street Journal, Apple is so insanely profitable that it is distorting the stock market as a whole.

How does one company manage to suck the profits out of an entire industry?

I think the answer is obvious: every other major company in the mobile phone value chain is focused on winning business. Apple is focused on winning profitable business.

So where other handset manufacturers and the carriers are beating each up on price and trying to outdo each other on feature-by-feature comparisons, Apple is delivering an entire customer experience. Apple's experience isn't just about the hardware: it covers everything about the phone including hardware, software, support, upgradeability, battery life, and the list goes on. Even the experience of removing a new iPhone from the box is meticulously designed.

Not every customer wants what Apple is selling, but a lot of them really want it and are willing to pay a premium (or put up with a carrier they don't like) to get it. As a result, the most profitable business flows to Apple and the carriers feel obligated to make crazy deals to get the iPhone or lose customers.

I don't think there's anything magic about Apple's formula. But it requires a company to focus on what it really does well, think about the whole customer experience, and not get distracted trying to respond to every ripple in the marketplace. That's very had to do in our business culture where the goal is usually profitability measured in three-month increments and the mantra is "win every battle at any cost."

Doing a deep dive on the National Customer Service Survey on Banks we recently discovered a startling fact: in our survey, of customers who reported a problem with their bank's IVR, 20% of the time the problem was somehow related to authentication. For example:

• Nonsensical Authentication: The customer was required to enter an account number to find the hours of a branch.
• Impossible Authentication: The customer was calling to report a lost or stolen card, and did not have his account number readily available.
• Technical Failures: The customer gave a valid account number and PIN, but the system didn't recognize it--either because of a speech recognition problem or because of some other failure
• Prospective Customers: A prospective customer wants information, but is asked to enter an account number first.

In all, these problems affected about 5% of all the people calling one of the four large banks in our survey. 5% may not sound like a lot, but it represents tens of thousands of customers every day who have to call back, don't get what they want, and have a negative view of their bank as a result. It also represents millions of dollars in expenses to these four companies due to repeat customer calls and unneeded time with a live agent.

So why does this happen? I believe it's because these IVR systems were designed from the company's perspective, not from the customer's perspective. The company decided that it would be helpful for them to know the customer's account number, and so they ask for it even when it makes the customer's experience worse.

The solution is to ask how the company can best serve the customer, not how the customer can be helpful to the company.

Automated customer service surveys (via e-mail, web, or IVR) have one huge virtue: they are cheap. Big companies can afford to collect feedback from millions of customers, creating a broad statistical profile which can be used to measure customer satsifaction and other key metrics with a lot of granularity. However, automated surveys often suffer from poor response rates, and can usually only have a handful of questions before customers start abandoning the survey. That severely limits the quality and depth of the feedback.

Phone interviews, on the other hand, are almost exactly the opposite: customers are much more willing to participate when there's a live person on the other end, and they are willing to answer a lot more questions as long as the interview is well-structured. Interviewers can also probe and go in depth in a way a computer can't. But it's inherently more expensive to conduct a live interview, and there's just no way to escape that fact.

In some companies I advocate for a hybrid approach, combining the vast ocean of customer feedback afforable with an e-mail or IVR survey, with the depth and reliability only attainable through inverviews.

This takes advantage of the fact that in most companies, most customers have a reasonable experience most of the time. If a customer was satisfied with the experience, there's usally little to be learned about how to improve things. On the other hand, if the customer was not satisfied you can get a lot of useful information. Running both an automated survey and an interview process in parallel focuses the survey dollars and effort where it is most effective.

The automated survey is used to establish high-level statistics across the organization and identify underperforming segments. The live interviews are used to generate actionable feedback about how to improve. Customers are selected for one survey or the other based on whether the statistics suggest a high probability of dissatisfaction.

The end result combines the best of both worlds: tons of high-level statistics across the entire organization, and highly detailed in-depth feedback where the company needs it most, all delivered in a coherent package to the people who need to make both strategic and tactical decisions about how to deliver the customer experience.

In case you missed it, we issued two news releases this week: NCSS Mobile Phone results, and NCSS Tech support results.

AT&T also issued their own news release after ours came out. We didn't have anything to do with that, but we are glad to see them paying attention.

We've published Issue 53 of Quality Times, Vocalabs' newsletter.

In this issue we discuss the 2011 results for the National Customer Service Survey in mobile phones and computer tech support. These reports compare customer service quality at AT&T, Sprint, T-Mobile, Verizon, Apple, Dell, and HP.

Today we published two new reserarch reports for the National Customer Service Survey:

The National Customer Service Survey for Mobile Phone Customer Service Executive Summary covers quarterly trends at AT&T, Sprint, T-Mobile, and Verizon.

The National Customer Service Survey for Computer Tech Support Executive Summary covers technical support at Apple, Dell, and HP, showing trends on a semiannual basis.

These reports, along with many others, are available for free download on our Published Research page

A few weeks ago I linked to this graphic from Bloomburg, purporting to show all sorts of strange relationships such as that the U.S. housing bubble was caused by too many babies named "Ava."

The real message, of course, was to beware of statistical correlations: just because two things appear to be correlated does not mean that one causes the other or that they are even related. If you look deeply enough you can always find some bogus way to correlate two statistics.

I was remiss, however, to not link to the classic chart from 2007 purporting to show that pirates prevent global warming. Since the corelation-is-not-causation message can't be repeated too often, I'm linking to it today.

One thing every company has to deal with is the small number of people who want to rip the company off. From the employee slipping a few bills from the till to the customer who tries to get a refund on the torn and stained wedding dress, there is a tiny minority who will lie or cheat to get something they don't legitimately deserve.

So every company has processes and policies to keep dishonest customers from taking advantage of it. You can think of this as an organizational immune system: identifying the potentially harmful customers and transactions and stopping them in their tracks.

There is a cost to this. In addition to the direct costs of implementing systems, training, and policies, there is the problem that no system will ever be perfect. There will always be someone who find a way around the system, and there will always be the unlucky customer who really did get three defective products in a row and doesn't appreciate being treated like a thief.

It's hard to strike the right balance between a strict set of rules--which will frustrate legitimate customers with real problems--and a more permissive attitude which would benefit the bad guys.

I see a lot of companies which are far more worried about the slight chance a customer might get somthing they don't deserve than the chance a customer might go away annoyed or frustrated. For example, if a customer calls to complain about a $1 billing error, it costs far more to spend any time investigating the mistake than it does to just give the customer the buck. But try calling your phone company or credit card company to compain about a $1 mistake and see how much time they spend making sure you really deserve that dollar.

If the systems to prevent beign ripped off are like a corporate immune system, then irrationally strict policies are like an auto-immune disorder. It's the corporate immune system mistakenly assuming that legitimate customers are bad guys and attacking the wrong problems.

Treating customers as the enemy is no way to win long-term loyalty.

Smart Money has a "Ten Things" list today about customer service. The article is a sad recitation of many problems industry insiders have known about for a long time: getting a problem or complaint resolved is hard, companies often care more about saving money than helping customers, and so forth.

I think the most insightful quote in the article is right on the first page where Emily Yellen says that customer service is designed as a firewall against the customer. This attitude is almost never spoken aloud, but comes across loud and clear in the decisions companies make: design systems to "contain" and "deflect" customers (industry terminology, not mine), provide incentives to handle lots of calls quickly rather than well, etc.

This attitude is almost exactly backwards. I believe, as management uber-Guru Peter Drucker once said, that the purpose of a company is to create and keep a customer. In customer service you are bombarded with customers, all of whom have some need to be filled. In the Marketing department they would kill for that.

Unfortunately, because much customer service happens post-sale, the attitude is that it's nothing but added cost. This would be true if most businesses were run like a street bazaar, but they aren't. In nearly every business, nearly every customer represents an ongoing relationship, or at least the potential for one. Every customer service opportunity is also the opportunity to strengthen that relationship.

Businesses prosper when their customers prosper and find value in what the business offers. Business is not a zero-sum game where you win by grabbing every last penny off the table. Companies win when their customers win.

So stop thinking about customer service as a firewall to protect the company from customers. Start embracing customer service as an opportunity to solve the customer's problems and bring added value to the relationship.

I tell anyone who will listen that improving the customer experience is a "process, not a project." In other words, you can't just spend a bunch of time and money improving things and be done. Instead, you have to put in place the systems and incentives to make sure that everyone from the front-line to senior management makes the right customer-focused decisions.

Collecting customer feedback is, itself, a process: it consists of certain well-defined steps taken to achieve the desired goal of gathering opinions to better understand what your customers think.

What's more, a customer feedback survey should have other related processes in order to make sure the survey is as effective as possible:

• Follow-up Process: If a customer asks for a follow-up during the survey (for example, if there's an ongoing problem), the request needs to be handled promptly by a qualified supervisor. The request and the outcome should be tracked and logged, so that all customers who want a follow-up get one, and the root causes can be analyzed for ways to build a more effective service experience.
• Training Process: Customer feedback should be incorporated into ongoing coaching and training sessions for employees, as a tool to help build empathy for the customer and to help direct training towards each employee's strengths and weaknesses.
• Appeal Process: If customer feedback is used for compensation or performance review (as many companies do), then it's vitally important to give employees a mechanism to challenge negative feedback. No survey is perfect, and acknowledging that fact and having a way to discard obviously bad data will improve the credibility of the survey and help engage employees in the feedback process.
• Discovery Process: Every organization is a dyanmic environment, and there should be regular, scheduled reviews to see what is changing and new in the customer feedback. Even if there isn't a known problem with service levels, taking a fresh look will often yield new insights.
• Quality Assurance Process: The customer survey process itself needs to be subjected to a rigorous and ongoing quality assurance process. Speaking as an industry insider, I'm often shocked at how lax some companies are about making sure their surveys are giving the intended results--or even that they work. Automated surveys need to be tested regularly, interviews should be recorded and reviewed, and all customer feedback should be subjected to a "does this make any sense at all?" gut-check.

Not only is improving the customer experience a process, it's a process made up of many other processes. Taking an ad-hoc approach to customer feedback probably won't yield the results you're looking for.

Our initial National Customer Service Survey report on banks was published in October 2011 and covered the six months ending September 2011.

We have updated that report to include all our 2011 data, April through December. This suvey tracks customer service quality at Bank of Americs, Chase, Citi, and Wells Fargo on an ongoing basis, and the updated report includes results of over 1,200 customer interviews from these four companies. Each interview took place shortly after a customer service call, with most interviews being less than five minutes after the end of the customer's experience.

You can download the Executive Summary of the full 2011 data online, or contact us for more details about subscribing to the complete data set.

I was interviewed recently for an article in The Street.com about our National Customer Service Survey research.

It's a fairly extensive conversation which covers msot of the industries we track. You can read the whole thing here: Best and Worst of Customer Support.

I see a lot of examples of bad customer surveys. Most of them are products of ordinary lack of attention and the usual breakage which happens from time to time with technology. Then there's this (click for a larger view):

This survey displays the symptoms of someone trying to be entirely too clever. It's clear that the first row is supposed to represent a range from happy to unhappy.

The second row, with pictures ranging from "struck by lightning" to "asleep" was a little harder. I eventually figured out that it was supposed to range from "excited" to "bored," though "high" to "drunk" could be a viable alternative.

The third row, however, remains a complete mystery. "Small" to "big"? "Distant" to "close"? "Unimportant" to "important"? Even being in the survey business and having some insight into what the company might be asking about, I really can't figure it out. And if I can't figure it out, I doubt many customers guess correctly either.

My hope is that this was part of an evaluation of a new survey form--and that the form was promptly refined or thrown out. A side-by-side test against existing survey instruments is important when straying outside the  familiar boundaries of the usual customer satisfaction questions.

If not, and this is actually the customer survey Avis is using, then I can't imagine what kind of data they're getting.

(Originally spotted on The Daily WTF.)

The latest edition of our newsletter, Quality Times, just went out. This issue talks about the "three abilities" of effective customer feedback: Dependability, Actionability, and Credibility.

I don't think it's any secret that some companies "hide the ball" in customer service, and made it hard to reach a person in hopes that customers will use self-service instead. Most industry experts agree that this is backwards thinking that went out of style with Duran Duran, but it still happens.

But does it work? How often will a customer successfully use self-service despite being frustrated by an inability to reach an agent?

It turns out that as part of our National Customer Service Survey where we interview customers immediately after a customer service call, we have some pretty robust statistics around this question. The answer surprised me, and it might surprise you, too.

The answer is: For all intents and purposes, Never.

Among customers who (a) reported that it was hard to reach an agent AND (b) did not speak to an agent, less than 5% reported that they got what they needed on the call.

The use pattern where a customer starts out trying to reach a person, finds it difficult, then successfully uses the IVR instead does not exist to any meaningful extent. Instead, the customer starts out trying to reach a person, finds it difficult, then eventually gives up in frustration (perhaps to call back later).

So why do so many companies keep making it hard to reach a person? Three reasons:

• They think it should work.
• It looks like it works when customers hang up without talking to anyone (in reality, most of them will probably call back later).
• They don't bother to measure customer opinions about self-service.

It has been a little over ten years since I founded Vocalabs with the goal of giving companies the customer feedback they need to make intelligent decisions about their customer experiences. Back then, our initial focus was on the contact center (where we still do a lot of work, though we now do somewhat more in bricks-and-mortar customer service).

Over the past decade I've noticed a slow but definite shift in attitudes at many companies to how they provide phone service. For example:

When referring to IVR usage:

In 2001: "IVR containment rate" (customers need to be "contained" like toxic waste)

In 2011: "Self-service rate" (customers can choose to serve themselves)

When discussing key agent metrics

In 2001: "What's your average handle time?" or "How many calls per day does an agent take?" (the most important thing is how much expensive talk-time a customer consumes)

In 2011: "What's your first call resolution?" and even "We don't focus on handle time any more." (the most important thing is getting the customer's problem solved)

Design strategies for self-service automation

In 2001: "We want to push more self-service." (self-service should be forced on customers)

In 2011: "We want to understand why some customers don't use self-service." (many customers choose to use self-service. What's stopping the rest?)

While I don't think these changes are universal--I certainly still hear about "IVR containment" as a key metric--I think it is notable how many companies are taking a more customer-centric approach and it is paying big dividends. I was at a conference a few months ago where several large companies said they had stopped using average handle time as a key agent metric to focus on first call resolution--and as a result the overall costs went down. Putting the screws on agents to get customers off the phone had been leading to poor service, customer frustration, and higher cost.

It has also become clear that many customers really will use self-service, and even prefer it for many things. They just don't like bad self-service. But when a reasonable option exists (like online travel reservations, or ATM machines) the acceptance rate is phenomenal.

For the past ten years I've been preaching that the best way to get customers to change their behavior is by seducing them with a better option, not by forcing them to do something they don't want. Based on what I've been hearing, the message is getting across.

I was interviewed a couple weeks ago for an NPR story about speech technology in customer experience. I didn't make it to the final cut (I guess I have a voice made for blogging), but I can't argue with the results.

Listen to (or read) the story here.

It seems I never get tired of repeating that "correlation is not causation."

Today, though, Bloomburg said it better.

Consumerist has a horror story up today about an American Airlines customer trying (and failing) to get some sort of proof that he has a credit available after a cancelled flight.

The story is typical of the genre, featuring Kafka-esque company policies, indifferent employees, and a seeming inability to take even the smallest step to accomodate an entirely reasonable and understandable customer request.

Until this statement, about a third of the way through:

"I was told that customer service at American no longer speaks to customers"

I truly thought I'd seen and heard everything in the customer service business. I guess I was wrong.

Today's XKCD comic captures one of my own pet peeves: sloppiness around percentages vs. percentage points.

A percentage change is a relative measurement, as compared to some other value. If Grayton started at 20% support in the polls, and his support drops by 19%, that means he's now at a little over 16% support. 19% of 20% is 3.8 percentage points, and 20% - 3.8% = 16.2%.

A percentage point change is an absolute difference in the value being measured. If Grayton's support drops by 19 percentage points, that means he went from 20% to 1%. That 19-point drop is a 95% drop in his support.

From the context of the news, we can surmise that the reporter probably meant that Grayton's support dropped by 19 percentage points. Unless, that is, a substantial number of voters are anti-Christmas and pro-drunk driving.

Confusingly, the media often uses the "percentage change" language to mean both a percentage change and a change of a certain number of percentage points. Reporters covering political polls will often say a candidate's support has change by so many percent when they mean percentage points. Medical stories often use the correct meaning, which also happens to sound more impressive: reducing heart attacks from 1% of patients to 0.5% of patients is a 50% drop in heart attacks.

STELLAService recently published a survey of hold times at major airlines during the busy pre-Thanksgiving travel week. They placed five calls per day to each of 13 domestic U.S. carriers, and found that the average wait ranged from a minute and a half to over half an hour.

These are interesting statistics, but of course I want to know more:

• Were the calls placed by a robot, an ordinary consumer, or a trained analyst?
• STELLAService describes the calls as about a variety of "general customer service questions." Did it matter what the question was?
• For that matter, did they make any effort to use the self-service options? (I know, I know, that's not what the survey was about--but it's still interesting to know).
• How consistent were the wait times? Was there any time of the day which was generally better or worse?

The term "Voice of the Customer" has, through overuse, become almost meaningless. I've seen it used to mean almost any customer feedback process from e-mail surveys to focus groups to suggestion boxes.

And that's a shame, because the actual Voice of the Customer--an audio recording of a customer speaking his or her opinion--is one of the most powerful tools for building an effective customer feedback program.

If a customer feedback program is to result in a better customer experience, then it has to change people's behvaiors and decisions within the company. That's true at all levels of the organization: executives have to be pursuaded to make better decisions about how to invest resources in customer service; while customer-facing employees need to use the feedback to improve their customer service skills.

Recordings of customer interviews are pursuasive in a way that statistics and written feedback are not. The recording conveys emotion and helps the listener empathize with the customer, and overcomed the usual defensive barriers most people have to negative feedback.

Of course the other elements in a customer feedback process are important: one customer's opinion, however strongly held, should not drive corporate decision-making. But people make decisions as much with their hearts as their heads. Hearing the actual voices of actual customers can be just the tool needed to break through the logjam of defensiveness, resistance to change, and analysis paralysis that often keep companies from making positive changes.

Many large companies are now offering smartphone or tablet apps for customer service. These give what is supposed to be a convenient way to do certain tasks (check your balance, book a flight, etc.) from the convenience of your gadget and (not incidentally) without talking to a customer service rep.

When you pick up the phone and call, you never know in advance what sort of experience you will have. One cool thing about these customer service apps is that you can see the user rating right there on the page. I looked up a few iPhone apps from some big companies:

Cream of the Crop

• Zappos: 4.5 Stars

Nice Job

• Chase Banking: 4 Stars
• Best Buy: 4 Stars
• Amazon.com: 4 Stars

Better Than Nothing, But Not Great

• Wells Fargo Banking: 3.5 Stars
• Bank of America: 3 Stars
• Citi Bank: 3 Stars
• AT&T: 3 Stars
• Delta Airlines: 3 Stars
• United Airlines: 3 Stars

Why Do They Bother?

• Charles Schwab: 2.5 Stars
• Verizon: 2.5 Stars
• American Airlines: 2.5 Stars

What Were They Thinking?

• Southwest Airlines: 1.5 Stars

In my personal experience, iPhone apps with fewer than 4 stars usually have some significant deficiencies, and a three-star (or worse) app is usually pretty bad. Clearly some companies have done a better job than others in their mobile customer service apps--and clearly some companies need to rethink this strategy entirely.

We are not yet at the point where many customers make buying decisions based on how many stars a bank or airline's mobile app gets. But that day might not be so far off.

Many customer feedback programs struggle with credibility, especially among front-line employees who receive negative feedback.

That's understandable, and human. Most people take pride in their work, and don't want to believe that they aren't providing good service. So the natural reaction is to be defensive and look for ways the feedback might be mistaken.

Employees won't change their behavior based on customer feedback unless they believe the feedback is fair and reasonable. Here are some ways to make feedback more credible for the recipients:

• Focus on stories, not statistics. Statistics carry no emotional value, but a story does. Rather than "Your satisfaction score last month was five points below our threshold," try something like, "Let's talk about Mary's feedback. What do you think she wanted, and how could you have served her better?"
• Use the literal voice of the customer (if possible). Audio recordings of customer feedback makes it easier to empathize with the customer, as compared to written feedback or transcripts.
• Provide a route to appeal. Survey mistakes do happen, and if there's an evenhanded way to correct errors you can get people to buy in to the process as a whole.
• Be consistent. The message needs to be that the feedback process is here to stay, it is taken seriously, and the easiest route to better scores is through better service.