The Customer Service Survey

Vocalabs' Blog

Blogs

Too Small for Surveys?

When is the right time in a company's growth to put in place a customer feedback program?

Not every company needs or would benefit from surveys. Very small businesses may be very intimate with their customers and wouldn't learn anything new from a survey process. But as organizations grow in size and complexity, the need for a survey program becomes greater.

The role of a survey program is to provide the organization with visibility into how it is performing from the customer's perspective. So the right time to think about a survey program is when the company no longer gets that visibility in the ordinary course of business.

Here are two questions to ask when deciding if it's time to start surveying your customers:

  1. Is there any one person in the company who personally interacts with a significant fraction of the customers? In a smaller company, there are usually people who directly touch a lot of the customers. For example, a B2B consultancy where the CEO meets with all customers, or a pharmacy where the pharmacist personally talks to a large percentage of the people who walk through the door. This direct personal contact gives a lot of customer insight and intimacy, and probably means that you won't learn anything new from a survey. But if your business is big enough that no one person has the time or ability to touch more than a few percent of the customer base, then the only way you're going to get the big picture of how your customers feel about you is through a survey. This is a good application for a relationship survey, where you can take the temperature of your customers from time to time and make sure everything is on the right track.
  2. Is there any one person in the company who personally oversees all customer-facing employees? Just as a larger customer base makes it harder to see the big picture of your customer relationships, having a larger number of customer-facing employees makes it harder to see the big picture of how your employees are relating to your customers. If you're big enough to require at least a couple of supervisors, you should also have a transactional survey in place to collect data on specific customer interactions. This will provide more visibility and insight into how well the employees are dealing with customers, give better opportunities for coaching, and minimize the chances that a poor employee will slip through the cracks.

Smaller companies with fewer customers and simpler operations often don't need to perform customer surveys.

The time to think about a survey process is when you get big enough that you lose that customer and operational intimacy. When there's nobody who personally knows your customers or is personally responsible for supervising all the employees who deal with your customers, a formal feedback process gives you back some of that visibility you had as a smaller organization.

B2B Customer Feedback

Customer surveys are just as important a tool in business-to-business relationships as in business-to-consumer relationships, and we see a lot of interest from B2B companies in launching or improving their feedback programs. Most of the basic principles of survey design apply just as well in the B2B world as in the B2C world, but there are some important considerations to keep in mind.

Business-to-Business relationships are usually more complicated than consumer relationships, and have much higher lifetime value. There are often multiple decision-makers and decision-influencers, making it hard to get a definitive read on the overall strength of the relationship at any given time. However, we've found that it's often not hard to get customers in a business relationship to provide some feedback, since the relationship is often very important to the customer, too.

Here are some things to keep in mind when setting up a survey program for business customers:

  1. Consider the entire customer journey. Because B2B relationships usually have many different people involved in different aspects of the relationship, you want to try to capture feedback throughout the customer journey. Experiences like customer service calls and closing trouble tickets are obvious times to offer a survey, but you should also be asking for feedback after new orders, deliveries, invoices, training sessions, and any other point where the customer interacts directly with you.
  2. Respect the customer. "Respect the customer" is the first principle of Agile Customer Feedback, and it's even more important for B2B relationships because of the number of people involved and the value of the relationship. In practice, this means:
    1. Have strong exclusion rules in place. The same person should not get asked to take a survey over and over. I generally recommend that if a customer is asked to provide feedback, the same person won't get asked again for at least 30 days for any survey (even if it's about a different experience). And by all means, if a customer asks not to be surveyed, respect that.
    2. Be on the ball with closing the loop. If a customer had a bad experience or needs attention, get to it right away. Communicate back to your customers the importance of their feedback and anything you're doing differently because of it.
    3. Respect the customer's time. Keep transactional surveys short and relevant, and schedule time for longer relationship surveys. Don't call out of the blue and ask for more than five minutes.
    4. Make it personal. Having a real person call communicates that you take the relationship seriously; sending an email communicates that you don't want to spend money listening to your customers.
  3. Have a customer-centric view. Make sure you have the ability to pull together different surveys completed by different people at the same customer. Each person is going to have a different perspective on the relationship, and you want to be able to place all those pieces of feedback into context with each other. The goal is to see both the forest and the trees.

Building an effective feedback program in a business-to-business relationship isn't any harder than in a consumer relationship. Pay attention to the basics, respect your customers, and take into account the complexity of B2B, and your program will be off to a strong start.

Can you spot the survey mistakes?

Here's an amusing/horrifying story about a customer survey process gone horribly wrong:

Me: “Sir. Why are you giving us bad grades on the survey? You said everything was good.”

Customer: “Oh. Everything was good. I just didn’t like the movie. It was confusing.”

Me: “Sir, the surveys are not for the film itself. They’re for the theater and our staff.”

Customer: “Oh, but I want the studios to know I didn’t like the movie.”

Me: “That’s not how these surveys work. We don’t make the films; we just show them. The surveys are for customers to give feedback on how we performed. It’s a common mistake people make, but I’m going to strongly encourage you not to submit that survey.”

Customer: “Why not?”

Read the full story. Can you spot all the things this company is doing wrong in its survey process? Here's a partial list of mistakes I saw:

  1. The customer has to ask for a survey form, from the staff.
  2. The survey is designed in a way that it doesn't deal with the (apparently common) problem of customers reviewing the movie not the theater.
  3. At least some customers think the survey goes to the studio, not the theater chain.
  4. Customers can fill out the form with staff watching, and the staff can apparently try to talk the customer out of the survey.
  5. Despite the flaws in the process, the survey is apparently used to fire and promote people.
  6. Even a single bad survey is enough to cause serious problems for the theater staff.

For extra credit: how would you design a feedback process for a movie theater which actually works for its intended purpose?

New Case Study Posted

We've just posted a new case study on one of our clients, a B2B medical technology company where we are conducting customer interviews after a technical service call. You can download it, and please contact us if you have any questions.

What's Effective?

I use the word "effective" a lot in the context of building a customer feedback program.

As in, "to build an effective survey you should...." or, "effective customer feedback programs usually have....." or, "that's not an effective survey technique."

"Effective" is something we all want our surveys to be, but how do you know if your survey is effective or not?

"Effective" just means that something has the desired outcome or effect. So an effective survey is simply one which achieves its goals.

It seems like stating the obvious, but if you don't have a good handle on why you're conducting a customer survey, it's unlikely you're going to get much out of the process.

So the first step in trying to understand whether your survey is effective is clearly stating the goals of the process. Some common goals are:

  • To track opinions about the customer service level month-to-month (a common, if not very ambitious, goal)
  • To coach and train employees based on customer feedback
  • To identify customer pain points and broken business processes
  • To validate changes or improvements to the customer experience

Once you know what the goals of the program are, it's usually pretty easy to determine whether the survey is effective or not. What to do about an ineffective survey is a different issue, though often if the goals are well-understood, it's also pretty clear why a survey isn't meeting those goals (i.e. not enough data, not enough detail, data is not timely enough, surveys can't be connected to specific experiences, etc.).

But more often than not, the root cause of an ineffective survey is simply that it's not clear what the survey was supposed to accomplish in the first place.

So if you're trying to build an effective survey program, the first step is to make sure everyone understands what the goals are. Without that, you don't even know what "effective" is.

Latest issue of Vocalabs' newsletter has been published

We just published the latest issue of our newsletter, Quality Times.

In this issue we announce the availability of 2014 Executive Summary reports for the National Customer Service Survey in Communications Services and Banking. This is our unique syndicated research product where we interview customers immediately after a customer service call to one of the companies we follow, allowing us to collect detailed and in-depth research about specific customer experiences.

As always, I hope you find our newsletter interesting and informative. Email subscribers should be receiving their copies shortly.

Whatever happened to that study?

This particular story comes to us from the Department of Homeland Security, probably one of the most dysfunctional federal agencies (and that's truly saying something). But it will probably be familiar, in lesser form, to many people in large organizations struggling to build an effective feedback program.

You see, DHS has a problem. Its particular problem is having the lowest morale of any federal agency. So they commissioned an employee survey, which pointed to several changes management could do to improve things.

But nothing happened after that study was completed. So they paid for another survey, which pretty much said the same thing.

Still nothing happened. Nothing happened after the third study, either. Or the fourth.

Now, though, a new factor has emerged to weigh on the depressed morale of DHS workers: too many internal surveys.

The problem is that surveys are just a tool, and like many tools, they can be used for many different purposes. The same hammer which can be used to build a house can also be used to smash the windows. It all comes down to the intent of the wielder.

Surveys can be used very effectively to gain insights, identify root causes of problems, and support a program of continuous improvement. Surveys can also be used to delay and hinder change, and create the appearance of action where none exists. It all comes down to the intent of the wielder.

For a dysfunctional bureaucracy like DHS, which apparently does not have the organizational will to face its problems and make real changes, the employee survey is a very effective tool for resisting change. "We need to study the problem" is followed by "we need to finish the study before we do anything," then "we need to do another study," and finally, "whatever happened to that study?"

The lesson is that a survey, by itself, can't change anything. The organization and its leadership has to be committed to improvement before the tool can be used as it should be used. 

Download the latest NCSS reports

in

We just published the 2014 Executive Summary reports for the National Customer Service Survey (NCSS). This is our ongoing syndicated research project where we track the quality of customer service at selected large companies, by conducting phone interviews immediately after a customer service call.

Our Communications Services report covers AT&T, CenturyLink, Comcast, DirecTV, Dish Network, Sprint, Time Warner Cable, T-Mobile, and Verizon. This report is based on over 12,000 customer interviews from 2009 through 2014, and we include nine key metrics in our Executive Summary.

The Banking report includes Bank of America, Chase, Citi, and Wells Fargo. For these four companies we have data based on over 4,000 interviews between 2011 and 2014. The Executive Summary includes the same nine key metrics, and historical trends for all nine metrics.

Both reports are available from our website; in addition to the Executive Summary reports, subscribers receive real-time access to survey data as it comes in throughout the year, full responses to our 30-question interviews, and audio recordings of our customer interviews.

>> Download Communications Services Executive Summary

>> Download Banking Executive Summary

Drowning in Data, Starved for Stories

Most customer surveys today are drowning in data, but starved for stories. They're swamped with statistics, but eschew empathy. They're loaded with Likert scales, but....you get the idea.
The typical feedback program asks the customer to rate their experience in several different ways using a fixed scale. This lets you gather metrics, track changes, compare how well different parts of the organization are performing, and generally quantify the customer experience.
That's valuable, but it isn't the whole story. Statistics are very useful for understanding how you're doing in aggregate, but have nothing to offer when it comes to understanding the individual customer's experience.
And it's that individual customer's journey which is most important to the customer experience. Think about it: customers do not interact with a company en masse, they do it individually. Each customer has his or her own journey and own story.
This is where I see a lot of feedback programs falling down. Most programs are designed around the statistics. They do a good job collecting and reporting lots of aggregate data about how customers feel in aggregate. But they do a relatively poor job of communicating the individual customer's experience.
But if you want to improve the customer experience, you have to improve a lot of individual customer experiences. That means paying attention to those individual stories: How did this customer encounter the various touchpoints of the company? What could have been done to improve this customer's journey? What did this customer experience? How did this employee interact with this customer?
Those stories are often available, since customers are usually given the opportunity to provide some open-ended feedback. But the survey doesn't usually go out of its way to ask customers for a lot of details and specifics, and in a distressingly large number of cases those stories don't make it to the people who really need to hear them.
In some cases today, the customer comments never even get read by a person. Instead, they get categorized by an algorithm and dumped into a data warehouse, never to be seen unless some analyst happens to get curious about that particular customer.
The solution is to make an effort to collect stories and not just data, and deliver those stories to the people who can use them to do a better job:

  1. Limit the number of metrics on the survey, and instead ask more open-ended questions. You don't need to measure 53 different things, but a survey that long will dissuade customers from taking the time to tell their stories. Just a handful of metrics is all you need.
  2. If the customer interacted with a person, send the supervisor the customer's comments as soon as the survey is complete. Immediate feedback in the genuine voice of the customer is highly motivating and a powerful coaching tool.
  3. If the customer didn't interact with a person, treat every survey as a potential escalation event. Someone should review each survey as soon as possible, determine if further action is needed, and reach out to the customer if appropriate.

 

Post-Christmas Survey Design

It's that time of the year again, and if you're a parent and Customer Experience professional like me, you're probably busy designing your post-Christmas survey right now.

We all understand how important it is to have an effective VoC (Voice of the Children) process to help improve the Christmas Experience. So here's a few best practices which, like Rudolph's nose, will help guide you in your survey design.

1. Timely Feedback is Important

Transactional feedback is much more effective in real-time. This is just as true at Christmas as other times of the year. You want to invite your children to participate in the survey as soon after opening gifts as possible, and in our home we try to offer the survey within five minutes. This real-time survey gets a higher response rate and much more detailed feedback than if we waited a day or a week for the survey.

Real-time feedback is important on the reporting side, too. Getting an immediate survey back about something that just happened a few minutes ago is a powerful tool to help both employees and grandparents really listen to the VoC and internalize the feedback they're getting. We don't ever want to let a coaching and training opportunity slip by, so when the survey is positive we make sure to acknowledge that to everyone ("Uncle David got all 5's for the Minecraft playset! Keep it up, David!"). When the survey is negative, of course, we'll use that to do some one-on-one coaching to make sure the underperforming relative is on his or her best Christmas game for next year.

2. Interviews vs. Online Surveys

There's no one right answer about whether to use interviews or online surveys for your VoC program. Interviews have a higher response rate and get more detailed feedback, while online surveys are a lot less expensive. So when deciding which survey channel to use, take into consideration the goals of the VoC program and your budget.

Lately we've also seen a lot of interest in a hybrid approach, where some children are selected for the more in-depth interview format, and others for the online survey. This lets us target the detailed feedback where we think it will be most valuable (for example, if the VoC survey consistently shows that Grandma Joan's sweaters always flop, we may target her gifts for the interview so she can get more feedback on how to improve), but still get a lot more surveys done than we could afford with a 100% interview approach.

3. Keep It Short!

Nothing ruins the Christmas spirit like being asked to sit still for a 15-minute interview, or answer 75 questions in an online survey. So keep the survey short and to the point, and focus just on your most important metrics. I suggest using one or two high-level questions (such as Net Promoter, Effort, and/or Christmas Satisfaction) along with a handful of more specific questions about things like gift quality, wrapping paper, anticipation factor, etc. Don't forget to include a free response question!

My rule of thumb is to try to keep the VoC surveys to under 5 minutes for an interview, or a single screen (no scrolling) for an online survey. You'd be surprised what you can get done in a five minute interview, plus the response rate will be higher and the survey will be less expensive.

4. Implement a Closed-Loop Process

More and more people are realizing that much of the value of a VoC program isn't in the survey data per se, but in what you do with it. And one of the most valuable things you can do with the data is use it to drive a closed-loop process.

That means that you contact dissatisfied children in order to resolve their problems and understand the root cause of the dissatisfaction. By going through this process you will both directly increase satisfaction, and also be able to prevent future Christmas disasters. So for example, if Sylvia is upset that she got the 75-piece chemistry set instead of the 150-piece set, you may be able to offer an apology and a small piece of candy in the moment; and also implement a process to ensure that in the future all relatives understand exactly which SKU she wants for each item on her list.

5. Have Robust Reporting

Improvement over time requires careful attention to detail and the ability to slice and dice the VoC data in a lot of different ways. You want to be able to track key metrics over time, by gift and by gift-giver, by wrapping style, etc., in order to understand what's driving Christmas satisfaction and implement the appropriate process changes.

You will also want to make sure the data is delivered to the right people to enable improvement. Each year, before the gift buying season begins, each relative should have a report of prior VoC scores, along with the children's comments and the reporting tools to allow them to dive deeper into the data. Just having the right reports is going to make a huge difference in your Christmas experience!

Remember: This Is a Process, Not a Project

Bringing the VoC into the Christmas Experience is not going to happen overnight. Some relatives are likely to question the value of collecting feedback at all (remember: those are also the same relatives who are likely to score badly on the survey). My advice is to be professional but persistent. Once everyone starts seeing real data, the value of a robust feedback process will become obvious to everyone.

Mandatory Survey Questions are Evil

You're merrily filling out a survey for some company when you come to a question you just can't answer. Maybe the question is asking about something you don't have any experience with, or maybe the question doesn't make sense (not every survey is well-written). So you skip the question and click the "Next" or "Done" button.

Only to be sent back to the same page with angry red highlighting on the skipped question and a stern warning: "You must answer this question to continue."

At this point you can (a) abandon the survey, or (b) make up a fake answer. Honestly answering the question isn't an option.

This is why mandatory survey questions are evil:

  • Mandatory survey questions don't respect the fact that the customer is doing you a favor. The first rule of conducting customer surveys should always be that the customer's feedback is a gift, and should be treated as such. If a customer doesn't want to answer a question, you need to deal with that fact and not act like you're entitled to an answer.
  • There is no situation which actually requires using mandatory survey questions. A good survey designer and a well-designed process can easily find better ways to do the things mandatory questions are there for. Skip logic? Design a path for "no answer" responses. Blank surveys? Filter them out in analysis instead of rejecting what feedback there might be. Giving incentives? If someone turns in a blank survey send them an "Oops, you forgot to fill out the survey" response.
  • Skipped survey questions are a problem of survey design, not the customer's intent. Mandatory questions treat skipped questions as though the problem is the customer doesn't want to do the survey. But you already know the customer wants to do the survey because he's doing the survey. If you have skipped questions, the problem is the survey is poorly designed. Chances are it's too long, or you're asking questions which don't make sense to the customer. In the real world, well-designed surveys have very few skipped questions: we expect just a couple percent of people to skip any given closed-ended survey question.
  • Mandatory survey questions tell the customer you don't care about his feedback. Forcing the customer to answer a survey question he doesn't want to (or can't) answer says that you would rather get no feedback at all from the customer than a survey with a single box unticked. That hardly seems like the message you want to give to someone who is, after all, taking the time to help you improve.

If you have a survey with any required questions, just make them all optional. You will get better feedback, your response rate will go up, and you will sleep better at night knowing that you aren't aligned with the forces of darkness.

It's Time to Ditch That Long Survey

Is your customer survey too long?

In my opinion, if your transactional survey is longer than five minutes (for a phone interview) or one screen (for an online survey), then yes it is too long.

But even if you're not as ruthless about survey length as I am, there's a lot of surveys out there which everybody agrees are too long. I'm talking about the five-page 25-question online surveys we all know and loathe, or the 10-minute phone surveys which just seem to go on forever.

These surveys are amazingly common, yet I have never found anyone, not a single person, who disagrees with the notion that such a long survey is much too long. And I've spoken with a lot of people who currently field long surveys, and even people who designed them in the first place. Some are even our clients. And every single person I've spoken to thinks the long survey should be shorter.

So if everyone can agree that a survey is too long, why doesn't it get changed? I've heard a lot of excuses reasons, and I'm here to tell you why these don't hold water:

  • Reason #1: It takes time to build consensus to change the survey. Guess what? Everybody agrees the survey is too long. You have consensus. Go change the survey.
  • Reason #2: We've identified 57 different business drivers we need to ask about. You can't focus on 57 different things. If you're trying to focus on that many things, you're not actually focusing on anything. Chances are there are only 3-5 of the questions that anyone ever pays any attention to, and you already know what they are. Get rid of everything else. Or if you truly cannot get away from all those secondary drivers, ask each customer about a random set of three or five. This will at least let you track the overall trends and your customers will thank you.
  • Reason #3: We need to maintain continuity with our historical data set. There's really no point maintaining a worthless data set. And anything beyond the handful of questions you're really using is not serving any function and has no value.
  • Reason #4: We have a lot of process built up around the existing survey. Change is hard, we all get it. But maintaining all that process costs time and money. There's no point wasting the resources to keep delivering reports about things nobody cares about.
  • Reason #5: We are legally required to ask all these questions, in exactly this way. Okay, this is probably the one real reason not to fix a survey which is too long. Those working in the healthcare sector probably know I'm thinking of the HCAHPS survey, a legally-mandated monstrosity comprising 32 questions (which some vendors absurdly supplement with a set of their own proprietary questions). So if this is you, sorry, you're stuck. Go write a letter to your senator.

Well-designed short surveys almost always have a higher response rate, yield more useful data, allow more focus on business goals, leave a more positive impression on customers, and are easier to manage. They're usually cheaper, too.

So what's your excuse?

We Value Your Feedback!

Via the Daily WTF, it seems that Adobe has a unique way to show customers just how seriously the company takes its customer surveys (click the image for a bigger version which is actually legible):

I mean, c'mon guys, I'm sure your QA department is pretty busy testing all your software products, but would it kill you to proofread the survey once in a while?

Be Thankful for Negative Feedback

It's that time of year again: airports are filling up, frozen turkeys are on sale at the supermarket, and writers are searching for a seasonal hook for their articles.

So, right.

Most people don't like getting negative customer feedback. But getting a bad survey from a customer is actually something to be thankful for (see what I did there?). Here are three reasons why:

  1. Negative Feedback Helps You Improve: Can you imagine a basketball coach who gives nothing but positive feedback to his players? I can't, and if such a person existed, I doubt he would be very effective. Of course, everyone would rather hear the good news than the bad news, but it's the negative feedback which gives us the ability to improve. So if you want to improve your customer experience, a customer who gives negative feedback is like a coach. Sometimes coaches are angry, rude, insulting, and hard to listen to. But without that coach to point out your mistakes, you're simply not going to get any better.
  2. Negative Feedback is More Honest: Not only would most people rather hear positive feedback, most people would also rather give positive feedback. It's uncomfortable to criticize other people (even through a faceless web page), and that's why so many customers give top scores on surveys. So when a customer makes the effort to criticize instead of just saying things were fine, you're probably getting the straight dope.
  3. Negative Feedback Keeps you Grounded: The Dunning-Kruger Effect is a big problem in the customer experience world: many many companies think they provide an above-average experience when in truth they stink. A healthy dose of criticism from your own customers is one way to keep your self-perceptions grounded in reality.

So next time a survey comes back with bad scores and withering criticism, don't get upset or defensive.

Instead, take a deep breath and be thankful that you have customers willing to help you in this way.

And pass the cranberry sauce.

Money vs. Attention

Techdirt ran another article about Comcast's reputation for poor customer service today. In it, the author repeated a common conclusion which I think is probably wrong:

 
...Comcast has no meaningful competitive incentive to change, and therefore simply refuses to spend the money necessary to fix the problem.
 

He's right that Comcast has no competitive incentive to change, but wrong in assuming it's all about not spending money.

Here's the thing: it's possible to spend money on customer service efficiently, and it's possible to spend money on customer service inefficiently. If you have bad service but spend the money well, then spending more money is likely to improve the service.

But if you're spending the money poorly, then spending more won't get you better service, just more bad service.

The kinds of complaints you see about Comcast have all the hallmarks of money being spent very badly.

For example, take the complaints about very long hold times. While occasional long hold times can be caused by a surge in calls, the hold queue costs money. All those people sitting on hold are tying up phone ports and running up long distance charges. Sure it's fractions of a cent per person per minute, but it adds up. And if those people don't get through today, they're calling back tomorrow, and if they don't call back tomorrow they're probably taking their business elsewhere. Persistently long hold times are a symptom of a company digging itself ever deeper into a hole of unresolved customer problems.

And it's very expensive when a customer talks to a rep but doesn't get his problem fixed on the first call, because that customer is calling back and it's going to cost twice as much. Complaints about poor resolution rates and having to make many calls to solve a problem mean that the company is wasting money by not taking the time to fix problems properly the first time.

But even that's nothing compared to the money Comcast wastes when they have to send a truck to the customer's house multiple times for the same problem (or when they send a truck for something the customer could have fixed herself).

I have no inside information, but my gut tells me that Comcast is actually spending far more on customer service than they should have to. The problem is they are wasting most of the money by delivering very poor service.

My guess is that if Comcast got its act together, it could deliver much better service and save a ton of money. But that would require an upfront effort to (a) train agents better, (b) allow support reps to spend more time with each customer to solve the problem on the first call, (c) empower reps to solve customer problems, and (d) allow different parts of the company to coordinate better.

The irony is that this investment would primarily be in the form of management time and attention, not money. The savings would probably start rolling in pretty quickly.

So to me (as an outsider looking in), the real problem is not that Comcast doesn't want to spend the money to fix its service. The real problem is that the senior leadership doesn't want to pay enough attention to the problem to get it fixed.

Gamification vs. Incentives and Recognition

Call me a curmudgeon, but I have a hard time getting behind "gamification" in the workplace.

For those not familiar with gamification, it basically means using the principles of video game mechanics in a real-life situation, like a call center, to motivate people and change their behavior. Gamification, as a buzzword, has been around long enough to develop both hype and backlash.

My problem is that, at its core, gamification in the workplace is really nothing more (or less) than the systematic use of employee recognition, rewards, and achievement as a way to motivate employees. But people have been doing that for as long as there have been workplaces. So the "new" thing in gamification is just the idea of being deliberate and systematic about employee motivation.

But by calling it "gamification" instead of (for example) "employee recognition program," you're implying that a cleverly-crafted set of achievements can somehow transform a dreary workplace into something fun like Mario Cart. At the end of the day, though, a boring job will still be a boring job.

It can also be insulting if done wrong, implying that handing out meaningless "achievements" is just as good as giving employees raises or bonuses.

So while I'm all on board for the idea of rewarding and recognizing employees, and I think the game industry may have a few things to teach us about what motivates people, can we please stop pretending that work, for most people, is anything other than work?

So much for my life's dream

Sadly, it seems I will never be a Mad Scientist. At least not if I stay in the survey business.

Troubleshooting a Survey: What Can Go Wrong

A lot of things have to happen to build an effective customer feedback program.

The flip side of this that if you have a customer feedback program which isn't effective, there's a lot of potential reasons. Using a systematic approach to troubleshooting the feedback process can help avoid wasting time on implementing the wrong solutions.

So to help with an ineffective survey process, here's a short troubleshooting guide for common survey problems.

Problem: Low Survey Response

General Troubleshooting Questions:

  • Are you are getting accurate contact information for customers?
  • Does the survey work (no errors, database problems, etc.)?
  • Is the survey a reasonable length (one page with no scrolling for online surveys, five minutes or less for phone interviews)?
  • Does the survey appear to come from a legitimate source?
  • Are you ensuring that customers don't get over-surveyed?
  • Can the customer take the survey immediately when asked, or does the customer need to remember to do it at a later time?
  • Does the survey require the customer to go through extra steps (copy a code from a receipt, call a phone number, etc.)?
  • Does the survey have mandatory questions?
  • Is the customer asked to take the survey a long time (days or even weeks) after the transaction?

Troubleshooting Questions for Email/Online Surveys:

  • Are survey invitations being marked as spam?
  • Does the invitation look professional and legitimate?
  • Does the invitation explain why you want the customer's feedback?
  • Does the invitation promise that the survey will be short (note: the survey must actually be short)?

Troubleshooting Questions for Phone Interviews:

  • Do the phone interviewers sound polite, empathetic, and professional on the phone?
  • Do the phone interviewers have noticeable foreign accents?
  • Is the Caller ID set to a real phone number which customers can call back to verify the survey is legitimate?
  • Does the interview script give an honest estimate of the survey time?
  • Do interviewers identify themselves and the sponsor of the survey?

Problem: No Follow-Through With Customers

Troubleshooting Questions:

  • Do you have a closed-loop process for customers who may want or need extra attention?
  • Is there tracking to ensure customers who need follow-up are actually contacted?
  • Are follow-up calls conducted by someone empowered to solve the customer's problem?
  • Do you capture and track the root causes of customers' issues?
  • Are follow-up calls conducted by someone other than the person who may have caused the customer's problem?

Problem: Survey Responses Are Not Relevant to the Business

Troubleshooting Questions:

  • Has the survey been updated recently?
  • Have you reviewed the performance of each survey question, and removed questions which are not yielding useful information?
  • Have you experimented with new survey questions relevant to current business issues?
  • Are you asking follow-up questions when customers have negative feedback?
  • Do you ask business stakeholders to provide feedback on what questions are relevant to them?
  • Do you regularly update the survey as the business needs evolve?
  • Do front-line employees have access to raw customer feedback in real time?

Problem: The Business Does Not Fix Known Problems in the Customer Experience

  • Is there a leadership commitment to improve the customer experience?
  • Do other parts of the organization get data to show how they impact the customer experience?
  • Are you using individual customer stories to persuade the organization that these issues are important?
  • Is the customer survey perceived as credible?
  • Does the company culture encourage listening to customer feedback?
  • Can you connect poor customer experience to financial metrics (through churn, increased operational expense, higher customer acquisition cost, etc.)?

Problem: Too Much Survey Data and Not Enough Useful Information

  • Have you reviewed the performance of each survey question and removed questions which are not yielding useful information?
  • Are you giving customer-facing employees direct and real-time access to their customer feedback?
  • Are you asking follow-up questions when the customer gives negative feedback?
  • Do you have a reporting tool which allows easy filtering of customer feedback?
  • Are you tracking general categories of customer comments in free response questions?
  • Do your categories evolve as the business needs change?
  • Do you keep the number of categories manageable, so you don't have categories which are either irrelevant to the business or statistically insignificant?

Problem: Survey Reports Are Ignored

  • Is there a leadership commitment to improve the customer experience?
  • Does the company have a culture of listening to customer feedback?
  • Are survey reports tailored to the needs of the individual recipient, or does everyone get the same reports?
  • Can recipients of survey reports modify the reports (filter the data, calculate new metrics, read customer comments, etc.)?
  • Have you asked for feedback on survey reports from the people who receive them?
  • Do recipients of reports feel they have a stake in the customer feedback process?

These questions get to a lot of common underlying problems we see with customer feedback processes. This doesn't cover everything that can go wrong, but it's a good place to start if you don't think you're getting the results you should.

Vocalabs Newsletter #83 is published

We just published issue 83 of Quality Times, our newsletter about measuring the customer experience.

In this issue I have two articles: one about the difference between "insightful" and "useful" data (spoiler alert: they are not the same!), and another about what makes a customer experience brittle.

As always I hope you find this interesting and informative. You can also subscribe to receive our newsletter via email, and receive it as soon as it's published every month.

Explosion of Really Bad Surveys

Local newspaper columnist James Lileks takes some well-deserved (and hilarious) potshots today at bad surveys.

He also reveals that, back in college, he did a turn as a telephone interviewer. So he at least has some sympathy for what it's like to be in the survey biz.

It does seem like there's been an explosion of really bad surveys over the past several years. Personally I blame the confluence of several factors:

  1. Online surveys have gotten really cheap and easy. This means organizations do more surveys but at the same time put less care and effort into designing the survey. Gone are the days when doing anything but the smallest survey meant hiring a market research company (for a minimum of $50K). It's distressingly common to see surveys riddled with typos, nonsensical questions, and other problems which make it clear nobody could be bothered to do a good job.
  2. Yet the long-form survey style somehow persists. When surveys were rare and expensive it made sense to ask every imaginable question because you needed to squeeze every possible insight from each participant. Today this mindset continues, even though surveys are cheap and common, and it's not unusual for a consumer to be asked to respond to literally hundreds of questions about a single three-day trip.
  3. And consumers are refusing to respond to bad surveys. Across the industry you hear people complaining that response rates are down on email surveys. But instead of asking the sensible question ("Why don't people want to take our survey?"), many companies respond by simply sending more survey invitations. To the same badly-designed, overly-long survey that 98% (or more!) of their customers won't fill out.

These problems won't be easy to solve, mostly because the root cause is that most organizations don't care as much about the customer experience as they say they do. This has always been the case--when it comes to customer service most companies talk the talk much better than they walk the walk--but the difference is that today it's easy to just do a survey instead of doing something.

When Does Bad Customer Service Become Evil?

There's a story making the rounds the past few days of a customer who had tried for over a year to get Comcast to correct a series of billing mistakes. Eventually he got fed up with ongoing mistakes and incompetence, and called the office of Comcast's controller to lodge a complaint.

That didn't work either, and that's when things got weird and evil. This customer happens to be an accountant, and mentioned in one of his complaints that he thought Comcast's billing problems should be investigated by the Public Company Accounting Oversight Board.

Which, to me, seems like a reasonable thing to say if you're an angry customer subjected to over a year's worth of overcharges and billing mistakes. I'm not an accountant, but persistent billing mistakes sure sound like an accounting problem to me.

But apparently Comcast thought otherwise: Comcast contacted the customer's employer and apparently said something that got the customer fired. Did I mention that the customer's employer happens to do a lot of business with Comcast?

Needless to say there's a lawsuit now, and a lot of he-said-she-said. The Consumerist has a good summary in a pair of articles: one about the customer getting fired, and another about Comcast apologizing for bad service but not getting the customer fired.

So it's easy to take the angle of tsk-tsking Comcast for another horrific example of bad customer service. No, make that evil customer service, since this has gone beyond the realm of incompetent into malicious.

But what I want to know is what the heck is going on at Comcast? Someone, somewhere inside Comcast at some point thought it was OK to call a customer's employer and say something that got the customer fired because of a billing complaint.

(A complaint which, by the way, Comcast has acknowledged was legitimate.)

Someone in the company had to have known how this would play in the media, to say nothing of the courts. Yet it happened anyway. I can think of a few explanations:

  1. Comcast thinks it's above the law and public opinion and can act with impunity. This is probably not strictly true, but the company has been persistently successful despite its poor reputation, so maybe people think they can get away with stuff.
  2. Someone panicked. Part of me thinks that it's really plausible that someone in Comcast might panic over the prospect of an accounting review of its billing systems. Just by the company's reputation for mistakes, it seems fair to assume that Comcast's systems aren't really ready for their close-up.
  3. Internally, Comcast is just out of control. Media reports over the past few months have painted a picture of a company divided into fiefdoms and disorganized, so it's possible that there just isn't enough adult supervision going on.

I don't know which of these theories, if any, is right. But it's clear that something weird is going on at Comcast.

Too bad the ATM at my bank doesn't do this

This video is old, but it's new to me. Take 45 seconds to watch it and see a customer experience that practically defines "delight."

Apparently in Japan, if you press the "help" button on a train ticket kiosk, a guy actually pops out from the wall behind the kiosk to lend assistance--to the delight and befuddlement of foreign tourists.

Doing a little research, I discovered that the guy isn't just sitting behind the kiosk all day waiting for Americans to push the help button. His main job is keeping the machines stocked with blank tickets, which is done from behind the kiosk so it doesn't disrupt normal operations. But as long as he's there, he can lend a hand as needed.

Of course, to the Japanese this is just normal and not the least bit delightful. That's the problem with the treadmill of customer delight.

I'm also reminded of the New York subway (and many other large American cities), where attendants are also available at many stations. But in America, we tend to put the attendants inside glass booths instead of having them magically step out of the wall when needed.

The Japanese way somehow seems so much more...delightful.

You Had Good Runs

Given the circumstances, 1-800-Flowers can't be happy to see their name in the New York Times over an epic miscommunication.

The flower arrangement for Grandma's funeral was supposed to read, "Farewell Grammy, you had a good run."

What it actually said, as transcribed by the phone agent who, according to the customer, "spoke English as a second language," was "FAR WELL GRANDMY YOU HAD A GOOD RUNS".

When the customer sent an e-mail complaint he heard no response--probably because he sent it to a "do not reply" address, which is where his order confirmation came from.

Someday I'll write an article about why using "do not reply to this email" email addresses are a dumb idea from a customer experience perspective. But this is not that article.

Instead, I'd like to pose the question of why 1-800-Flowers--a company whose entire existence is based on people's desire to be sensitive, communicate, and do what's culturally expected--can't hire employees with the sensitivity, communication skills, and cultural background to understand that while "you had a good run" is an appropriate (if cheeky) message for a funeral while, "you had good runs" is not.

Of course we all know the reason: it costs more money to hire good employees than bad ones.

But, as the Times columnist notes, 1-800-Flowers has attracted hundreds and hundreds and hundreds of complaints about botched orders.

Apparently the company has decided (for now at least) the negative publicity and bad word-of-mouth is a reasonable price to pay. I'll be curious to see how long that lasts.

Newsletter #82 is Published

The 82nd edition of our newsletter, Quality Times, has been published. If you are an e-mail subscriber you should be seeing it in your inbox shortly.

This month, the theme is cross-channel customer experiences. There are some very real challenges in providing a good customer experience when customers cross organizational silos, but some of the results I've seen make me think that this is one of the best opportunities for improving overall customer satisfaction. I think this is going to be an important area in the coming years.

As always I hope you find this interesting and informative.

Brittle Experiences

Think about what happens to a piece of glass when you hit it too hard: it shatters into a million pieces. We say that glass is brittle because it breaks before it bends.

Not all materials do this. Steel, for example, is likely to bend (maybe a lot) before it actually breaks. This is why we build bridges out of steel and not glass.

It's useful to apply the concept of brittleness to the world of business. Customer experiences, like bridges, are designed to handle a certain amount of strain before they start to fail.

When things start to go wrong, a brittle experience is likely to go catastrophically wrong for the customer or the company (or both). On the other hand, if the process is flexible enough to bend a bit and handle the unusual situation, it may not be that big of a deal.

For example, air travel today is very often a brittle experience. When everything goes well (as it usually does), you get to your destination on-time and with at least some dignity intact.

But if your travel plans go even slightly awry, the airline experience quickly goes from smooth to a stressful mess which could extend longer than the original trip. A brief thunderstorm at your departure airport means there's a long line of planes waiting to take off, and you sit on the ground for an hour or two. That departure delay means you miss your connecting flight. The next flight to your destination is overbooked, so you wind up spending the night at your connecting city waiting for a flight with an open seat to take you to your destination.

What started out as a minor hiccup (the brief thunderstorm) quickly turned into a stressful multi-day experience because the airports and airlines are too overloaded and too inflexible to handle even minor disruptions without it spiraling out of control. That's brittle: small problems become big problems and the whole thing goes very wrong for some passengers.

It's worth examining all elements of the customer experience under the lens of brittleness. Of course we expect that most of the time things will go smoothly for most customers, so the "normal" experience needs the most attention. But even the best-designed system won't be able to handle every situation.

So what happens when a customer has a problem? Are you flexible enough to deal with it gracefully? Or does the customer experience shatter into a million pieces like a piece of glass?

Syndicate content