The Customer Service Survey

Vocalabs' Blog


Issue #93 of Quality Times is Published

We just published the 93rd issue of Quality Times, our newsletter about measuring the customer experience. Email subscribers should be receiving their copies shortly, and you can read it on our website.
This month's theme is making sure you're putting your customer service efforts in the right places. Our first article is about how collecting more data isn't always a useful activity if it isn't the right data. Then we have an article about which customer experience efforts actually make a difference and why so many companies seem to focus on the low-value ones.
As always, I hope you find this useful and informative.

Treating Customers as Human

In a nice counterpoint to AT&T's "cease and desist" approach to customer suggestionsConsumerist has the aww-cute story of Delta's response to an 8-year-old's mailed suggestion. Rather than trot out the lawyers, Delta sent a friendly, personalized letter from an executive along with some Delta swag.

So instead of annoying a loyal customer and generating a slug of bad PR the way AT&T did, Delta gets some goodwill with a future customer and his entire family and just the sort of heartwarming spirit-of-Christmas story the media love to run this time of year.

All because Delta chose to respond to a customer's suggestion at a human level rather than as a legal threat to be promptly squashed.

I think the lesson is clear.

Apple Gets 998 Things Right and Two Things Wrong

Over the past few days both my wife and I managed to drop our iPhones and shatter the screens. I headed to Apple's website and clicked through the "Support" pages and easily found a price list for iPhone screen replacement showing that fixing our phones would cost only $109 for each phone.

Relieved that the price wasn't going to be $200 or $300 each, I clicked the "Start a Service Request" button. Apple's website stepped me through a series of easy-to-understand and beautifully-designed web pages asking for more information about my phone. Then came the page with the Repair Estimate: over $300 including tax and shipping to fix each phone.

Wait, Whaaaa?

I went back to the beginning to make sure I didn't miss any fine-print. No fine print visible, and the page still said $109 to fix a screen. It also clearly stated that broken screens are not covered by warranty, so that's not the issue. So I tried again, started a new service request, put in my info again...over $300 again.

Back to the beginning but this time I clicked the "text chat" button. It popped a new browser window which gave a mysterious error. Tried a couple more times with that, and got the same result each time.

Finally I clicked the "Have someone call me" button. My phone rang immediately and within a minute I was talking to a technician. I explained that I had to replace the shattered screen on my iPhone and asked how much it would cost.

"Let me look that up," he said. "One hundred nine dollars plus tax."

"That's what the website says, but every time I try to start a service request it says the price will be three hundred bucks."

"Yes, that's confusing and I get a lot of questions about that," the technician explained. "The way it works is that if your phone is repairable we will fix any problem for no more than $299 plus tax and shipping. If it costs less to repair we refund the difference. So if the only thing wrong with your phone is the screen is broken it will be $109 and you would get a refund of a little under $200. But if something else needs to be fixed the technician will repair that too and it will be more expensive."

This process was explained nowhere (that I could find) on Apple's website or as part of the service request, and from what the support technician said I was far from the only confused customer. But because of my confusion I spent far more time on this than I had to, and Apple paid for a support call that should have been avoided. I also came away feeling that this experience, while maybe not awful, wasn't as smooth as it should have been.

998 Things Right, Two Things Wrong

Apple, of course, is famous for its outstanding customer experience. But as I'm fond of saying, good customer experience is about doing a thousand little things right.

Here are some of the many things Apple did right in my support experience:

  • It was easy to find the price for replacing an iPhone screen.
  • The price was very reasonable and less than I expected.
  • It was simple to start the repair process from the website.
  • Apple collected my phone information easily with minimal effort on my part.
  • I was given multiple support options, and Apple didn't try to bury the option for phone support.
  • I was able to talk to a technician within a minute or two.
  • The technician was personable, had good phone skills, was patient, and was knowledgeable.
  • The technician was able to give me different support options and solve my problem quickly.
  • I am confident my phone will be fixed promptly at the price I was quoted.

Here's what went wrong:

  • The price prominently displayed on the website did not match what I was asked to pay and no explanation was offered.
  • Text chat wasn't working.

But even though almost everything about this support experience went right, those couple of support misfires wound up costing me time, costing Apple money, and added up to a meaningful amount of un-Apple-like frustration.

What's more, it was clear from the technician's comments that at least some people inside Apple know that this is a problem. So it's a little mysterious to me why Apple, again famous for its customer experience and attention to detail, hasn't noticed that the support center is fielding calls from customers confused about the cost to fix an iPhone screen and done something to relieve the confusion.

Given my general satisfaction with Apple, as a customer I'm not going to hold this against them for long (unless similar things keep happening in the future).

But even a company like Apple sometimes doesn't always get all the details right. And as my experience shows, getting even a couple details wrong can be costly in time, money, and customer frustration.

Cease And Desist From Your Customer Feedback

A couple weeks ago David Lazarus of the LA Times wrote a column about an AT&T customer who emailed the company president with a couple of suggestions (one being unlimited data for DSL customers, the other being a 1,000 text message bundle for $10). AT&T responded with a legalistic letter from the Chief Intellectual Property Counsel which, while not technically rude, didn't really match the spirit of the customer's suggestions:

AT&T has a policy of not entertaining unsolicited offers to adopt, analyze, develop, license or purchase third-party intellectual property ... from members of the general public.

Therefore, we respectfully decline to consider your suggestion.

When contacted by Lazarus, an AT&T spokesperson doubled down on this customer-hostile response, stating, "In the past, we've had customers send us unsolicited ideas and then later threaten to take legal action, claiming we stole their ideas. That's why our responses have been a bit formal and legalistic. It's so we can protect ourselves."

In other words, it's policy. Send a suggestion to the president, get a hostile response from the lawyer.

I have no doubt that some small number of mildly deranged AT&T customers have in fact threatened legal action in this kind of scenario. A company with as many millions of customers as AT&T has gets legal threats on a daily basis. But a legal threat is a long way from an actual lawsuit, and filing a lawsuit is a long way from actually winning damages.

But AT&T's response tells us a lot about the company's culture. From the outside it appears that AT&T management is so focused on the slight chance that a customer might file a frivolous lawsuit that they're willing to annoy or anger a lot of customers to mitigate the risk. Remember that these are customers who are trying to be helpful. And a company spokesperson--someone specifically given the job of communicating to the media--apparently didn't see anything wrong with this policy.

These actions seem to indicate a culture where customers are viewed as potential liabilities, not assets.

I'm sure that from inside AT&T the company views this entirely differently. AT&T leadership and employees probably genuinely believe that they value customers and manage them as assets, and that this kind of customer-hostile policy is a reasonable response to some bad things that happened in the past.

That just highlights why being a customer-centric organization is so hard. Remember that Managing Customers as Assets is one of the five key competencies required to be customer-centric. But it can be hard when you're steeped in a company's culture and constantly exposed to the internal logic that drives customer-hostile decision making.

Vocalabs Newsletter #92 is Published

The October issue of our newsletter, Quality Times, has been published and sent to email subscribers.

We lead off this month with an introduction to A/B testing in the context of customer experience, and some things to keep in mind if you want to use this powerful tool to help improve your Customer Experience (CX). Our second article is about an interesting attitude we've noted in a few of our clients, that no customer problem is unimportant or unfixable. Of course in the real world some problems really are more important or easier to solve than others, but approaching CX from that direction leads to some interesting places.

As always, I hope you find this useful and informative.

Big Data is the Industrial Byproduct of the 21st Century

I read a thought-provoking and contrarian perspective on Big Data a few days ago by Maciej Cegłowski, Haunted by Data. Maciej argues that data is like radioactive waste, in that it's extremely persistent and dangerous if leaked. He draws parallels between the hype and promises of big data today and the hype and promises of radioactivity a hundred years ago when people sold products like radium cigarettes and radioactive underwear.

Personally, I think this analogy is extreme. A more apt metaphor is that Big Data is the industrial byproduct of the 21st century. Like the sludge that spewed from factories in the 20th century, vast quantities of data are produced by almost every commercial activity today. Some of this data is valuable, but the overwhelming majority is worthless and potentially dangerous. And we are only beginning to appreciate the risks of these storehouses of data.

Unlike the physical kind of toxic goo, data is cheap to store and easy to destroy (as long as it remains contained). So there's a strong temptation to hold on to all data just in case some value is discovered in the future, but in many cases the responsible thing to do is get rid of it.

The problem with having all this data lying around is that, while any single piece of information may be fairly innocuous, we're finding out more and more often that it's possible to piece together lots bits of data to learn remarkably personal things. Anyone who knows your recent purchases can figure out not just your hobbies and interests, but also knows your medical condition including whether you or your partner is pregnant and whether you suffer from a particular illness. Anyone who has your list of Facebook friends also knows your sexual orientation and marital status and can probably figure out how faithful you are.

And let's not even think about what someone can figure out from your search history, the websites you've visited over the years, or the GPS tracking of your phone.

Fortunately there is a middle ground that lets companies find the value in their customer data and dramatically mitigate the risk of uncontrolled leakage: statistical sampling. We use it all the time in customer feedback, since it's usually not practical to try to survey every single customer.

It only requires a surprisingly small random sample of data to find a result that's remarkably close to what you would get if you look at all the data. Sampling 10,000 customers out of a population of a hundred million--looking at only 0.01% of the data--will almost always get within 1% of the result of looking at all the data. That means you can throw out 99.99% of the data and not get a meaningfully different analysis.

Of course the details of the statistical sampling matter, and it needs to be designed to meet the requirements of the particular analysis. But the key point remains that companies which keep all data just in case it might be useful someday are holding far more than they actually need, and creating a lot of risk to themselves and their customers in the process.

So think before you hold on to data. If it doesn't have a well-defined reason to be kept, you are probably just creating industrial waste.

And don't come back!

Another report of a car dealership's bad behavior in a customer survey: this time, a Ford dealer banned a customer because the customer gave honest but negative feedback.

It's as though the dealer doesn't actually care about providing a good experience, just getting a good survey.

Sadly, stories like this aren't even surprising anymore. The auto industry's survey processes are so heavily abused that it's almost more surprising if a dealer doesn't try to manipulate the survey.

I've written in the past that in situations like this, the company should just stop doing surveys rather than trying to fix such a broken process. The survey clearly isn't providing an honest measure of customer satisfaction, and all the cheating and manipulation is actively making the customer experience worse.

The Ford dealer who banned a customer for a bad survey is a great example of a company which has fallen into the "Metric-Centric Trap." The Metric-Centric Trap catches companies which, in an effort to become more customer-focused, become so caught up in measuring the customer experience that they lose sight of the actual customer.

Companies caught in the Metric-Centric Trap tend to focus their energies on gathering and improving their metrics rather than trying to understand and improve the customer experience. The problem with being Metric-Centric is that people are extremely complicated, and there is no way to directly measure all aspects of the customer experience. So any set of metrics is, at best, an approximate measurement of what's really going on.

Metric-Centric companies also tend to put heavy incentives on employees to improve their metrics. That can have the perverse incentive of encouraging employees to focus on the specific activities which are being measured, and ignore the things which aren't measured. And if it happens to be easier to "fire" an upset customer than train employees to do a better job, you get the situation with the Ford dealership.

Breaking out of the Metric-Centric Trap is not easy, and requires a significant cultural change. But companies caught in this situation often waste considerable time and money spinning their wheels on customer experience, and may even be making things worse despite as a result of the effort.

Vocalabs Newsletter #91 is Published

We just published the 91st issue of Quality Times, Vocalabs' newsletter about customer experience and surveys.

This month the theme is making the right decisions in Customer Experience. I discuss how good CX is about enabling and encouraging people throughout the organization to make a lot of little decisions the right way, as opposed to making a handful of big strategic decisions. That leads into one of the big strategic decisions many organizations get wrong: collecting survey data for the sake of collecting data.

This newsletter is one of the ways we get to know prospective clients. So if you find this useful and informative, please help us out by forwarding this to other people who might also enjoy it, and encourage them to subscribe. You can subscribe to this newsletter on our website.

Retention Departments Should Die

Retention departments can be some of the worst cesspits of customer experience, and it's not hard to see why. By definition, the role of a retention agent is to take a customer who has already decided to leave and make that customer not leave.

Rather than serving customers, a retention center tries to stymie the customers' express wishes.

And when retention agents aren't empowered to give customers what they want and are incentivized to maximize retention anyway, the result can be a toxic brew.

Witness, for example, the letter published in the Dallas Morning News from someone claiming to be an AT&T retention agent (which I heard about via Consumerist). The letter claims, among other things, that:

  • Each rep is given only allowed to offer a limited number of promotions per week, which are generally gone by the end of Monday.
  • Reps are expected to not only retain customers but upsell them, too.
  • The only way to meet retention quotas is to lie to customers and be a sleaze.
  • The culture promotes promising to do things for customers (such as cancel their service or provide a discount) and simply not doing it in order to meet retention targets.
  • Senior managers turn a blind eye to these practices.

AT&T's response to these allegations is basically that they don't know the writer is really an AT&T employee, and they train all their call center agents. Which is not really a response.

But to me, the claims seem entirely credible. I've heard of enough instances like this that none of the claims surprise me. The toxic combination of impossible retention targets, inadequate agent empowerment, aggressive accountability, and indifferent management is going to lead to problems every time.

This goes beyond just a bad experience. The reporter asked Daniel Lyons, a law professor with experience in telecommunications, to offer an opinion. Said Lyons, "if a company promises a customer incentives to either sign up for service or renew an existing contract and those incentives are not delivered, in many cases, that’s fraud."

So what's the alternative?

The key is that trying to increase market share by aggressive (or possibly even fraudulent) customer retention efforts is almost certainly counterproductive in the long run. That's because today's former customer might be tomorrow's new customer, but customers who feel they were mistreated are far less likely to ever want to come back.

For proof, just look at the experience of the mobile phone industry in the U.S. when number portability was introduced in 2003. Suddenly it was much easier for customers to change carriers, but the mobile phone companies quickly adapted with new ways to reduce churn (like the much-hated two-year contract). In the end, customers could leave without talking to a retention specialist and the industry did just fine.

Customers have lots of reasons for wanting to take their business elsewhere: it's not always about price. So if a customer wants (or needs) to leave, isn't it better to make that a positive experience rather than a negative one?

Bigger Data Is Not Always Better Data

"When in doubt, collect more data."

That could easily be the guiding principle of business in the year 2015. Collecting data is easy and storing it is cheap. You never know what insights might be gained from just a bit more data.

But like any simple idea, reality turns out to be more complicated. Not all data is useful, and while storing data is cheap, the tools and expertise to find those hidden insights turn out to be fairly expensive. And, as companies occasionally discover to their regret, data can be a liability as well as an asset.

I see this attitude in the customer experience world, too. Often it's a lot easier to just do more customer tracking or conduct more surveys than take action. There's a lot of data collection for the sake of data collection going on.

To be effective, customer surveys should have an underlying purpose. For example: to answer a specific question (i.e. "How many customers call us after logging in to the website?"), or to support a specific business activity (such as coaching employees, or tracking customers' satisfaction with their purchases).

Often, however, I see surveys designed backwards. Rather than starting with the goal of the survey, people will begin by thinking of all the things they might possibly find interesting and add all of them to the survey.

The result is usually a mess: a long survey where most questions are never really used for anything. This tends to drive the response rate down and make it harder to take action based on customer feedback.

So before you collect more data--whether that's enlarging a survey sample or adding more questions--take a few minutes to ask yourself:

  1. Is the new data likely to tell me something I don't already know?
  2. Do I know what I'm going to use the additional data for?
  3. When I consider all the costs of collecting additional data, including reduced survey response, customer goodwill, and the effort to analyze the results, is it worth the expected benefit?

If you can answer Yes to all three questions, not only is the data probably worth collecting but you've also got a good start on taking action based on the results. But if you answered No, it may be that you're collecting data for the sake of collecting data.

Does Your Effort Have Value?

There's a lot of different activities that go into an effective customer feedback program, but not all of those activities have equal value.

Based on my experience, some of the high-value activities are:

  • Closing the loop with individual customers
  • Coaching and training individual employees using voice-of-the-customer data
  • Discovering and improving customer pain points and broken processes
  • Disseminating customer feedback throughout the organization in a way that's relevant to each business user
  • A/B testing of different ideas for business processes

Some of the activities which tend to have less value include:

  • Calculating and tracking survey metrics
  • Paying bonuses based on survey scores
  • Disciplining employees for poor survey scores
  • Building high-level survey dashboards
  • Collecting survey data with little or no free-response feedback from customers

Keep in mind that less value does not mean no value. There is certainly some value in tracking survey metrics. But there's a lot more value in having a closed-loop process--and ideally a feedback process should have both.

And yet many companies seem to spend almost all their effort in low-value activities, completely ignoring the things which are most likely to lead to better customer experiences and a more efficient business. These companies have little to show for the effort they put forth to improve their customer experience.

The common thread among the low-value list is that they are all centered on improving customer survey scores as a goal in itself. A company which focused on improving metrics but ignores the underlying customer experiences has fallen into the trap of becoming metric-centric instead of being customer-centric.

It's easy to fall into this trap. Survey scores are concrete, quantitative, and measurable. Executives and managers can make the mistake of thinking that improved survey scores should be the goal, instead of a side-effect of achieving the true goal of becoming customer-centric.

In contrast, the activities on the high-value list are all focused on using specific customer feedback to directly improve the customer experience. Some activities, like closed-loop processes and coaching to the voice of the customer, work at the level of the individual employee or customer. Others, like A/B testing and identifying pain points, are higher level.

So just because your organization puts a lot of effort into the customer experience does not mean anything is likely to improve. You need to ask whether your activities are truly helping to understand customers' stories and improve the experience, or whether you've fallen into the trap of becoming metric-centric and spending a lot of effort on low-value activities.

What does it mean to be customer-centric?

Business writers like to talk about the benefits of being customer-centric. But what does it mean, and how do you know whether a company is customer-centric or not?

Like any other aspect of organizational culture, being customer-centric can be hard to define. There's no simple test or checklist that says you're customer-centric.

Being customer-centric is about considering the impact on customers in every decision the company makes. A customer-centric organization will:

  • Prioritize efforts that remove pain points for its customers.
  • Consider the impact on customers on decision-making throughout the organization, not just in the traditional areas of customer service and sales.
  • Train employees in all departments that the decisions they make can affect customers, including back-office functions.
  • Have leadership that takes an active interest in customer issues, both in aggregate and also individually.

These are all organizational outcomes, they are the things that come naturally to a customer-centric organization  as part of its culture.

Getting there is another matter. That's where the five competencies Jeanne Bliss talks about in Chief Customer Officer 2.0 come into play.

Amateurs Talk Strategy, Professionals Talk Execution

Amateurs Talk Strategy, Professionals Talk Logistics

That's an old military quote that sometimes gets pulled out at business leadership conferences. Strategy is the easy part. The hard part, the stuff the pros worry about, is the nuts and bolts of getting everything lined up and in the right place at the right time so the strategy can work.

It's an important message for customer feedback programs, too.

Developing a survey strategy is easy, and a lot of people have a lot of opinions on how to do it (some better than others).

But actually building an effective feedback program requires a lot of attention to detail. You need to:

  • Determine who to ask to participate in the survey
  • Decide what questions to ask
  • Determine the right time and channel to invite the customer to take the survey
  • Offer the survey to the customer in a way that makes the customer want to help
  • Route the survey responses to service recovery teams in real-time when appropriate
  • Coach front-line employees based on their individual survey responses
  • Deliver data to business users throughout the organization in a way that's timely and tailored to their individual needs
  • Monitor the survey process for signs of manipulation or gaps in the process
  • Adjust all aspects of the process on an ongoing basis as business needs change
  • Focus the entire organization on using customer feedback as an important tool to support both operational and strategic decision making

(As an aside: one thing not on this list is "Track your metrics and set goals," because tracking metrics is both easy and low-value. Everyone does it, but many organizations stop at that point in the mistaken belief that improved customer experience will magically follow.)

So just as military pros understand that wars are won and lost in the unglamorous details of moving people and supplies to the right place at the right time, survey pros understand that the effectiveness of a feedback program is built on the nitty-gritty of collecting and delivering the right data to the right people at the right time to help them do a better job.

What the amateurs don't recognize is that you can't just move an army on a whim, or improve customer experience by throwing some survey metrics at it.

So we circulated a Word doc...

So someone emailed around a Word doc with the survey design, and someone else edited it, then forwarded it to another person who copy-pasted it into the survey software, and the first person said it was good to launch so a fourth person uploaded the customer list and sent the invitations, and....wait, wasn't the Word doc already signed off? Why do we need to proofread it again?

Via The Daily WTF

No Customer Problem is Unimportant or Unfixable

In a couple of my clients, I've noticed an uncommon attitude towards the customer experience.

Where most companies often push back on trying to solve customer problems, these unusual companies take the opposite approach. They assume that No customer problem is unimportant or unfixable.

Compare that to the litany of reasons most companies give for not fixing their customer experience problems:

  • "Only a few customers are complaining about that."
  • "It would be very expensive to provide that level of service."
  • "That would require major investment of IT resources."
  • "That customer is just trying to get something for free."
  • "If we did that our customers would scam us."
  • "The way we're doing it now is better."
  • "You can't please every customer all the time."

What makes these excuses so insidious is that they are, very occasionally true. Some problems really do arise from freak circumstances (but usually if one customer complains, there are many others who have the same problem and aren't complaining). Sometimes systems are so big and outdated that it would be uneconomical to fix them (but at some point they will have to be replaced, and next time around you shouldn't let your systems get so far behind the curve). Some customers really are trying to scam you (but the overwhelming majority of customers are honest). And it is true that some customers will never be satisfied no matter what you do, but those customers are very rare.

Often one (or more) of those reasons is trotted out as a way to avoid taking a serious look at fixing some issue with the customer experience:

"What are we going to do about the complaints about how we verify customers' identities over the phone?"

"Only a few customers are complaining about that. Plus, if we changed the authentication then people would scam us."

"Oh, then I guess we shouldn't change that."

But if you take the attitude that No customer problem is unimportant or unfixable, then the conversation becomes completely different:

"What are we going to do about the complaints about how we verify customers' identities over the phone?"

"Only a few customers are complaining about that. Plus, if we changed the authentication then people would scam us."

"You might be right. But No customer problem is unimportant or unfixable, and this is definitely important enough to some of our customers that they took the time to complain. So we should at least explore some options and see if there's a better way to do things."

This attitude, that No customer problem is unimportant or unfixable, can dramatically shift a culture towards being customer-centric, especially when it comes straight from the top.

It's not an easy change, because it directly attacks the deep resistance to change in many organizations. But try making this your catch-phrase and see how it changes the discussion.

Doing a Thousand Things Right

Creating a good customer experience is often about doing a thousand little things right.

It's easy to lose sight of that fact when you're trying to think strategically about process improvement and engineering a better customer experience for your organization. Statistics can conceal the fact that behind every data point is a customer, and that customer received either a good experience or a bad one.

So while it's important to make sure the right processes are in place to enable a good customer experience, it's more important to make sure that the people who are part of those processes have the tools they need to make those thousand little decisions in the right way.

Every employee of every company is pulled in different directions by competing priorities. You have to balance things like working faster vs. more carefully; satisfying an upset customer vs. saving money; or solving a problem yourself vs. calling for help.

Even if a company says it cares about customer experience, what really matters is how employees are making those decisions on a day to day basis. To make the right decisions, a company needs to ensure:

  • Employees understand what customers expect and how to deliver it (you need good training)
  • Employees get regular, specific, and detailed feedback about how customers perceive the experience (you need a well-designed closed-loop survey)
  • Employees aren't pressured to make bad decisions (compensation needs to align with customer experience, or at least not pull the wrong way)
  • Employees know the leadership cares (customer experience needs to be an ongoing effort, not a one-time project)

This holds true for employees throughout the organization, not just the ones who deal directly with customers. A website designer or billing specialist can be subject to the same negative forces (work faster, save money, ignore the complaints) as a contact center rep or salesperson. If anything, back-office employees may be more susceptible to taking customer experience shortcuts since they don't have to deal with customers directly.

The good news is that most people genuinely want to do a good job, and if given the right tools and training and if shown that the company cares, they will be highly motivated to make the right decisions about customer experience.

If the leadership can just do a few big things right, it's not that hard for everyone else to do a thousand little things right.

Who has time to proofread?

Today's gem of a survey mistake comes to us via The Daily WTF.

I wish I knew where this was from, but it's maybe just as well that it remains anonymous.

A/B Testing for Customer Experience

A/B testing is one of the most powerful tools for determining which of two (or more) ways to design a customer experience is better. It can be used for almost any customer experience, and provides definitive data on which design is better based on almost any set of criteria.

Stripping off the jargon, an A/B test is really just a controlled experiment like what we all learned about in 8th grade science class. "A" and "B" are two different versions of whatever you're trying to evaluate: it might be two different website designs, two different speech applications, or two different training programs for contact center agents. The test happens when you randomly assign customers to either "A" or "B" for some period of time and collect data about their performance.

Conducting a proper A/B test isn't difficult but it does require some attention to detail. A good test must have:

  • Proper Controls: You want the "A" and "B" test cases to be as similar as possible except for the thing you are actually testing, and you want to make sure customers are being assigned as randomly as possible to one case or the other.
  • Good Measurements: You should have a good way to measure whatever you're using for the decision criteria. For example, if the goal of the test is to see which option yields the highest customer satisfaction, make sure you're actually measuring customer satisfaction properly (through a survey, as opposed to trying to infer satisfaction levels from some other metric).
  • Enough Data: As with any statistical sampling technique, the accuracy goes up as you get data from more customers. I recommend at least 400 customers in each test case (400 customers experience version A and 400 experience version B, for 800 total if you are testing two options). Smaller samples can be used, but the test will be less accurate and that needs to be taken into consideration when analyzing the results.

In the real world it's not always possible to do everything exactly right. Technical limitations, project timetables, and limited budgets can all force compromises in the study design. Sometimes these compromises are OK and don't significantly affect the outcome, but sometimes they can cause problems.

For example, if you're testing two different website designs and your content management system doesn't make it easy to randomly assign visitors to one version or the other, you may be forced to do something like switch to one design for a week, then switch to the other for a week. This is probably going to work, but if one of the test weeks also happens to be during a major promotion, then the two weeks aren't really comparable and the test data isn't very helpful.

But as long you pay attention to the details, A/B testing will give you the best possible data to decide which customer experience ideas are worth adopting and which should be discarded. This is a tool which belongs in every CX professional's kit.

Vocalabs Newsletter #90

I just published the 90th issue of Quality Times, Vocalabs' newsletter. This month I use the trend towards cord-cutting in cable TV to discuss what can happen in an industry where customer experience "doesn't matter" because of a monopoly or because customers have a hard time switching. I also offer my first-ever book review, of Jeanne Bliss' Chief Customer Officer 2.0.

Email subscribers should receive the newsletter shortly. As always I hope you find it interesting and informative.

Book Review: Chief Customer Officer 2.0

Longtime readers will know that I don't normally write book reviews on this blog. In fact, this will be my first. But when I was asked if I would review Jeanne Bliss' new book, Chief Customer Officer 2.0, it was easy to say Yes. Jeanne is a bona-fide guru of customer experience, and this update to her 2006 Chief Customer Officer is a book I was probably going to have to read anyway.

Chief Customer Officer 2.0 is a good introduction to the principles and practices of customer experience, aimed at the executive who needs to make it work (or the manager who needs to work with the executive to make it work). Jeanne lays out the five major competencies of a customer-centric organization: managing customers as assets, aligning around experience, building a customer listening path, being proactive in the experience, and one-company leadership/accountability/culture.

For each of these competencies, the book provides a description of what's involved, the benefits to be gained by achieving competence, a few anecdotes from various organizations' journeys, and a few ideas to get you going.

What Chief Customer Officer 2.0 won't provide you is all the nuts and bolts of how to execute each of these competencies. You will learn the importance of managing customers as assets and a general sense of what that means, but there's no accounting formula for establishing what that means to your company. You will understand the value of listening to customers as they travel along your customer journey, but this book doesn't instruct you on how to write a good survey.

Instead, Chief Customer Officer 2.0 sticks to the big picture. This is, in my view, the right approach. Each of the five competencies requires a lot of experience and getting a lot of details right in order to gain maturity. Trying to include all that in a single book would get bogged down in a level of minutiae and make it hard to grasp the larger picture.

If I have one complaint about this book, it's that Jeanne presents the five competencies as roughly equal in importance. While all five are important, I would argue that the fifth competency, Leadership, Accountability, and Culture, is the most important of the group. In my observations, companies which have the Leadership, Accountability, and Culture usually develop enough competence in the other four areas to drive a customer-centric organization (even if they never embark on a formal Customer Experience program). But if that leadership element is lacking, even a high level of maturity in other areas can go to waste because the organization doesn't care.

At the end of the day, if you or your organization want to become more customer-centric, this is a book you want to read. It won't give you all the answers, but it will give you the framework to start asking questions.

Vocalabs Newsletter #89

We published the 89th issue of Quality Times, our mostly-monthly newsletter. Email subscribers should already have their copies.

This month is a deep dive into survey methodology and process. In my first article I discuss a recent academic paper comparing the popular metrics NPS, CSAT, and Customer Effort and find that the research isn't all it's cracked up to be. Then I get into the subject of whether and when to pay an incentive to take a survey. My answer: "almost never."

As always, I hope you find this useful and informative.

Don't Treat Your Customers Like Mushrooms

"They treat me like a mushroom," as the old joke goes. "They keep me in the dark and feed me ****."

Nobody likes being treated like a mushroom, but some companies do that to their customers anyway. This week Spirit Airlines learned why this might not be a great idea.

Spirit, for those not familiar with the company, is the low-cost airline which has elevated poor customer experience to part of its brand image. In an industry where calibrated misery is an art form, this is saying something.

Over the past week, Spirit has been plagued with delays and cancelations. Customers didn't always buy the "bad weather" explanations offered by the gate agents. And let's face it, "weather" doesn't sound like a credible reason for a flight to be canceled when the plane is at the gate and both the departure and arrival airports are clear.

But Spirit makes it very difficult to talk to anyone at the company, and doesn't monitor social media (all part of that low-cost brand!). In the absence of any official word from the airline, rumors started to spread that there was some sort of pilot strike going on. Even when contacted by the media, at first Spirit didn't respond, leading some outlets to publish articles about the rumored labor action. "Pilot strike" is probably second only to "maintenance problems" on the list of things which will make passengers choose a different airline.

I have to think that this article (even when updated several hours later when Spirit finally gave an official explanation for the delays) did substantial damage to Spirit's bottom line. It creates the impression that the company doesn't care about its passengers even when stranding them far from home, and that the airline won't make an effort to keep customers informed when the schedule is becoming massively disrupted.

Surely it would have been cheaper just to communicate clearly in the first place.

Underpaid, Inexperienced Workforce

Bruce Temkin wrote a blog post recently about Walmart's plans to give a pay raise to many of its lowest-paid employees. Bruce's conclusion, after looking at this decision through the lens of customer experience, is that in the long run Walmart is likely to save money through lower staff turnover leading to better customer experience and more loyal customers.

I think Bruce is spot-on. The only part I disagree with is that this is surprising or counter-intuitive.

The simple fact is that employee turnover is expensive, inexperienced employees are more likely to make mistakes, and underpaid employees aren't likely to go out of their way for customers. People want to be valued in their jobs, and underpaying them to do menial work is a great way to communicate that they are not valued.

My guess is that by paying employees more, Walmart will save money through reduced turnover alone, nevermind any customer experience benefits. Over the longer term, better morale among employees could also pay dividends through better customer experience as a bonus.

We don't have to guess, though. Compare Sam's Club, a warehouse store run by Walmart, to Costco, a direct competitor known for relatively generous pay and benefits and low employee turnover.

Which has the better customer experience, higher customer loyalty, stronger reputation, and better financial results?

Dear Xcel Energy: Here's Why Nobody Takes Your Survey

Browsing the Xcel Energy website recently I was accosted by one of those ubiquitous popup surveys. You know the kind: one which asks if you'll take a survey about your browsing experience.

These surveys typically have an abominable response rate, and it's not hard to see why. Should you agree to take the survey you'll be asked to answer screen:

after screen:

after screen:

after screen:

after screen:

of questions. Thirty-two questions in all, of which 25 are required.

Clearly someone didn't get the memo about survey length.

What if you happen to notice a problem with Xcel's website, maybe a broken link somewhere, and you want to be a good samaritan and point it out to them?

Good luck with that. Because if you just write a short note in the comment box about the broken link and hit "Submit," here's what you get:

So much for being helpful (this is one of the reasons why Mandatory Survey Questions are Evil). If you're not interested in rating the "balance of text and graphics" on the site, or providing no less than three different ratings of the search function, Xcel doesn't want to hear from you.

Not that it would have mattered anyway: notice the text at the very bottom of the survey, "Please note you will not receive a response from us based on your survey comments."

To the customer that means, "Nobody will read the comments you write."

Xcel might as well save the bother of putting up a survey and just have a page that says, "We don't actually want your feedback but thanks anyway." It seems like a lot less trouble.

Shockingly Good CX from T-Mobile

It's often easier to find things to complain about in Customer Experience than examples of things that went well. But yesterday I had a customer service experience with T-Mobile that went shockingly smoothly.

Here's the setup: I needed to make a small change to my T-Mobile plan. I logged in to my account online and tried to make the change, but got an error message that said I needed to call customer service for help.

So I called customer service. Here's what happened:

  1. The speech recognition system offered "Representative" as the very first option.
  2. I was not told to go to the website for faster service.
  3. I was not asked to play Twenty Questions before being routed to a person.
  4. I was asked to authenticate myself by entering the last four digits of my SSN. When I was connected to the representative, she already knew I was authenticated and didn't ask me to authenticate again.
  5. The CSR was pleasant, friendly, and relaxed. She did not try to rush me, and took a moment to chat.
  6. The representative immediately understood my problem, fixed it, and explained how long it would take for the change to show up in my online account. She even saved me a couple bucks by tweaking the effective date of my plan change (something that's not possible online).
  7. A few hours later, as promised, my plan change was visible when I logged in online.

None of this is rocket science, but it's amazing what a difference paying a little bit of attention to the customer can make.

Syndicate content