The Customer Service Survey
VocaLabs' weblog providing news and commentary on the challenges of providing good customer service.
How to Design a Phone Menu
Wednesday - March 26, 2008 03:55 PM

The conventional wisdom for decades has been that the ideal phone menu (aka IVR tree, aka callflow) should have no more than five options at each level. Any more than that, it was thought, would overtax the short term memory of the caller and lead to confusion and poor experiences.
Never mind the clearly observable fact that menus designed according to this rule of thumb still led to confusion and poor experiences.
Broad vs. Deep
As far as I know, however, nobody had ever experimentally tested the question of how broad (=lots of choices at each level) or deep (=multiple levels and submenus) a design should be for a given application. Until now.
A recent article in the journal Human Factors, A Comparison of Broad Verses Deep Auditory Menu Structure, by Patrick Commarford and James Lewis (both of IBM) et al. tackles the question of broad vs. deep menus head-on and comes up with the surprising result that (at least for the application they tested) it works better to give callers a single menu with lots of options than to give callers fewer options in each menu but have submenus..
In other words, the "no more than five options" rule of thumb is wrong.
The particular application Commarford and Lewis used for this experiment is navigating e-mail through a speech application. They identified 11 different functions (Next, Previous, Delete, Reply, etc.), and built a "Broad" and a "Deep" version of the e-mail system. The "Broad" menu provided all options in a single menu, and the "Deep" version offered four top-level choices (Listen, Respond, Distribution, and Details) with the 11 options contained in the four top-level choices.
There were some design oddities about the "Deep" menus: for example, the "Delete" function was under the "Respond" menu (I can easily write a whole blog entry about the uniquely user-centric design process which led to this, ahem, unique design decision). In the final analysis, though, I think it's fair and reasonable to conclude that for this application a broad design is probably superior to a deep design, even though that leads to more than five choices in a single menu.
What Does It Mean
Some people may read this result to mean "make the menu as flat as possible," presumably by putting every option into one single uber-menu, but I think what it really means is "there's nothing magical about five menu options." There will always be a design decision about how broad or deep to make a tree, unless the system has few enough choices to put in a single menu. If the IVR needs to route the call among hundreds of different destinations or provide several dozen functions, a single menu will simply be unwieldy (of course, one thing to look at carefully in those situations is whether the total number of functions or destinations can be consolidated).
The key to usability is making sure that the menu choices are obvious (which may require including some choices in more than one menu, if they don't fit cleanly in one place), and that the caller's time and mental effort is respected. That means striking a balance between broad and deep, and the right balance will be different in every situation.
The real lesson of this research is that real-world usability testing is likely to show that general rules of thumb break down when applied to a specific design. At the end of the day, usability testing is critical to making sure that the design is right.
Simple Solutions
Monday - March 24, 2008 02:52 PM

There are some lessons which I need to learn over and over and over. The old engineering advice to KISS your problems ("Keep It Simple, Stupid") is one of them.
The problem, in this case, was how to include the name of an agent's supervisor in our database of scores from a Quality Assurance review. We already receive the call recording and the name of the agent from the client's telecom system, but the name of the supervisor isn't available in any system currently interfaced to the telecom system.
We started to research options, such as building a client-specific web interface for maintaining an organizational chart in our database, or auto-populating from a periodic batch dump, either of which would require several days (at a minimum) to develop and test, and some unknown amount of ongoing maintenance.
Then, "Hey, why not add one question to the QA review for the reviewer to enter the agent's supervisor? It's a small call center, they've only got a handful of supervisors, and the reviewer sits on the same floor as the agents--they've got better information about the org chart than anyone." A five minute change which, while not fully automated, will give exactly the results the client wants with almost zero headache.
KISS, indeed.
Posted by Peter Leppik
Factors for Good and Bad Service
Thursday - February 21, 2008 01:41 PM
We gave participants a list of factors which they think are important for providing good customer service. The number one choice? "Make it easy to reach a live person (if necessary)," selected by 79% of participants so far. The #2 choice is "Be courteous, polite, and professional," chosen by 73% (participants are allowed to choose more than one option).
So far, nothing too surprising, though it is interesting to note that making it easy to reach a person was selected more often than "Wait only a short time to talk to a person" (selected by 67%). Consumers seem to be saying it's less frustrating to wait on hold than having to hunt for the option to talk to an agent.
Another question asked participants to choose factors which are most important in bad service. The top two options (a statistical tie at about 68% each) were "When you have to call more than once for the same problem, you have to start over from scratch each time," and "You're forced to go through repetitive or irrelevant steps to get help."
Given that these choices beat out even such dramatic problems as "The person you spoke to was rude or unprofessional," and "The company doesn't do things it promised to do, like fix a problem, issue a credit, etc." is really intriguing, and I'm guessing that the top two choices won out because they are much more common problems. Companies have (for the most part) gotten pretty good with call monitoring, coaching, and other QA problems, so rude agents and broken promises don't happen as often as they once did.
On the other hand, the top two bad service issues have the common thread of being systemic to the operations of a call center. The roots of these problems are buried in a company's disinterest in looking at its customer service from a customer's perspective, so nobody ever asks key questions like, "How does this call come across to the customer from end-to-end, when I consider the IVR, agents, transfers, etc. as a whole rather than individually," and "Is this step necessary from the customer's perspective, or is it just a hoop we're making him jump through for the convenience of the company?"
I'll have more on this survey data in a few days when it's completed.
Posted by Peter Leppik
Confusing Statistics
Thursday - June 28, 2007 10:49 AM
For example, the same statistical distribution is often called a "Gaussian Distribution" by physicists, a "Bell Shaped Curve" by social scientists, and a "Normal Distribution" by mathematicians. In other words, one calls it by the name of one of the guys who developed the math, another uses a descriptive term, and the third makes a value judgment.
(Thanks to David for this particular example).
Consider also the use of the terms "average," "mean," and "expectation value" to refer to identical statistical concepts.
In many cases, the same statistical methods were first developed by guys in the Renaissance trying to figure out how to win at dice, rediscovered in the 19th century by physicists working on thermodynamics, and then discovered a third time by social scientists.
The result is unnecessary confusion and overlap: a "Statistics for Finance" textbook will contain much the same material as a "Statistics for Physics" text, but because they use different names and notation, it may take a while for a student to realize that he's re-learning the same concepts.
Posted by Peter Leppik
Ten Dumb Things about Customer Service
Monday - March 12, 2007 03:42 PM
- If you hear "this call may be recorded for quality assurance," you can be assured that nobody ever listens to the vast majority of call recordings.
- Most recording systems aren't even capable of recording the entire call, and only catch the live agent portion.
- Many call center agents are paid more to hang up on you than to take care of your problem.
- Most automated customer service systems are never tested for ease-of-use, even after customers complain.
- Most companies at least review satisfaction survey results, but very few ever make any changes based on the feedback in the survey.
- In many companies, the call center does not communicate with other departments: Marketing doesn't inform customer service about new promotions or ads, product development doesn't hear about customer complaints, etc.
- Many companies view customer service as a "weed-out" department, where inexperienced employees are either quickly promoted (if they show promise) or fired (if they don't). Few companies have a meaningful career path which stays inside customer service.
- Senior management is often genuinely surprised to learn that the front-line customer service representatives are inexperienced, unqualified, and/or unmotivated.
- The vast majority of corporate executives believe that their company provides above-average customer service.
- Live agents and automated customer service are usually managed completely separately, even though a typical customer involves both. Nobody ever reviews an entire call from end to end.
Posted by Peter Leppik
Control
Thursday - February 22, 2007 11:04 PM
But during the session, something which has been banging around in the back of my head for a long time finally crystalized: The key to excellent customer service is letting the customer control the call.
The biggest customer complaints about bad service--especially bad automatedcustomer service--generally revolve around the company trying to force the customer down a path the customer doesn't want to follow. Roadblocks to reaching a live agent are the biggie, of course. But overly-scripted agents, asking irrelevant questions, and even unexpectedly transferring to an agent when the self-service system fails are all in the same category.
Before a customer calls, he already knows what he wants to accomplish, and (often) an idea of the steps he needs to follow to get that task done. Maybe the customer's idea of how to get the task done isn't really practical, but the idea exists, and anything which pushes the customer off his mental "script" of the call is going to cause frustration. If he's calling about a billing problem, he expects to have to talk to a human and explain the problem. If he's calling to find the hours of a local bank branch, he doesn't expect to have to enter an account number.
The challenge for the company is to handle that customer's call efficiently without knowing in advance why the customer called. What's more, the customer is a whole lot smarter than the automated customer service system which (in the majority of cases) will first attempt to handle the call.
So by letting the customer keep control of the call, the company is both taking advantage of the customer's knowledge of why he called, and also the customer's superior intelligence in how to handle the task.
In many cases, the philosophy of letting the customer control the call is mostly, well, philosophical. An automated call will always progress from state to state based on customer inputs; and customers will often want to do things which are impossible or against business rules. But there are some practical implications:
* Don't try to force customers down a particular path. Encourage where appropriate, but always leave an out: "You can make your balance transfer in our self-service system, or dial zero at any time to connect to an agent. Let's begin...."
* Make sure there's always a way to back up, start over, or bail out.
* Ask for information only when needed.
* Enlist the customer's help by explaining what's going on and why. For example, if the speech recognition is having a hard time, don't just drop the call into an agent queue, give the customer a choice: "I'm sorry we're having so much trouble on this call. If you want to start over with a live agent, just press zero at any time. Otherwise, we can keep trying self-service...."
Posted by Peter Leppik
Effective Surveys
Friday - February 09, 2007 01:57 PM
This is a problem I've observed a lot, and the bad news is that there's no simple solution.
In order for a survey to be effective, it has to be part of a much larger feedback loop, which I made the focus of Chapter 2 of Gourmet Customer Service. The steps are:
1) Set Goals
2) Gather Data
3) Take Action
4) Validate Improvements
5) Go back to step 1
The surveys take place in steps 2 and 4 (validation means measuring whether the action in step 3 had any actual effect). The challenge is that this loop has to be something that the organization as a whole is committed to. Steps 1 and 2 are often as far as anyone gets, because if there's no commitment to change, it's always easier to do nothing than something. If there's a short-term push for improvement then a company may get as far as step 3, but without validation there's no way to know if the changes actually improved things, and without closing the loop by setting new goals, any improvement is likely to be short-lived.
The good news is that this process isn't rocket science. The magic comes in the commitment to make changes, measure results, and then go back and make more changes--there's nothing special about the expertise of the consultants or the three-letter acronyms used. Over time, even a marginally competent department will figure out which changes are improving things and which aren't, and improve the overall service level.
So hard work and corporate commitment? Yes. Special techniques and magic consulting wizards? Not necessary.
Posted by Peter Leppik
SectorPulse, the Background
Tuesday - January 16, 2007 01:21 PM
Our Press Releases on SectorPulse show a head to head comparison in the categories of overall caller satisfaction as well as the probability of completing a callers business without having to call back. But we also track five additional categories: Automation is the percent of callers able to complete their business without speaking with a live agent. Caller Frustration is a measure of the difficulty in reaching live help. Call time, or more accurately named "Transaction time" measures the total amount of time including any multiple calls required to complete a piece of business. And we track customer loyalty both pre and post phone call.
A summary of the comparative results in these latter five categories is a part of the available Executive Summary, and in varying levels of depth, we can also provide historical data, and consumer feedback to better understand why/how caller opinions get formed plus the impact good/or poor service has on company revenues.
The same criteria apply to SectorPulse-Financial that tracks customer service quality among major US banks plus PayPal.
Feel free to call or drop us a note any time on how SectorPulse might be used and how we can aid in making good service a more likely result.
Posted by Rick Rappe
Net Promoter- The Ultimate Question
Thursday - October 12, 2006 02:07 PM
Great question. We've been asking it for years, but missed the opportunity to get our 15 minutes of fame by writing a book about it.
The topic came up again in a conversation yesterday, with the unfortunate theme that asking this question was all the customer satisfaction surveying that needs to be done. Sigh... Would that this were true.
Had we written the book, we would have said that "Would you recommend..." is a fine high level indicator of company image, but that it is an inadequate measure of customer satisfaction. While it answers how customers feel, it doesn't tell you what you did, right or wrong to cause the opinion to form, why it happened and when it happened. Without knowing what, how, why and when, how can you improve?
Posted by Rick Rappe
"Cecil, Connect Me To Kellogg-257 Please."*
Wednesday - September 20, 2006 04:02 PM
My problem is that the author skirts around WHY Paul is right, and he makes some questionable comments in the process.
People have no problem with self service customer care so long as it meets their needs. It is bad IVR that is the cause of the problem. No vendor including Genesys is blame free. This is a subject I've written and spoken about many times: "If you build a system to save money over serving the caller, it will do neither. But if you build it with a caller focus, it will do both."
In attempting to explain the "symbiotic" relationship that should exist between the IVR and the human CSR, he uses the recent Citibank "dial 0"-get a human advertising campaign as both evidence of consumer backlash against self service customer care and as evidence of this symbiotic relationship. Perhaps this wasn't such a good example. Recall that VocaLabs has a constantly running survey on service quality among the nationwide banks, and Citibank has consistently had the lowest overall satisfaction scores.
During the period that the dial "0"- get a human ad campaign was running, all the banks except Citi improved their satisfaction scores, Citi's dropped such that caller satisfaction with Citi was actually worse after a service call than immediately before. I have not investigated deeply enough to make an absolute statement, but my sense is that the ad campaign raised caller expectations, and when they actually called and found service wasn't any better, Citi customers were more disappointed than had the campaign never run. (In fact, I suspect that more callers tried to reach a live operator after seeing the ad and Citi failed to add enough people to the phones to allow for it.)
The author then moves on to tout how speech technology allows the humanization of the computer voice, inducing the caller to stay with the machine. And he accepts this as a positive. Not necessarily. There is a quite strong argument that giving the machine human attributes can be a bad idea; that it insults the caller since they know they are dealing with a machine incapable of real empathy, and that humanization actually distracts from the caller completing their business with a minimum of fuss.
He then closes by comparing our resistance to talking to a machine to a half century ago when there was some resistance to dial telephones replacing speaking the phone number to the local operator to be connected. We think this is a poor analogy, as Peter said to me this AM: "If only speech recognition DID work as well as dial phones, then there would be no need for a Paul English."
[*In the 50's our small town operator was a lady named Cecil Stuart. The town was so small Cecil could recognize voices and where the call was coming from. Didn't really need numbers. All I had to do was pick up the phone and ask for Grandma, and she'd know what connection to make. We knew enough not to call after Cecil went to bed unless it was a real emergency because she'd tell our parents.]
Posted by Rick Rappe
Fool Me Once, But Not Twice
Thursday - August 24, 2006 06:37 PM
So Witness has continued to add value to their product with capabilities such as matching the screen images the CSR was looking at with the call recording. Their systems now integrate with CRM technology so that reports can be generated including client data. They now include the ability to monitor e-mails as well as phone calls, and are even beginning to deploy speech analytics to search through the call recordings for key words and phrases to lower the need for a human to monitor every recording. Impressive stuff.
But when the Witness speaker got to that point in his talk about the importance of including caller surveys in the quality assurance process, that's when I fell off the bus. Statisticians call it "confirmation bias" which simply means that if we hear or read something that fits our belief system, we easily assimilate the new information. But if we hear something that doesn't readily fit, we tend to discard the data. A corollary to confirmation bias is that we might be willing to accept the pronouncements of a speaker until they say one incorrect thing on any subject for which we hold strong opinions or good knowledge. Our brains say: "Oops, if I can't trust the speaker on this thing I know about, what else has he/she said that might also be wrong?
In this case, one of the opening slides and remarks was that a satisfaction survey can't tell you about brand loyalty. Horse apples. Of course it can, you just need to know what and how to ask the questions. So my warning antenna was already up and when the speaker talked of the importance of surveys in the quality assurance process, I began to soften. But he quickly blew it by recommending the end of call survey (which Witness has the technical ability to provide), without any concession to the fact that this is a terrible method by which to gauge true caller opinion. (It excludes anyone who hangs up or the CSR steers away and so tends to skew satisfaction scores upward by an estimated 20-40%.)
That's too bad. I want to continue to think highly of Witness Systems technology, but since I can't trust them on this point, where else might I be wrong about them?
Posted by Rick Rappe
Oh Great! Still More Departments That Won't Talk to Each Other.
Friday - August 18, 2006 04:35 PM
This was a networking gathering of several marketing oriented membership groups, and what struck me was how segmented "marketing" has become. I won't get all the association acronyms right, so won't try. But as I understood it, there are associations for PR professionals, another for advertising, another for marketing copy writers, another for marketing managers, another for consumer affairs and still another for "sales and marketing". Whew. In any given day, I do ALL those things. Now, I can concede that in some organizations there needs to be a differentiation between sales and marketing. I can also argue that everybody in a company is, or should be, a marketer of their business. We're in a more specialized business world today, but aren't six different local association chapters to break up what used to be a single discipline a bit extreme?
Or am I just behind the times?
Posted by Rick Rappe
The Message is Getting Out
Tuesday - August 15, 2006 02:37 PM
I wrapped up the discussion by including the admonition that while pleased that the industry is now recognizing the importance of user focus and thorough design tests of IVR applications, it is important to understand how-what-when and why to test so as to avoid misleading results. I pointed out that surveying and consumer feedback assessment is a science itself and used the examples of the end-of-call survey that inflates satisfaction scores* and that later feedback studies are limited in the delivery of actionable improvements because humans forget details so quickly.
*Sample bias arises because dissatisfied callers who hang up never get to the survey, and agents can manipulate who participates. The result is our estimation that the end-of-call survey inflates true caller satisfaction scores by 20-40 percent.
Posted by Rick Rappe
[UPDATE: 8/25/06 Corrected the name of Melissa's company.]
What Do You Call a Room Full of Engineers?
Friday - August 11, 2006 09:51 AM
More thoughts from SpeechTek later. Stay tuned.
Posted by Rick Rappe
Doing the Wrong Things Excellently
Wednesday - August 02, 2006 11:53 AM
Time and again we encounter systems that work fine technically, but that fail to meet caller needs. It is more often bad design, not faulty technical performance that frustrates callers, and Traversal Testing tells you little to nothing about whether a technically functioning system is actually serving the caller well.
For example, IQ Services does automated testing of self-service systems, and they have many very happy clients. Their special area of expertise is system monitoring for technical performance. Are all the trunk phone lines working? Is the link between the system and the billing computer for access to customer records efficiently communicating the data?
IQS has just announced a new traversal test service they call Feature/Function Testing. Good for them. But they crank up the marketing hyperbole a bit too much when they claim that: "Feature/Function Testing will give you confidence that...your customers will have the best possible experience using your system."
We have tested hundreds of systems for usability, and so say with confidence that a system can be doing the wrong things just fine. In order to truly provide the best customer experience, you need to performa a variety of tests, including performance testing and traversal testing, and also usability testing with live callers, and ongoing caller surveys.
Anything less is like looking at the world through a paper-towel tube.
Posted by Rick Rappe
I'm Not a Consultant, But...
Friday - July 28, 2006 02:13 PM
I went on to say that the flip side is that it is impossible to have the above experience and not form some strong opinions; that I would be more than happy to meet with the client, but that I would not set VocaLabs up as experts in customer service automation design, best practices or anything that gave the appearance of favoring one vendor.
They agreed, and stressed that the independent third party view was exactly what was needed, and no issue, even if negative, was taboo. So I've accepted, and a call came in a few minutes ago saying that a meeting is being arranged for next week.
Here is what I will tell the prospect. It is advice applicable to anyone considering the installation of a large Speech Recognition customer service application:
- If you build it with a focus on saving overhead versus serving the caller, it will do neither. But if you build it with a customer service focus over cost control, it will do both.
- Humans actually prefer self service over speaking with an agent if the application serves their needs well. It is not "talking to a computer†that people object to; but badly designed automation that we all love to hate.
- Have an intra company meeting to get all departments to buy in and have input on the design. But then have one leader to get it done so as to avoid the "too many chiefs" problem that leads to mediocrity.
- Trust the designers. They have a good handle on best practices, how to phrase instructions, the impact of the voice persona the computer will use, etc.
- But, getting a second opinion on design issues is never a bad idea. Voice User Interface designers (VUIDs) at both Nortel and Nuance are top shelf, but they are also engineers at heart and this can lead to problems. (We've seen too many instances of clever designs, that didn't focus first on caller needs.) If pressed, I will recommend a particular consulting firm I consider the leading experts in self service automation best practices and usability design.
- Test, test, test. During the design phases, at pre-launch, and regularly once the system is operational. Things change. Over time user expectations rise, new products are sold, new marketing strategies unfold; the system is adjusted, and eventually service and caller satisfaction drops, and no one is quite sure why. We call it "application drift" and the best solution is a constant test-revise-test again plan that keeps the system performing.
Posted by Rick Rappe
Selling the Problem a Solution to the Problem
Friday - July 07, 2006 01:51 PM
While that is a harsh view coming obviously from a marketing oriented perspective, and we regularly hear contact center management complaining about marketing not keeping customer care managers in the loop; in the final analysis that PR executive was making a very valid point. In a follow up e-mail to me, that same PR executive said:
"I can tell you that the phenomenon is more than just anecdotal. We work with a LOT of marketing executives in our day-to-day lives, and "access to the customer" is always a point of pain--whether that access is guarded by sales, distribution or customer support. You will definitely touch a pain point if you address how the contact center gives access to customer insights that marketing isn't tapping fully."
One of the surprising statistics I encountered when first joining Vocalabs was a study that showed that a staggering 93% of callers into customer care say they form their opinions about a company based on the quality of service they receive. Not advertising, sales, or public relations, but customer service!
Yet I can't think of a single organization in which customer service reports directly to the head of marketing or sales. And that just doesn't make a lot of sense. Regardless, if any reader has experience or insights on selling customer service satisfaction testing while avoiding what might be seen as an end run around customer service management, I'd love to hear from you at rrappe@vocalabs.com Thanks.
Posted by Rick Rappe
It's Our Fault
Monday - June 26, 2006 02:29 PM
People are smarter than machines. They know when self service will meet their needs and when they need to speak to a real human. So when systems are built to force use of self service with such tactics as hiding the "opt out" to a live person, the caller will try and beat the system and will often call several times to try and get around the computer. Repeat calls are expensive just for the added telecom costs, and so defeat the save money motive. And if the caller fails to get their business done, they will frequently give up and simply take their business elsewhere. So if a customer service operation is sophisticated enough to realize and measure the impact poor service has on repeat business and customer retention, a case can reallybe made that forced use of automated technology in order to save overhead is a false and very foolish strategy.
Posted by Rick Rappe
Inherently Bad Service
Friday - June 02, 2006 02:34 PM
The Two Logo Problem is when you buy a product and there's two (or more) logos on it. For example, you buy a computer, and it says "Dell" and "Microsoft." Dell provides the hardware, and Microsoft provides the software. Or you buy a mobile phone, and the phone reads "Verizon" and "Nokia."
So when something goes wrong, where is the customer supposed to go? To Dell, or to Microsoft? If everything is working properly, the customer should be able to get good service whatever happens.
But in practice this doesn't always happen. If business is a little slow this quarter, or Wall Street is putting the screws on for better profitability, it can be all too tempting for one company to dump some of the support costs on its partner: "PC running a little slow? Sorry, you'll need to call Microsoft on that one. It sounds like a software problem." This doesn't even have to be a management policy; it may simply be the consequence of pressuring customer service reps to reduce call times and support expenses.
(By the way, don't read anything into the fact that I'm picking on Dell, Microsoft, Verizon, or Nokia in particular--they're just convenient, recognizable names.)
Inevitably, the customer feels like she's getting the runaround.
On the other hand, if there's only one logo on the box--for example, "Apple," which has consistently scored at the top of the pack for customer satisfaction with its technical support--the customer always knows where to go for help, and the company can't push her off to some other company.
The lesson is obvious: business partnerships are great, but make sure the customer knows which company she's doing business with.
Posted by Peter Leppik
Aren't We All Average?
Friday - March 31, 2006 03:36 PM
I bring it up because I'm chipping at the edges of a profound business thought as it relates to customer care, and specifically the design of self-service technology, to wit: That a key to personal as well as business growth is humility. Regardless of how you might be dazzled by your own brilliance, if the customer can't use your design; if it fails to delight, then swallow your pride and do what it takes.
If you can honestly say that the more experience you develop, the less you know, maybe THAT is a sign that you've really become good at something.
Posted by Rick Rappe

