The Customer Service Survey
Fool Me Once, But Not Twice
Thu - August 24, 2006 06:37 PM in
I just finished sitting in on a Webinar presented by Nortel and featuring their relationship with Witness Systems. The title "Learning How to Measure Customer Loyalty" had caught my attention. In sum, the presentation didn't exactly match the title, but I do have some observations to share.
I want to like the products of Witness Systems. They haven't rested on their laurels, quickly realizing that sales of call recording technology stalled because it has been too human resource intense to listen to many thousands of contact center calls in order to find problems. In truth, I regularly hear that companies actually only monitor 1-5% of calls, and in more than a few cases, not even that.
So Witness has continued to add value to their product with capabilities such as matching the screen images the CSR was looking at with the call recording. Their systems now integrate with CRM technology so that reports can be generated including client data. They now include the ability to monitor e-mails as well as phone calls, and are even beginning to deploy speech analytics to search through the call recordings for key words and phrases to lower the need for a human to monitor every recording. Impressive stuff.
But when the Witness speaker got to that point in his talk about the importance of including caller surveys in the quality assurance process, that's when I fell off the bus. Statisticians call it "confirmation bias" which simply means that if we hear or read something that fits our belief system, we easily assimilate the new information. But if we hear something that doesn't readily fit, we tend to discard the data. A corollary to confirmation bias is that we might be willing to accept the pronouncements of a speaker until they say one incorrect thing on any subject for which we hold strong opinions or good knowledge. Our brains say: "Oops, if I can't trust the speaker on this thing I know about, what else has he/she said that might also be wrong?
In this case, one of the opening slides and remarks was that a satisfaction survey can't tell you about brand loyalty. Horse apples. Of course it can, you just need to know what and how to ask the questions. So my warning antenna was already up and when the speaker talked of the importance of surveys in the quality assurance process, I began to soften. But he quickly blew it by recommending the end of call survey (which Witness has the technical ability to provide), without any concession to the fact that this is a terrible method by which to gauge true caller opinion. (It excludes anyone who hangs up or the CSR steers away and so tends to skew satisfaction scores upward by an estimated 20-40%.)
That's too bad. I want to continue to think highly of Witness Systems technology, but since I can't trust them on this point, where else might I be wrong about them?
Posted by Rick Rappe
So Witness has continued to add value to their product with capabilities such as matching the screen images the CSR was looking at with the call recording. Their systems now integrate with CRM technology so that reports can be generated including client data. They now include the ability to monitor e-mails as well as phone calls, and are even beginning to deploy speech analytics to search through the call recordings for key words and phrases to lower the need for a human to monitor every recording. Impressive stuff.
But when the Witness speaker got to that point in his talk about the importance of including caller surveys in the quality assurance process, that's when I fell off the bus. Statisticians call it "confirmation bias" which simply means that if we hear or read something that fits our belief system, we easily assimilate the new information. But if we hear something that doesn't readily fit, we tend to discard the data. A corollary to confirmation bias is that we might be willing to accept the pronouncements of a speaker until they say one incorrect thing on any subject for which we hold strong opinions or good knowledge. Our brains say: "Oops, if I can't trust the speaker on this thing I know about, what else has he/she said that might also be wrong?
In this case, one of the opening slides and remarks was that a satisfaction survey can't tell you about brand loyalty. Horse apples. Of course it can, you just need to know what and how to ask the questions. So my warning antenna was already up and when the speaker talked of the importance of surveys in the quality assurance process, I began to soften. But he quickly blew it by recommending the end of call survey (which Witness has the technical ability to provide), without any concession to the fact that this is a terrible method by which to gauge true caller opinion. (It excludes anyone who hangs up or the CSR steers away and so tends to skew satisfaction scores upward by an estimated 20-40%.)
That's too bad. I want to continue to think highly of Witness Systems technology, but since I can't trust them on this point, where else might I be wrong about them?
Posted by Rick Rappe
Posted at 06:37 PM | | | | |