Using the Customer's Voice in IVR Usability Testing

Moderated by: Dale Graff, Vanguard Communications

Nothing says “voice of the customer” quite like having the customer voice be the reason they are making a call for customer service within 12 seconds of reaching the IVR.  Let me explain.  Back on July 7th I described how I am working on a project that uses Natural Language Understanding (NLU) as a front door to an IVR system.  The system was brought in for Wizard of Oz usability testing recently and the project received very valuable feedback that will help a health insurer reach much higher self-service rates for members calling with questions.  Here are some quick observations from the testing.

First, the testing validated the strategy of using an open ended question (“In just a few words, tell us why you called”) to avoid multiple tiers of menus.  What could members of a health insurance plan call about?  Dental eligibility, vision benefits, medical claims issues, ordering replacement ID cards – even things that the health insurer doesn’t handle like setting up an appointment with the member’s own primary care physician – these are the subjects of calls.

Participants in the usability tests, who were actual members of the health insurance company, all commented on the speed with which their issues were addressed.  Said one participant, “If I knew ordering ID cards was this easy by phone, I wouldn’t use the internet site.”  Even when the “in just a few words” prompt was ignored and the usability test participant provided their life story as a preamble to their request to check on a denied claim, the NLU function did an excellent job at directing the call to the claims functionality.

Lesson learned here: If you want callers to stay in the IVR and self serve, have them voice their issue early and avoid time-wasting announcements, disclaimers, and confusing menus that frustrate callers and make them want to transfer to agents.

Second, the usability testing occurred later in the development stage as opposed to the design stage.  This resulted in being able to use the actual voice-user interface (VUI) software running on a small server with the real voice files from the selected voice talent.  Doing the usability testing in the design stage is great for validating call flow and menu wording before much development effort is expended.  However, using the actual system for usability testing reveals much more.  While the NLU opening prompt was successfully demonstrated, the testing revealed issues with how member ID numbers were being interpreted.   As a result, the strategy for collecting the membership number will be changed and a major potential issue will be avoided.

Armed with the results of the usability testing, the project team is encouraged that we are on the right track to dramatically improving the member self-service rate for this health insurer.  Now that is something to be thankful for this holiday season.

If you have any comments or questions, reach out to me.