/blog
For the average small-business owner, there are not enough hours in the day. The to-do lists can be endless, full of tasks that require a great deal of time and focus. Take bookkeeping for example. Though it is an incredibly vital aspect of any business, it is also very complicated and requires a lot of expertise. Business owners usually find it to be one of the most time-consuming parts of their operations.
Intuit QuickBooks, serving over 5.6 million businesses, knew there had to be a smarter way for their customers to get the support they need when they are doing their own accounting. In 2017, they introduced the QB Assistant, a conversational chat experience that business owners can turn to for any number of support questions, to their Self-Employed product.
The benefits of a virtual assistant are clear for user experience and bottom line alike. Escalation to a human agent is the most costly part of Intuit’s customer support solution. Not only are the expenses greater, but it often takes longer. QuickBooks customers would have to wait longer in order to get in contact with a support agent, whereas the automated, AI-powered QB Assistant can instantaneously answer any question.
“Our ultimate goal is speed-to-benefit,” said Scott Ganz, Principal Content Designer at Intuit QuickBooks. “When someone wants to know how to do something or needs information, it should be faster than even the fastest human help.”
To facilitate that speed and reduce escalation, conversations with the virtual assistant need to be perfectly clear with zero confusion. Ganz, along with Senior Content Designer Nicholas Pelczar, is constantly optimizing the QB Assistant’s content and updating conversation flows. In order to get the insights necessary to do that, the team relies on Dashbot’s one-of-a-kind conversational analytics.
High loads of unstructured data
Given the scope of QuickBooks’ functionality, users can ask about a broad number of topics in a wide variety of ways. No matter how thorough the content designers are, it is inevitable that unanticipated situations will come up. When that happens, users will either be presented with a mishandled Intent or a default “fallback” response. Both are frustrating to see when you are trying to get support.
It was also important to Intuit to be able to assess the value of the conversations they already built out. Knowing where the QB Assistant succeeds, and where it needs work, keeps the team pointed in the right direction.
The content team also has the additional challenge of pinpointing where users are experiencing frustration in order to figure out which parts of the conversation need optimizing or building out. The QB Assistant sees millions of messages from users, so they also need a way to quickly sort through all the transcripts to find what they were looking for. They needed a tool like Dashbot that would give them the visibility into every conversation.
Find actionable needles in a haystack with Dashbot
In order to determine how to build out the QB Assistant’s content, the team tracks which Intents are being triggered most often. They use Dashbot’s Top Intents In to determine what users are actually asking for in their conversations. Having those statistics allows the content team to create a more streamlined experience and reduce fallbacks.
During Intent content jam sessions where the team will brainstorm which conversation flows need building, the team will turn to the transcripts for ideas. Thanks to Dashbot, they have a full transcript of every session a user has with the virtual assistant. They are able to optimize the conversation flow based on the actual conversations QuickBooks customers are having. By using real data, they are also able to pinpoint any usages or phrases that were not anticipated.
“If I see a lot of people asking for an Intent, then I know it definitely has a more compelling business case,” Ganz said. “It shows me things I should build towards.”
Knowing what people are asking for is only one part of the equation. The second part is knowing how people are asking.
For example, QuickBooks has a bank widget that connects with financial institutions to pull data. People can reach this Intent by asking to connect their bank, but asking to add a credit card will also trigger it. The content team realized through Dashbot’s Transcript Search feature that when people asked the QB Assistant how to add a credit card to pay for QuickBooks, they were getting the “add a bank” Intent and getting frustrated. From there, they were able to change the wording and disambiguate those two Intents, clearing that point of frustration for their customers.
The content team also needs a way to quickly identify mishandled Intents among the millions of messages the QB Assistant receives. They are looking for big misses around disambiguation, so the fact that Dashbot automatically groups semantically similar utterances into “Phrase Clusters,” and shows what Intent they are getting tagged with is a huge benefit. The Intuit team can quickly spot areas where the virtual assistant missed the mark. Additionally, instead of typing each individual phrase into their Natural Language Processing (NLP) system to update the model, they can export the Phrase Clusters as a CSV file and import them.
“The fact that Phrase Clustering does a little bit of NLP magic on its own is nice,” Ganz said.
Reducing escalation with better conversations
Through Dashbot, the content team is also able to track their own performance based on how well the chatbot is handling customer support questions. They are able to see the number of messages that trigger the fallback Intent by accessing the Unhandled Report. For a year, Ganz kept an eye on the percentages of messages that were hitting the fallback Intent and the percentage of interactions that resulted in human escalation, the most expensive kind of failure for Intuit. Through this report, they were able to show that even though overall usage for the QB Assistant was up, human escalation had gone way down.
Dashbot also gave the content team the proof to back up the decisions they were championing. It democratized access to data and made it more straightforward for creatives to understand, according to Pelczar. He added that even as someone who does not have a numbers background, the onboarding process to Dashbot was smooth.
“From a creative perspective, we’re always begging for data and we’re not always given access to it,” Pelczar said. “Dashbot empowers content people to be able to tell stories with those numbers without having to pester a data scientist for them.”
Intuit plans to leverage the insights gained from working on the QuickBooks Self-Employed product and expand the QB Assistant to QuickBooks Online, a much more complex and full-scale product. Those additional complications will require even more content to cover the wider scope of functionality— and more opportunity to leverage Dashbot’s insights for success.
About Dashbot
Dashbot is an analytics platform for conversational interfaces that enables enterprises to increase satisfaction, engagement, and conversions through actionable insights and tools.
In addition to traditional analytics like engagement and retention, we provide chatbot specific metrics including NLP response effectiveness, sentiment analysis, conversational analytics, and the full chat session transcripts.
We also have tools to take action on the data, like our live person take over of chat sessions and push notifications for re-engagement.
We support DialogFlow, Alexa, Google Assistant, Facebook Messenger, Slack, Twitter, Kik, SMS, web chat, and any other conversational interface.