Deploying Generative AI Just Right

WRITTEN BY YOCHAI KONIG

Deploying Generative AI Just Right

Generative AI is finally having its moment. While interest in the field has been steadily growing over the past year, according to Google, search activity for the term is at an all-time high.

Over the last 20 years, I have worked on applying artificial intelligence (AI) to customer service applications. I’ve always been a passionate advocate for the application of AI — a commercially viable, if not critical, technological advancement — and it seems the rest of the world is catching on.

Customer service was one of AI’s earliest adopters. Its legacy starts in the contact center, when companies realized that the simple, repetitive, and incredibly mundane task of translating company knowledge and policies is a job better done by automation.

When the internet and SMS came on the scene, people suddenly had more channels available to reach out to the companies they do business with.

I remember the development of the first commercial bots that allowed people to interact with companies in text-based chat and over SMS in the early ‘90s.

I was a graduate student at Berkeley, and I worked on a voice bot, designed to be an automated consultant whose domain of knowledge was restaurants in the city: aptly named “BeRP.” The BeRP system served as a testbed for several future research projects.

During that time, it quickly became clear that customers wanted 24/7 access to customer service on these new channels, which presented a huge opportunity for the application of AI technologies.

And AI proved successful at doing so. Over time it became — and continues to be — more convenient for customers to engage with companies when the mood strikes them, on the channels of their choice, and in the languages they speak.

The launch of ChatGPT is accelerating the possibilities. Some customer service automation platforms powered by AI can quickly connect to a company’s knowledge base, instantly digest information, and translate it — with human-like understanding — into answers and chat flows that align with the brand voice.

……we can begin to imagine a future where customers actually prefer speaking to a highly trained AI bot.

This allows companies to start resolving the majority of customer inquiries without the need for human intervention. There’s a frenetic buzz around this potential, and the speed at which it can be deployed, which explains why businesses are jumping at the opportunity to attach their names to Generative AI or GPT.

On the other hand, there’s serious concern and caution, even among tech leaders about these tools. Large Language Models (LLMs) tend to be assertive even when the answer is wrong.

As technology like ChatGPT starts to automate more complex tasks the errors can be compounded.

Say, for example, you’re asking AI to guide a new investment. This is a simple two-step process.

First, the AI digests and summarizes articles on investment.

Second, the AI uses that summarized knowledge to provide you with guidance on the investment you should make in the chosen market based on the information.

But if the AI is basing its summary on incorrect information, then the investment itself would be based on this too, compounding two types of errors: the wrong interpretation of investment advice by LLMs, and investment based on the wrong, or hallucinated, investment advice. Due to this, many have called for pause on training on any new AI models until the risks can be better understood.

The goal for customer service leaders is to strike the right balance: embracing the innovation while continually monitoring and mitigating the issues and the risks associated with the deployment of Generative AI technology. While we should expect to see a lot more volatility over the next two to three years, here are five things to help you choose your “just right” AI strategy.

1. Embrace an “AI-First” vision for customer service.

In 2022, Fortune Business Insights reported that the global CX management market was worth $11.34 billion. The majority of this is spent on facilitating conversations with customers, including human agents, call routing software, and system operations.

Despite all this effort and investment, dissatisfaction with customer service is at its peak. Customers are expecting all their inquiries to be resolved instantly through conversational experiences. But many automation or chatbot companies don’t have the technology to actually solve the problems, sending customers to long queues and back on hold to reach live agent support.

When you combine these conditions with the power of Generative AI, we can begin to imagine a future where customers actually prefer speaking to a highly trained AI bot. Where customer service success is measured not by inquiry deflection or containment, but by automated resolution (AR).

A company’s ability to improve their resolution rates will be dramatically affected by AI, as we learn how to incorporate more generative features into the builder side of automation.

AI bots may not be the best answer for every inquiry. But they have the potential to be the preferred option for a vast majority of support interactions e.g., general company knowledge and FAQs.

This allows customer experience (CX) organizations to free up more agent time and effort to handle more complex customer inquiries: ones that require human empathy or have multiple issues that need to be resolved.

2. Account for industry and company expertise.

On its own, Generative AI cannot power a holistic, consistent, and helpful customer service strategy. Think of Generative AI, like ChatGPT, not as a solution itself, but as the underlying technology that enables applications.

Getting AI to go from rhetoric to results requires three critical components that work in harmony:

  • LLMs. To start, these models intake and interpret customer queries and deliver answers with the sophistication and tone of a human.
  • The Application. To be effective, AI must be grounded in customer service domain expertise and a depth of company-specific content to generate an accurate and substantive response.
  • The Context. One of the most important factors in guiding and aligning your AI is giving it context with both the industry and your own company information.

3. Train AI on truthful and accurate content.

One of the most important factors on the path to achieving the vision for AI in customer service is quality, and ultimately, trust. Since Generative AI came on the scene, we have seen several public gaffes made by chatbots, offering inaccurate and sometimes highly offensive responses. Automating more customer conversations may be a noble goal, but not at the expense of your company’s reputation.

For example, we expect customer requests to be followed by actions that align with company policy. Through iterative learning and experimentation, you will begin to identify when your AI bots are making the wrong choices. Or even worse, hallucination, which occurs when bots offer completely convincing but made-up answers.

Before rushing to deploy Generative AI in direct interactions with customers, take the time to establish an expected standard for responses and a threshold for accuracy that protects your brand. A well-aligned AI application enables bot managers to correct and align it based on company policy and guidelines.

4. Expand scope through continuous improvement.

Once we have established context and accuracy, we can further fulfill the vision for AI in customer service by expanding the scope of customer inquiries that can be resolved by automation. In order to effectively reach AR, the conversation between the customer and company needs to be:

  • Relevant. Effectively understand the customer’s inquiry and provide directly related information or assistance.
  • Accurate. Provide correct, up-to-date information with respect to the company’s knowledge and policies.
  • Safe. Interact with the customer respectfully and avoid engaging in topics that could cause danger or harm.

We start by replacing the lowest complexity requests, like “where is my package?” If these responses are deemed safe, only then do we move on to responses that require action. And eventually, bots can learn to address customer service needs that require reasoning and a level of judgment.

But how quickly a company increases the scope of Generative AI-powered bots should be tied to performance, learning, and measurement. Not unlike a human agent, you need to think about onboarding and training bots in order to build the learning needed to expand automation’s scope.

5. Pick partners that match your speed and quality standards.

Expect to be inundated with information and opportunities for incorporating AI into your customer service function. To stay cutting edge without becoming overwhelmed, choose a set of criteria for the type of companies and tools you would consider engaging with along two key dimensions.

First, think about the experience level of a potential partner and your appetite for working with companies that are brand new, through to those that offer add-on components to a legacy system.

Second, consider the openness of the AI model, from completely open ended, to proprietary models that may offer vertical expertise. Along each continuum will be tradeoffs between risk and speed.

There is no question that the vision for AI in customer service is real. We will soon see a shift in customer behavior and a preference for interacting with a smart bot rather than a human agent.

The next two to three years will be chaotic, with new players, experimentation, and “things being broken” before they get fixed. While AI’s uncertainty may prevent you from locking in your entire tech plan, you can and should set parameters and evaluation criteria that protect your brand along the way.

Yochai Konig

Yochai Konig is the Vice President of Machine Learning & AI at Ada. He has over 30 years of experience in ML research, conversational AI, and deep learning. Yochai is the inventor of over 100 patents and authored numerous academic papers.