Chatbot UX Design Lessons Only 2 Years, 4 Bots, and 2,000 Customers Can Teach You

Posted by Matt McMillan on Jul 14, 2017 8:06:00 AM

Talla-Rocket-Ship.png

Being the lead designer for Talla has been an exercise in navigating uncharted territory. Talla was founded to bring real artificial intelligence to chatbots in conversational systems like Slack. We started in 2015, several months before the Slack app store even existed.We had to learn everything; there was no playbook out there for how to create a product like this.

The best way to learn was by doing, so we simply built and deployed several experimental bots to test our design and engineering ideas out on real users. Our first three bots included:

  • A recruiting assistant that scheduled interviews, built candidate dossiers, and managed workflow processes around recruiting
  • A natural language task manager
  • A slack training assistant that delivered information on making the most of the Slack platform each day

The lessons we learned from building, supporting, and refining these bots led us to our first, real, go-to-market product: An intelligent, conversational help desk assistant for IT, HR and other internal service teams. The goal of the real product is to automate all the grunt work of a help desk, so staff doesn't have to invoke the RTFM curse.

Over 2,000 companies have used those products so far, and we've learned a lot about what does and doesn't work in helping enterprise chat users interact with an intelligent assistant. (For the first year, our primary platform was Slack, so some of these lessons are Slack-specific.)

Here are the interaction challenges we've found, and how we've tried to solve for them.

Lesson #1 - Users Don't Read

As my colleague Jon Klein says, people don't read. Text is easy to overlook, and the longer the instructions you give to a chat user, the less likely they are to be understood and properly followed. To create a meeting, for example, you need the user to tell you the subject of the reminder, the time of the meeting, the place of the meeting, and whom else to invite to the meeting. If you ask for all four at once, it's likely the user will miss at least one.

To use the example from Jon's post:

User: Schedule a meeting for tomorrow
Talla: Who should I invite to the meeting?
User: Call the meeting “discuss 3rd quarter development goals”
Talla: I didn’t understand that.

User: Add a task due next Friday
Talla: What is the name of the task?
User: Oh actually, it should be due Saturday
Talla: I’ve added the task “Oh actually, it should be due Saturday”

User: Put a meeting on my calendar
Talla: Who should I invite to the meeting?
User: Show me the weather in Boston
Talla: I didn’t understand that.

As we watched these kinds of interactions, we made changes to the copy and formatting to try to make it more clear what information Talla needed to complete the task. But still, we continued to find poor user interactions which eventually led us to the realization: people don’t read what Talla says.

To get the data you need, employ concise copy, use whatever formatting tools are available to make your calls to action stand out, and respond to user inquiries with the absolute minimum of information possible. Unfortunately, if you get your questions minimized, both in how little they and how succinctly they ask it, that can leave way too much room for interpretation on the part of the user.

My favorite example is when, very early on in Talla's development, we needed to know if the user installing our bot was using Google or Office 365 -- because we had to use at least one of those for a particular  integration. We had Talla ask, “Do you use Google or Office 365?” Or entire conversation flow hinged on the user responding with one of those two options. Most of them said, “yes” -- and broke the installation. The lack of intonation in chat means it’s even more important to be precise and to communicate efficiently and effectively.  

For most designers, the way to overcome the tone-deaf nature of written text to compensate with visual design, which leads us to our next major insight.

Lesson # 2 - Web Design Rules Don't Work

It may be obvious that you can't simply copy over the rules of good web design to a chat client, but the differences between the two aren't as intuitive as you might expect.

First, chat clients do let you employ interactions besides simple text, but they are very limited and they vary between chat platforms. (About the only common non-text element of most chat systems are emoji.) That means you can't really standardize your UX between, say, Slack and Microsoft Teams, but you can use buttons and some images to limited effect. Ironically, it's text styling that is less flexible in chat, as changes to fonts, sizes, and colors are generally unavailable, and you can only do so much with bold, italics, and underlining.

The end result is that it's hard to make your bot's requests for information stand out from any other chat message, and harder still to give guidance about the breadth of information the bot needs from the user. More simply, it's hard to put good calls to action in a chat conversation.

A well designed web page offers multiple calls to action, but there's a clear hierarchy to them, so that there's an obvious difference between, for example, making a purchase, tracking an order, and contacting customer service. Within chat, that visual hierarchy is largely impossible, so you're limited to one to call to action at a time.

Once a user has chosen a call to action on a website, they tend to follow a pretty linear workflow to complete that action. For example, if a buyer doesn't complete an order form, the page prevents them from proceeding, even going so far as to highlight which form fields are missing or improperly filled out. Once the order form is filled, then the user can proceed linearly to the payment form, the confirmation page, and then the receipt page. If they abandon the process, or attempt to backtrack, you can breadcrumb their flow and help them resume the linear process later.

Chat, to our surprise, is less linear than a website. As we mentioned in Lesson #1, you can't ask for all the "form fields" to be returned in one call to action, as the details of the text will be ignored, and it's unlikely you'll get back all the right details in a single chat message. Instead, the chatbot has to tolerate being given any of the details of a request in any order, and be prepared to present follow-up calls to action to fill in any gaps necessary to complete a task.

This generally means stripping down the data required to complete an action to the bare minimum of parameters. Otherwise, you risk turning your chatbot into an interrogator, constantly badgering users with follow-up questions. That paradigm can be fairly limiting, which brings us to our most important lesson learned.

Lesson #3 - Don't Be Afraid to Go Outside Chat

The allure of platforms like Slack, HipChat, or Microsoft teams is that they are conversational hubs that users never have to fully leave. When designing chatbots for these platforms, it's tempting to try to confine all interactions with the chatbot to the chat platform. After all, the point of a chatbot is to enable a user to take an action from within chat, rather than forcing them out to another application.

Resist this temptation. Sometimes the best chatbot experiences happen outside of chat.

The virtue of a chatbot is that allows a user to initiate an action within chat, and that it provides a minimum distraction from the natural flow of a chat conversation. But, sometimes it's just easier to complete a task outside the confines of the user experience available within chat. It's perfectly acceptable, and sometimes optimal, to initiate a request in chat, leave chat to fulfil the request, and then return to chat when the request is completely configured.

For example, imagine how tedious it would be if you had to talk through every field of an Amazon order over the phone, with a customer-service representative entering information in every form field as you dictate it. It's faster and simpler to directly enter that data in a web form.

Instead, imagine of the chatbot, when prompted to order something from Amazon, popped up a minimalist order form that you could quickly complete. Once the form is submitted, it disappears and you're returned to the regular chat flow. That's much faster and much easier than trying to accomplish everything with a chat-based Q&A.

The same is true for many other form interactions. Date pickers are often easier than conversationally specifying beginning and end-dates and times. Dropdown lists are often better for choosing between several options. Checkboxes are easier for choosing between many possible combinations of options. These standard web controls evolved because they most efficiently accomplish a number of interaction tasks, and developing a chatbot is no reason to abandon these UX advances.

Eventually, we expect these chat platforms to mature to the point where these non-textual interactions are available. Until then, Talla is working on developing a robust "shadow chat" client that will intercede for the chatbot when a request is made that is best accomplished outside chat. Don't be afraid to launch a parallel experience when chat simply can't accomplish your UX goals in a useful or efficient manner. So long as you start and end a task in chat, and you spend as little time as possible outside the chat conversation, temporarily abandoning chat may be the best thing you can do to improve the performance of your chatbots.

Still Learning

These are just the first of many UX lessons we've learned in the first two years of Talla development. If you'd like to see these lessons in action, check out our latest generation intelligent assistants. And keep watching this blog; we and our chatbots still have so much to teach, and learn.

Start Your Free Trial of Talla's Service Assistant

 

Topics: Conversational UIs, ChatOps, Bots