It was 11 PM, and I was staring at the third threatening email that week from my leasing office about renewing my renter’s insurance. The deadline was the next morning. I’d done this dance before: call the insurance company, sit on hold for twenty minutes, and get told the renewal documents won’t exist until the exact renewal date. So this time I didn’t call. I opened a chat window on my provider’s site, mostly out of spite, expecting the usual runaround. Thirty seconds later the bot had understood what I needed, found my policy, confirmed the renewal had already processed, pulled the right documents, emailed them straight to my leasing office, and told me it was done. I closed the laptop genuinely surprised.
That’s the bar, and almost nothing clears it.
In 2023, Gartner found that only 8% of customers used a chatbot during their most recent customer service interaction. Of that sliver, only 25% said they would use one again1. The same year, Forrester reported that 71% of companies had invested in chatbot capabilities, but just 16% of US online adults use chatbots often to get help2. That’s a 55-percentage-point gap between investment and usage. Something is broken.
I’ve spent years in jobs built around talking to people, and now I design software for a living. I’ve also designed and shipped my own AI-powered chatbot, Sortly Sage, which doubled resolution rate and cut agent tickets 29% by its second iteration3. So I’ve watched this gap from both sides: as a customer who’s been failed by bad bots, and as a designer who’s had to fix the reasons they fail.
The patterns are so consistent that I can predict how a chatbot deployment will go wrong just by asking one question at the start: “Who built this for whom?”
The five most common UX mistakes in chatbot deployments
Forrester’s consumer research identified the top four things that annoy people about chatbots: they don’t understand what you need, they take too long to solve problems, they’re unable to help, and they make you start over2. That research was published in 2023, but the same patterns show up in NNGroup’s usability testing from 2018, and they’re still here in 2026. Here’s what each one looks like in practice, plus a fifth that doesn’t make the survey list because customers don’t always notice it, even when it’s the root cause.
Mistake 1: Designing to cut costs instead of help customers
This is the one customers can’t always name, but it poisons everything downstream. When the brief is “reduce support ticket volume,” the resulting chatbot becomes a gatekeeper not a helper. I call it the “AI facade”: a company looks like it offers 24/7 support while actually adding a layer of friction between you and an answer.
Here’s the version I experienced with a major clothing retailer. Every interaction followed the same arc: the bot only handled canned FAQ answers, misread anything about sizing or returns, and offered no clear way to reach a human. When a real person finally replied by email days later, they had none of the chat history, so I explained everything from scratch. The bot wasn’t designed to help me. It was designed to keep me from calling support. I felt it. Every customer feels it.
Forrester’s Max Ball put it bluntly: “Just adding a chatbot on your website or mobile app will not reduce customer service calls.”4 The people who want to chat are often not the same people who make phone calls. Building a bot for the wrong reason doesn’t just fail to solve the problem, it creates a new one.
Mistake 2: Failing the handoff between bot and human
This is the single most damaging pattern I’ve seen. Retail support desks, healthcare portals, e-commerce sites: when a chatbot can’t resolve an issue and escalates to a human agent, the context rarely comes with the customer.
Poor chatbot implementations create barriers rather than solutions
When I was designing Sortly Sage, I reviewed weekly conversation transcripts and found users were being pushed out of the product entirely to file a Zendesk ticket. They had to re-enter their email, username, and company ID, information Sortly already had. That friction wasn’t a bug. It was the default. Getting engineering to add an in-product ticket flow required making the cost visible with data, not just arguing the logic.
A patient in one healthcare chatbot study put it best: “Why aren’t chat logs automatically transferred from the chatbot conversation to the human representative? I’m tired of explaining my symptoms three times.” That line came from research I encountered while recruiting a conversational designer for a large healthcare company years ago. The fix sounds simple: pass the full conversation transcript, customer identity, a session summary, and what they were trying to accomplish. But it requires the chatbot team, the CRM team, and the support platform to agree on one handoff protocol. That’s where most orgs stall.
Mistake 3: Building for the happy path only
NNGroup’s foundational chatbot usability study found a pattern I’ve never stopped seeing: bots work beautifully as long as users stay on the prescribed linear flow. The moment someone deviates with a typo, an unexpected question, or a request that falls between menu options, everything breaks5.
Domino’s bot asked the user whether their location was an apartment or a house. The user typed “townhome.” The bot replied: “I’m sorry. I seem to be having trouble understanding.” No fallback. No clarification. Just a wall. The same study documented a Capital One user who had to clarify which credit card they meant after every single query, the bot carried no context between interactions, even within the same session.
These examples are from 2018, the era of rule-based chatbots with rigid decision trees. The underlying problem hasn’t gone away with LLMs. It’s just changed shape. Instead of “I don’t understand,” users now get plausible-sounding hallucinations. Different failure mode, same root cause: the bot was designed for the answers someone anticipated, not the questions users actually bring. This is why rule-based chatbots fail at complex customer service requests: they can only answer what was scripted. LLM-powered bots can answer more, but when they hallucinate, they do so with equal confidence.
Mistake 4: A personality that doesn’t persist
When I was hiring that conversational designer for the healthcare company, I learned that after three years in production, their chatbot audit revealed a voice problem: warm and conversational one moment, stiff and legalistic the next, sometimes inside the same response. Trust goes down when the bot sounds like three different people.
This isn’t a luxury detail. NNGroup found that Progressive’s bot “Flo” was playful at first, matching the brand’s TV personality, but became robotic the moment it started gathering data. Users notice the shift instantly. When a bot’s tone contradicts itself, it reads as unreliable, even if the underlying information is correct.
Mistake 5: No real escape hatch
NNGroup called this the “escape hatch”5, and it’s one of the few patterns that consistently gets worse over time, not better. The part most teams skip is chatbot fallback design: what happens when the bot can’t help. Some chatbots fight the escalation, asking three clarifying questions when a user types “agent” or “human,” as if exhausting them into submission. Others simply have no escalation path at all.
The UPS bot in NNGroup’s study did something worth copying: it warned users when it was about to repeat an answer and offered to connect them to a real person. Users appreciated the honesty. Owning the failure and offering an alternative was perceived more favorably than a wrong answer dressed up as confidence.
Forrester’s David Truog captured the stakes: “If you push callers to digital and your analytics tell you that they call you back within minutes, you’ll know that you have just executed an epic customer-experience fail.”2
What good chatbot UX actually looks like
The insurance bot that saved me at 11 PM didn’t succeed because of better AI. It succeeded because someone designed it around a real task, wired it into actual systems, and defined “done” as a completed outcome, not a delivered answer.
Here’s what that pattern looks like when you apply it systematically, drawn from NNGroup’s guidelines, Forrester’s research, and what I learned shipping Sortly Sage:
Effective AI chatbots solve complete customer problems, not just answer basic questions
-
Pick narrow, completable jobs. Don’t build “a chatbot.” Build a bot that can fully resolve three to five specific tasks: document retrieval, appointment scheduling, order-status checks, basic troubleshooting with a clear decision tree. Scope is safety.
-
Finish tasks, don’t describe them. The insurance bot didn’t tell me how to get my documents, it retrieved them and emailed them. Sortly Sage’s V1 was a full-page destination that gave accurate instructions, but users opened second browser windows to follow them. Accurate answers don’t matter if the container fights the user. In V2, we made Sage a collapsible sidekick that stayed open while users moved through the product.
-
Handoffs must carry full context. When escalation happens, the agent should receive the complete transcript, customer identity, session summary, and what the customer was trying to do. Sortly Sage’s ticket submission flow pre-populates account data so users don’t re-enter their own information. For copilot-style AI tools in support desks, this context transfer is the difference between an agent who picks up mid-stream and one who starts cold.
-
Keep one voice across bot and human interactions. Write voice guidelines once and enforce them everywhere. The bot, the human agent, the email follow-up, all sharing the same personality framework. Inconsistent tone is a trust killer.
-
Offer the escape hatch before the user has to demand it. Be honest about what the bot can and can’t do. When it hits its limit, say so clearly and offer a path to a human, a phone number, or a different channel. UPS-style: “I’m about to repeat myself. Want me to get a real person?”
When we rebuilt Sage as a collapsible drawer instead of a full-page destination, the numbers shifted fast: +44% Sage-handled solves, −29% agent-handled tickets, −15% Zendesk tickets, and −48% trial tickets in August, the steepest single-month drop in over a year. The AI’s accuracy hadn’t changed. The container had.
Measuring what actually matters
Most teams track conversation counts and response times. Those tell you almost nothing about whether anyone was helped. Forrester’s Max Ball flagged the core mistake: most companies commit to the wrong success metric for their business case4. If the pitch to leadership was “this will reduce support ticket volume,” the team will prioritize deflection over resolution. The metrics worth watching are the ones that measure customer outcomes:
Task completion rate is the one that matters most. What percentage of users finish what they came to do without human escalation? Aim for 80%+ on the jobs you designed the bot to handle.
Customer satisfaction on the bot itself, not your overall CSAT. Ask people if the chatbot helped them. Shoot for 4.0+ out of 5.
Escalation quality measures how the handoff actually goes. What percentage of escalations include the full conversation transcript? You want 90%+. If agents are re-interviewing customers from scratch, you’ve failed.
Return usage is the quiet trust signal. Do users come back to the chatbot for the same kind of task? 60%+ return rate says they trust it.
Business impact is the bottom line, but measured honestly. Does the bot lower support costs while keeping customers at least as happy? If satisfaction drops, you didn’t save money. You moved the cost onto your customers.
The actual future: humans and AI, together
The future of customer service isn’t humans versus AI. It’s designing systems where each handles what it’s good at. AI is great at routine tasks, 24/7 availability, consistency, and pulling up structured data. Humans are better at messy problems, emotional situations, and building relationships. The companies that win won’t have the smartest language model. They’ll be the ones that invest in real conversational design, define success around customer outcomes instead of ticket counts, spend real effort on the handoff, and iterate from actual usage data instead of internal assumptions.
The bot that solved my problem at 11 PM didn’t wow me with wit. It understood what I needed, solved it completely, confirmed the outcome, and let me go to sleep. That’s the bar, and the bar isn’t high. It’s just that almost nobody clears it.
Citations & sources
- Gartner, “Gartner Survey Reveals Only 8% of Customers Used a Chatbot During Their Most Recent Customer Service Interaction,” June 2023.
- Forrester, “People Avoid Chatbots, Here’s How Your Company Can Make Its Bot Better” (David Truog), November 2023. Also: Forrester’s Priorities Survey 2023 and March 2023 Consumer Pulse Survey.
- Emily Backes, “Sortly Sage: How I Doubled Chatbot Resolution Rate and Cut Agent Tickets 29%”, 2026.
- Forrester, “Build The Right Chatbot Business Case” (Max Ball), December 2024.
- NNGroup, “The User Experience of Chatbots” (Raluca Budiu), November 2018.
Keep reading
- Introduction to Conversational Design, Interaction Design Foundation
- AI Chat in Customer Experience, Nielsen Norman Group
- How Should We Evaluate Large Language Models?, Harvard Business Review
- Beyond Chatbots: The Future of Customer Service is Conversational AI, Gartner
