shares
There’s a moment most business owners recognize, usually a few months after their chatbot goes live, when it becomes clear that something isn’t working.
Customers are asking it questions it can’t answer. It’s giving outdated information with complete confidence. The handover to a human is clunky enough that people abandon the conversation entirely. Or, and this one is particularly discouraging, it simply goes unused, a widget that sits in the corner of the website while the same emails keep arriving in the same inbox.
A 2025 Gartner study found that 67% of businesses that deployed chatbots reported the technology did not meet expectations. Not catastrophic failure, just the quiet kind, where results were underwhelming enough that stakeholders started questioning the investment.
This is almost never a technology problem. The underlying AI models available today are genuinely capable: they understand nuanced questions, hold multi-turn conversations, and provide accurate, helpful answers when given the right context and training. The technology works. What fails is what happens before the technology is touched.
The businesses that build chatbots that actually work, that handle real volume, represent the brand well, and create measurable value, tend to have answered a specific set of questions before a single line of code was written. The ones that don’t, learn the hard way that a chatbot built on unclear thinking produces exactly that: unclear results.
These are the questions worth answering first.
Question 1: What specific problem are we actually solving?
This sounds obvious. It isn’t.
“We want a chatbot” is not an answer to this question. Neither is “we want to automate customer service” or “we want to be more efficient.” These are directions, not definitions. A chatbot built toward a direction rather than a specific problem ends up doing many things passably and nothing well.
The question that sharpens it is: what is happening right now that this chatbot needs to change?
Is it that your team spends two hours a day answering the same five questions, and those hours are genuinely valuable time lost? Is it that potential customers contact you outside business hours and leave before anyone responds? Is it that your sales team is fielding unqualified inquiries that a structured conversation could filter before human time is spent?
Each of these is a real, specific problem with a real, specific solution. A chatbot designed around “I need to answer the same five questions automatically” looks different from one designed around “I need to qualify leads at 11pm.” The scope is different, the training is different, the success metrics are different.
Fifty percent of businesses that haven’t yet deployed chatbots cite “lack of clear use case” as their primary barrier, according to industry research. The businesses that have deployed them and regretted it largely skipped the same step: defining the use case clearly enough that success could actually be measured.
Start there. Write down the specific problem in one sentence. If you can’t do it in one sentence, the problem isn’t defined yet.
Question 2: What are our customers actually asking, in their words, not ours?
This is the question that reveals the single most common cause of chatbot failure: the gap between how a business describes its own services and how customers actually talk about what they need.
Internally, a business might categorize customer inquiries as “billing inquiries,” “service availability questions,” and “onboarding support.” A customer doesn’t contact you and say “I have a billing inquiry.” They say “I was charged twice and I don’t know why” or “I thought this was included in my plan.” The language is completely different. The intent behind it requires different handling.
A chatbot trained on internal categories and business terminology will routinely fail to recognize what customers are actually asking. This isn’t a configuration problem, it’s a fundamental misalignment between the knowledge used to build the chatbot and the reality it’s supposed to serve.
The fix is deliberate but unglamorous: go through your actual customer communications before you build anything. Not a sample. A real volume of emails, support tickets, chat histories, enquiry forms. What words do customers actually use? What are they confused about most often? What questions appear repeatedly in almost identical form? What situations require nuance or judgment that a simple answer won’t resolve?
This exercise does two things. It tells you what to train the chatbot on, the real vocabulary, the real scenarios, the real edge cases. And it tells you what the chatbot should not try to handle, which is equally important.
Question 3: What should this chatbot never do?
Scope creep kills chatbots. The more a chatbot is asked to do, the worse it performs at everything, and the more likely it is to confidently give a wrong answer in a situation it wasn’t designed for.
The most effective chatbot deployments tend to be deliberately narrow in scope, at least initially. They do one or two things exceptionally well, and when a conversation moves outside those boundaries, they hand over gracefully rather than attempting to improvise.
This question is about defining that boundary explicitly: what kinds of conversations should always go to a human? What topics are too sensitive, too complex, too legally or ethically consequential for an AI to handle? What situations require judgment, empathy, or contextual knowledge that goes beyond what a well-trained chatbot can reliably provide?
Even the best-configured AI chatbots need to escalate 15 to 25% of conversations to human agents. This isn’t a failure, it’s correct behavior. The problem arises when businesses deploy chatbots expecting them to handle everything, without building the handover experience that makes escalation graceful rather than frustrating. Ninety percent of customers report having had to repeat information multiple times to resolve an issue. That experience almost always happens at the handover point, when the human agent has no context from the chatbot conversation they’re picking up.
The chatbot’s limitations should be designed, not discovered. Define them before you build.
Question 4: What does this chatbot need to know, and how current is that information?
A chatbot is only as good as what it’s trained on. This is the statement that sounds obvious until you think about what it actually requires.
It means the chatbot needs access to accurate, complete, up-to-date information about your services, your pricing, your processes, your policies, and your brand. It means that information needs to be organized in a way the AI can retrieve reliably, not buried in a PDF from 2022, not scattered across five different documents that partially contradict each other.
And it means that when anything changes, a service is updated, a price changes, a policy shifts, the chatbot’s knowledge base needs to change with it. A chatbot operating on outdated information doesn’t know it’s outdated. It answers with the same confidence either way.
This is one of the most underestimated operational realities of chatbot deployment. Sixty-one percent of businesses deploy chatbots without a proper data foundation in place, according to research into AI implementation. The chatbot launches with incomplete training and degrades further over time as the business evolves and the knowledge base doesn’t.
Before building, audit the information the chatbot will need. Is it documented? Is it accurate? Is there a process for keeping it current? Who owns that process? These aren’t technical questions, they’re operational ones, and they need to be answered before a technical solution is built on top of them.
Question 5: How will this chatbot sound, and will customers recognize our brand in it?
A chatbot that technically answers questions correctly but sounds nothing like your business is still a brand problem.
Brand voice isn’t decoration. It’s the texture of every interaction a customer has with you, the warmth, the specificity, the way you handle a difficult question, the tone that tells a customer whether they’re dealing with a company that cares about them or one that’s just trying to reduce ticket volume.
An off-the-shelf chatbot has a generic voice by default. It sounds like every other chatbot, because it was trained on the same kind of generic content. For a business that has invested in building a distinctive brand, a specific positioning, a particular way of communicating, a voice that customers associate with quality, deploying a generic chatbot is actively counterproductive.
Brand voice training for a chatbot means more than telling it to “be friendly.” It means defining specific vocabulary the chatbot uses and avoids. It means establishing how it handles uncertainty, how it responds to frustration, how it introduces itself and how it ends a conversation. It means testing its responses against real scenarios and revising until the output is genuinely representative of how your brand communicates.
This takes time. It’s also the difference between a chatbot that customers feel good about interacting with and one that subtly erodes the impression you’ve spent years building.
Question 6: How will we know if it's working?
The most common measurement mistake in chatbot deployment is measuring volume instead of value. Conversations handled. Messages sent. Sessions initiated. These numbers tell you the chatbot is active. They tell you nothing about whether it’s actually helping.
The metrics that matter are different: What percentage of conversations is the chatbot resolving without human intervention? How are customers rating their experience? At what point in a conversation are people abandoning it, and why? What types of questions is it failing to answer? Are the inquiries it’s generating better qualified than the ones that came in before?
Define these metrics before launch, not after. If you don’t know what success looks like before you build, you won’t know whether you’ve achieved it, and you won’t have a framework for improving systematically when performance falls short.
Question 7: Who owns this after launch?
This is the question that separates chatbot projects that sustain value from those that degrade quietly until someone decides to remove the widget.
A chatbot is not a finished product at launch. It’s the beginning of an operational responsibility. Conversations happen that weren’t anticipated. New questions arise as your business evolves. Information changes and the knowledge base needs updating. Performance dips and someone needs to identify why.
Forty-four percent of organizations have experienced negative consequences from AI implementation, and the primary driver is the “deploy and forget” mentality: the assumption that once a chatbot is live, it runs itself.
Before building, assign ownership. Who monitors performance? Who updates the knowledge base when information changes? Who reviews conversation logs to identify gaps? Who decides when the chatbot’s scope should expand — and manages that expansion carefully? The answers don’t need to be elaborate, but they do need to exist.
The Pattern Behind All Seven Questions
Every question on this list points to the same underlying truth: the work that determines whether a chatbot succeeds or fails is almost entirely strategic, not technical.
The technology has matured to the point where it’s no longer the limiting factor. What limits most chatbot deployments is the thinking that precedes them, the clarity of the problem definition, the quality of the training data, the discipline of the scope, the rigor of the brand voice work, the honesty of the success metrics.
Businesses that skip this work and go straight to building don’t save time. They spend months managing an underperforming tool, eroding customer experience in small ways, and eventually rebuilding from scratch with the questions they should have answered at the beginning.
The businesses that answer these questions first, even when the answers are uncomfortable, even when they reveal that the chatbot isn’t ready to be built yet, build something worth deploying.
Where to Go From Here
If you’ve read through these questions and found clear, confident answers to all of them, you’re ready to build. The strategic foundation is in place, and a well-executed chatbot deployment will have the conditions it needs to perform.
If some of the answers are unclear, or if this raised questions you hadn’t considered, that’s useful information. It means there’s strategic work to do before technical work begins, and doing it now is significantly less expensive than doing it after launch.
At Sustainable Growth, we work through exactly this process with businesses before any development starts. The strategic questions first. The right scope. The right training data. The right metrics. And the ongoing management that keeps a chatbot valuable long after launch day.
If you’d like to think through where your business stands before committing to a build, that conversation starts here.
Fill out the Chatbot Readiness Assessment form to see how ready your business is.
Afroditi Arampatzi
Marketeer
Hi, I’m Afroditi.
I’m the founder of Sustainable Growth, a Thessaloniki-based consultancy specializing in performance marketing and brand strategy.
I help businesses strengthen their brand positioning and apply AI-driven solutions that support smarter marketing, better decision-making, and sustainable growth.
