"How do I know if my chatbot is working?"
It sounds like a simple question, but most businesses track the wrong metrics. They celebrate "10,000 conversations this month!" without knowing if any of those conversations actually helped anyone.
This guide cuts through the noise. We'll cover the metrics that actually matter, the vanity metrics to ignore, and how to build a dashboard that tells you whether your chatbot is earning its keep.
Why Metrics Matter
Without proper measurement, you're flying blind. You might have a chatbot that:
- Handles thousands of conversations but frustrates customers
- Gets great satisfaction scores but doesn't impact business results
- Looks busy but could be replaced by a simple FAQ page
Good metrics help you:
- Justify investment: Prove ROI to stakeholders
- Identify problems: Catch issues before customers complain
- Guide improvement: Know where to focus optimization efforts
- Set goals: Establish meaningful targets and benchmarks
Vanity Metrics to Ignore
These metrics look impressive in reports but don't tell you anything useful:
Total Messages Sent
Why it's meaningless: A chatbot that can't understand questions sends lots of messages going in circles. More messages often means more confusion, not more value.
Number of Conversations Started
Why it's meaningless: This just measures how many people clicked on your chat widget. It says nothing about whether those conversations helped anyone.
"Engagement Rate" (Undefined)
Why it's meaningless: Unless you define exactly what engagement means and why it matters, it's just a feel-good number. Engagement without outcomes is noise.
Messages Per Conversation
Why it's misleading: More messages could mean great engagement OR frustrating back-and-forth. Without context, this metric can be interpreted either way.
Core Metrics That Actually Matter
1. Resolution Rate
What it measures: Percentage of conversations where the chatbot successfully helps the customer
How to calculate: (Resolved conversations / Total conversations) × 100
Why it matters: This is the single most important metric. If your chatbot isn't resolving issues, it's not doing its job.
Good target: 70%+ for support, 30%+ for lead gen
2. Containment Rate
What it measures: Percentage of conversations handled without human escalation
How to calculate: (Conversations without escalation / Total conversations) × 100
Why it matters: High containment reduces support costs—but only if customers are actually helped, not just trapped.
Good target: 65-80% (with high satisfaction)
3. First Contact Resolution (FCR)
What it measures: Issues resolved in a single conversation without follow-up
Why it matters: Customers who need multiple contacts for one issue become frustrated. FCR indicates true resolution quality.
Good target: 75%+
4. Abandonment Rate
What it measures: Percentage of users who start but don't complete conversations
Why it matters: High abandonment signals friction—the chatbot isn't providing value quickly enough.
Good target: Below 40%
Track What Matters with BuiltABot
BuiltABot's analytics dashboard focuses on the metrics that actually indicate success.
Business Impact Metrics
These metrics connect chatbot performance to business outcomes:
1. Cost Per Resolution
How to calculate: Total chatbot cost / Number of resolved conversations
Compare to: Cost per human support interaction ($8-15 typically)
Why it matters: Shows direct cost savings from automation
2. Support Ticket Deflection
What it measures: Reduction in human support tickets after chatbot implementation
How to calculate: (Pre-chatbot tickets − Current tickets) / Pre-chatbot tickets × 100
Good target: 30-50% reduction
3. Conversion Influence
What it measures: How chatbot interactions affect sales/signups
How to track: Compare conversion rates: visitors who used chatbot vs those who didn't
Why it matters: Proves the chatbot drives revenue, not just support savings
4. Time to Resolution
What it measures: How quickly issues get resolved
Compare to: Human support resolution time
Typical improvement: 50-80% faster than human-only
Customer Experience Metrics
1. Customer Satisfaction (CSAT)
How to measure: Post-conversation survey: "How satisfied were you?" (1-5 scale)
Good target: 4.0+ average (80%+ satisfaction)
Pro tip: Simple thumbs up/down gets higher response rates than detailed surveys
2. Effort Score (CES)
What it measures: How easy it was to get help
Survey question: "How easy was it to resolve your issue?" (1-7 scale)
Why it matters: Low effort correlates with loyalty better than satisfaction
3. Net Promoter Score (NPS)
What it measures: Would customers recommend your service?
When to use: Broader customer experience metric, not chatbot-specific
Track change: Has NPS improved since adding chatbot?
4. Repeat Contact Rate
What it measures: Customers contacting again within 24-48 hours about same issue
Why it matters: High repeat rate = issues aren't actually being resolved
Good target: Below 15%
Industry Benchmarks (2026)
| Metric | Poor | Average | Good | Excellent |
|---|---|---|---|---|
| Resolution Rate | <40% | 40-60% | 60-75% | >75% |
| Containment Rate | <50% | 50-65% | 65-80% | >80% |
| CSAT Score | <3.5 | 3.5-4.0 | 4.0-4.5 | >4.5 |
| Abandonment Rate | >50% | 35-50% | 20-35% | <20% |
| Avg. Response Time | >5 sec | 3-5 sec | 1-3 sec | <1 sec |
Building Your Metrics Dashboard
Essential Dashboard Components
- Real-time metrics: Current conversation volume, active conversations
- Daily snapshot: Resolution rate, containment, CSAT
- Trend charts: Week-over-week and month-over-month changes
- Business impact: Cost savings, ticket deflection
- Alert thresholds: Flags when metrics drop below acceptable levels
Review Cadence
- Daily (2 min): Glance at volume and resolution rate
- Weekly (15 min): Review trends, satisfaction, investigate anomalies
- Monthly (1 hr): Business impact analysis, improvement planning
- Quarterly (2 hr): Strategic review, ROI reporting, goal setting
The Bottom Line
Measuring chatbot success comes down to three questions:
- Is it helping customers? (Resolution rate, CSAT)
- Is it helping the business? (Cost savings, conversions)
- Is it improving over time? (Trend analysis)
Track the metrics that answer these questions. Ignore everything else.
BuiltABot includes built-in analytics focused on the metrics that matter. Our dashboard shows you resolution rates, customer satisfaction, and business impact—not vanity metrics that look good but mean nothing. Start your free trial and see real chatbot performance data.