How to choose an AI agent for customer support
Practical criteria for knowledge grounding, context, limits, specialist handoff, access control and testing before launch.
Kanyman editorial team · about the publisher
In brief
A good AI agent is not one that answers every question. It knows its role, uses approved knowledge, preserves context and recognises when it must stop or involve a person.
Start with a support scenario
Define which requests the agent should resolve independently: product explanations, onboarding help, pre-consultation data collection or common order questions. A precise scenario makes knowledge preparation and quality checks much easier.
One universal agent rarely sells, supports and qualifies equally well. Separate roles make instructions shorter, metrics clearer and mistakes easier to identify.
Verify knowledge grounding
The agent should answer from approved company materials rather than filling gaps with confident assumptions. Check which sources it uses, how updates are applied and whether knowledge can be limited for a specific role.
Ask a question that has no answer in the knowledge base. Correct behaviour is to clarify, state the limitation or hand the conversation to a specialist.
- sources can be updated independently
- connected knowledge is visible
- updates do not break active conversations
- unsupported claims are not presented as facts
Check context and handoff
Context includes the current request, previously collected details and the active project or channel—not just the last few messages. It must move with the conversation when a specialist takes over or a chat becomes a ticket.
Define situations where the agent must stop: refunds, disputes, missing data, requests for a person or company-specific restrictions. The specialist should receive the original question, history and collected details.
Protect data and actions
Only provide information required for the role. Passwords, tokens, private contracts and unrelated personal data do not belong in a support knowledge base.
Trust customer context only when it is signed by the product server. Browser-controlled values can improve the interface, but they must not decide access to orders, balances or another customer’s requests.
- knowledge stays inside its project
- secrets never enter prompts or history
- external data is verified server-side
- mutating actions have separate controls
Test before connecting a real channel
Use a dedicated test chat for common questions, ambiguous requests, missing knowledge, attempts to obtain restricted data and specialist handoff. Record the acceptable result for every scenario.
Repeat the same small test set after knowledge or instruction changes. Compare correctness, unsupported answers, handoff quality and time to resolution rather than the style of a single successful response.
- answers are supported by connected materials
- the agent stops when data is missing
- restricted information is not disclosed
- the specialist receives the full context