Before You Buy a Customer-Service AI Agent, Test the Refund and Complaint Boundary

A customer-service AI agent can look capable in a demonstration because the questions are tidy and the consequences are small. The buying risk appears later, when a real customer asks for a refund outside policy, disputes a cancellation, reports a vulnerable circumstance or wants to make a formal complaint.

The important question is not simply whether the agent can answer. It is whether the system knows what it may decide, what it must never decide, what evidence it must retain and when a person must take over. Those boundaries determine customer harm, staff workload and the true operating cost.

This guide helps small businesses test that boundary before buying. It does not assume that more automation is always better, and it does not treat a polished chat interface as proof of lawful, reliable service.

Map decisions, not just conversations

Start with one complete customer journey: arrival, identity checks, knowledge retrieval, proposed action, approval, execution, confirmation, record-keeping and any later challenge. Mark every point where the agent could change a customer’s money, access, contract or rights.

A reply-only assistant may draft information for a person to approve. An action-taking agent may issue credits, cancel a plan, change an address or create a return. The second system is not merely a better chatbot. It has a larger authority surface and therefore needs stronger controls, evidence and recovery.

Build an authority matrix before requesting a demo

Customer request	Agent may do	Human decision required	Evidence to retain
Order-status question	Read approved fulfilment data and explain status	When data conflict or delivery is disputed	Data source, answer and timestamp
Standard refund within policy	Collect facts and prepare the eligible action	For high value, repeated claims or uncertain eligibility	Policy version, reason, amount and approver
Cancellation	Explain steps and capture the request	When contractual timing, fees or bundled services are disputed	Request, effective date and confirmation
Formal complaint	Recognise the complaint, acknowledge receipt and route it	Investigation, remedy and final response	Original wording, attachments, handler and outcome
Vulnerable customer signal	Slow the interaction and offer a safe handover	Any consequential judgement or tailored support decision	Minimal necessary notes and escalation

The matrix should be owned by the business, not inferred by the vendor. Ask whether permissions can be set by action, value, product, customer type and confidence level. A single “autonomous” switch is not a sufficient control model.

Refunds reveal whether the workflow is genuinely controlled

Refund handling combines policy interpretation, transaction data, customer communication and an irreversible financial action. Test at least four cases: clearly eligible, clearly ineligible, ambiguous and exceptional. Include partial refunds, promotional bundles, multiple currencies, chargebacks and an order whose fulfilment data disagree with the payment record.

The Competition and Markets Authority’s 2026 guidance says businesses remain responsible for what an AI agent does, including when a third party supplies it. The guidance calls for accurate information, monitoring, human oversight and prompt correction when problems appear. For customer-service queries, it specifically points businesses back to consumers’ statutory and contractual rights.

Ask the supplier to demonstrate the refused action as carefully as the successful action. A safe refusal should explain the next route and preserve the customer’s ability to reach a person; it should not trap the customer in a loop or invent a policy.

Treat complaints as a separate workflow

A complaint is not merely a low satisfaction score. It may allege financial loss, discrimination, misleading information, repeated failure or poor treatment. The system must preserve the customer’s original meaning rather than rewriting it into a softer summary.

• Can the agent recognise complaint language that does not use the word “complaint”?
• Can a customer ask for a human without completing another automated questionnaire?
• Are attachments, prior messages and transaction records carried into the handover?
• Does the business retain the exact agent output, action and policy version?
• Can an authorised person reverse the action and notify the customer?

If the vendor cannot show the escalation record, the buyer cannot assess the real work left for staff or investigate a later dispute.

Cost the human review queue, not only model usage

Usage pricing matters, but it is rarely the whole cost. Public API price pages may separate model input, cached input, output, tool calls, storage and other services. A packaged agent may combine some of these charges, add platform fees or apply fair-use limits. Request the supplier’s current written pricing and model your own workload.

Illustrative calculation—not a forecast: suppose a small shop receives 2,000 support contacts a month. The agent handles 1,300 without an action, sends 500 to a five-minute human review and escalates 200 to a fifteen-minute investigation. Review time is 41.7 hours; investigation time is 50 hours. At an illustrative loaded labour cost of £22 per hour, human oversight costs about £2,017 a month before software, integration, quality assurance, incident repair or management time.

The point is not that automation fails. It is that an apparently high containment rate can coexist with an expensive or risky review queue. Measure time per outcome, repeat contacts, reversals and complaint rates—not only the percentage of conversations that avoided a person.

Include the full operating stack

Cost layer	Questions for the supplier
Licence and usage	Base fee, seats, conversations, tokens, tool calls, storage, currencies and overage rules
Implementation	Helpdesk, commerce, payment, identity and knowledge-base integration
Human oversight	Review queues, escalation coverage, quality sampling and out-of-hours responsibility
Knowledge maintenance	Who updates policies, prices, product data and expired content?
Data protection and security	DPIA work, contracts, access control, retention, redaction and incident handling
Error recovery	Refund reversals, customer remediation, root-cause review and workflow correction
Change and training	Staff training, new procedures, documentation and temporary double-running
Exit	Export formats, logs, prompts, knowledge content, configuration and deletion evidence

Convert each layer into a monthly fixed cost, a per-contact variable cost or a periodic cost. Then calculate cost per resolved contact and cost per satisfactory outcome. A cheap automated response that creates a repeat contact is not a cheap resolution.

Check data protection and meaningful human involvement

Customer-support records can contain names, addresses, purchase histories, account details and sensitive disclosures. The Information Commissioner’s Office says organisations should assess AI processing in context, document data flows, define controller and processor roles, consider less risky alternatives and record where human involvement occurs. When review is used, it must be meaningful and capable of overturning a decision.

Ask the vendor what data enter the model, where they are stored, who can access them, whether data are used to improve shared services, how retention is configured and what deletion evidence is available. Determine whether a data protection impact assessment is required for the proposed use rather than treating the vendor’s general compliance statement as your assessment.

Run a scenario pack with pass and stop conditions

NIST’s voluntary AI Risk Management Framework Playbook organises suggested actions under Govern, Map, Measure and Manage. For a buyer, that translates into named owners, a mapped use case, measured performance and a plan for incidents and change.

• Ordinary: correct answer from an approved source.
• Conflicting: policy page and order data disagree.
• Adversarial: customer asks the agent to ignore rules or reveal another person’s information.
• Emotional: angry or distressed message with an urgent demand.
• Consequential: cancellation, refund, credit, complaint or account restriction.
• Unavailable: payment, fulfilment or identity service is down.
• Changed: policy is updated after the agent was configured.

Define a pass condition for accuracy, action permission, evidence, tone and handover. Define a stop condition too: for example, any unauthorised refund, invented policy, lost complaint record or failure to offer a human route pauses the pilot.

Who this purchase suits—and who should wait

A controlled agent may suit a business with repeatable enquiries, reliable source data, clearly written policies, adequate ticket volume and staff who can own the review queue. It is more likely to help when the first use case is narrow and reversible.

Wait if policies are disputed, source systems regularly disagree, nobody owns complaints, the team cannot monitor outcomes or the only business case assumes that human support disappears. Highly sensitive or consequential cases may justify assistive drafting with human approval rather than autonomous action.

Compare realistic alternatives

• Do nothing yet: fix policy and knowledge-base gaps first.
• Use existing helpdesk automation: route by topic, apply templates and collect required fields without generative decisions.
• Add an internal assistant: let staff retrieve approved information and draft replies while people retain authority.
• Automate one low-risk action: start with order status or appointment confirmation before refunds and complaints.
• Improve self-service: clearer policies and account controls may remove contacts without introducing an agent.

These options may deliver less visible novelty but lower integration, governance and recovery cost. Compare them against the same outcome measure.

Make the buying decision from evidence

Before signing, require a workflow diagram, authority matrix, pricing schedule, data-flow description, escalation record, export sample and pilot plan. Speak to the people who will investigate exceptions, not only the sponsor who wants faster replies.

For the companion economics exercise, read Before You Buy an AI Support Agent, Cost One Ticket From Arrival to Resolution. For failure evidence, use the action-log audit guide.

The safest buying question is concrete: “Show us exactly what happens when the agent is unsure, wrong or challenged after moving a customer’s money.” The answer reveals whether you are buying a controlled service workflow or merely an impressive interface.

Optional marketplace discovery

MOC Marketplace is part of the MaryChuks.com business ecosystem. Its live landing page displays digital-business and AI-workflow offers, but displayed cards, badges, revenue figures and protection claims are not independent proof of operations, ownership or transaction performance. No individual customer-service agent listing was verified for this guide.

If you want to compare the kinds of digital assets currently presented, visit the general MOC Marketplace discovery page and apply the authority, cost and evidence tests above before making any decision.

Sources and limitations

• Competition and Markets Authority: Complying with consumer law when using AI agents.
• Information Commissioner’s Office: AI accountability, governance and DPIAs.
• NIST AI Risk Management Framework Playbook.
• OpenAI API pricing, used only as an example of multi-part model and tool pricing—not as a product recommendation or universal cost benchmark.

Sources and marketplace routes were checked on 16 October 2026. Laws, regulator guidance, vendor pricing and product terms can change. This article offers general buying and operational guidance, not legal, data-protection or financial advice. The calculations are illustrative, not observed customer results. The featured image is an original conceptual AI illustration created for this article.