AI Support Reports: Check What ‘Resolved’ Means

Imagine a learner asking an AI support agent why a purchased course will not open. The agent offers troubleshooting steps. The learner leaves the chat. In this hypothetical example, the next business report records a resolution. Has the learner started the lesson, decided to try later, or given up? The conversation alone may not establish which.

For an independent creator or small business owner, this is a question about what the report actually measures. A conversation classification can be useful without proving that every customer completed their task. Before using a headline rate to judge service quality, read the rule behind it.

Read the definition before interpreting the result

Intercom’s Fin outcomes documentation, dated 23 September 2026, distinguishes two kinds of resolution after an answer: the customer indicates satisfaction, or leaves without requesting more assistance. These are labelled confirmed and assumed resolutions respectively. An actual answer is required; responding only to a greeting does not qualify. Nor does silence after a clarification question create a resolution. The documentation also says a resolution is deducted if the customer returns to the same conversation seeking further assistance.

These qualifications matter. It would be inaccurate to describe the rule as counting every silent conversation as successful. Equally, an assumed resolution remains an inference from the interaction. It does not independently establish what happened outside the chat. Keep that distinction visible when discussing the result with colleagues.

Keep the denominator beside the percentage

Intercom’s current reporting definitions calculate the Fin AI Agent resolution rate from resolved conversations divided by all Fin-involved conversations. That population has its own eligibility rules. It is not simply every contact received by the business. The documentation separately lists legacy definitions, another reason to identify the metric actually being used.

Consider this invented calculation, illustrating the definition rather than reporting a product test. Suppose 20 eligible, Fin-involved conversations have reached these outcomes:

• Five customers confirmed satisfaction after an answer.
• Twelve left after an answer without requesting further assistance.
• Three conversations were escalated to a person after the AI’s attempt to help.

Combining the first two groups gives 17 divided by 20, or 85%. That is a resolution classification for this fictional group. It does not establish that 17 learners opened their courses. Nor does the smaller confirmed group prove that only five customers were helped. The missing evidence should remain missing, rather than being converted into either success or failure.

When comparing reporting periods, check whether the eligible population changed. A rate drawn from straightforward access questions answers a different management question from one covering complex account disputes. Put the count, included request types and period beside the percentage so readers can judge the comparison.

A closed ticket can lead to another ticket

The distinction between a service problem and its record also appears in Zendesk’s ticket status guidance. Agents can mark work Solved, and a period without updates can move the ticket to Closed. A customer reply to a solved ticket can reopen or update it. A reply to a closed ticket leaves that original record closed and creates a follow-up ticket.

The practical implication is that a count of reopened records cannot, by itself, describe every return for help. Ask whether the report includes linked follow-ups and how it treats a fresh request about the same underlying difficulty. Do not automatically merge everything from one customer either: their second question may concern a different task.

For the fictional course business, repeated access problems belong together for investigation even if they occupy separate records. A later question about course content belongs elsewhere. Decide what counts as the same issue before reviewing results, then document uncertain cases rather than forcing them into a convenient category.

Define the customer’s finish line

The GOV.UK Service Manual’s completion guidance, first published on 31 March 2016 and last updated on 19 February 2021, offers a useful measurement example. For qualifying government digital transactions, it divides completed transactions by all started transactions, including partial or failed attempts. It requires defined start and end points. General advice and enquiries sit outside its transaction definition.

This is guidance for government services, not a ready-made rule for every private support conversation. The principle worth borrowing is to define the task before choosing the evidence. A request for the location of a lesson may be answered by accurate directions. A report that an authorised learner cannot open it calls for evidence that the access problem was addressed.

Choose proportionate evidence already available in the service: perhaps a successful access event or the learner’s explicit confirmation. Opening a page would support an access outcome; it would not prove that the person watched, understood or valued the lesson. Keep each claim within the evidence supporting it.

Review a small set without filling in the gaps

For a manageable review, select cases using a stated method and record:

• The request: what the customer was trying to accomplish.
• The classification: how the support system labelled the conversation.
• The outcome evidence: what shows completion, continuing difficulty or uncertainty.
• Subsequent contact: whether a linked conversation concerns the same task.

Include assumed resolutions in the review rather than inspecting only explicit complaints. For each case, distinguish evidence that no problem remains from an absence of further information. Label a small, selectively chosen review as such; it cannot establish a precise success rate for the entire customer population.

Set a follow-up observation period that fits the task and state it in the report. A live workshop access problem and a question about a future delivery do not necessarily need the same window. Results still awaiting that window should be identifiable, so yesterday’s conversations are not compared uncritically with older, fully observed cases.

Give the leadership report an honest reading

A useful report can retain the provider’s metric while adding a separate account of reviewed customer outcomes. Show how many classifications were confirmed or assumed, what the review covered, and where completion remains unknown. If those figures differ, investigate the records and definitions before judging either the software or the support team.

Once the outcome is clear, the MaryChuks guide to costing a support ticket from arrival to resolution helps connect service quality with the work and resources needed to deliver it.

The leadership decision is whether customers can accomplish the things they came to do, and whether the business can explain the evidence. A resolution rate contributes to that decision when its meaning, boundaries and remaining uncertainty travel with it.

Sources

• Intercom: Fin outcomes — dated 23 September 2026.
• Intercom: reporting metrics and attributes.
• Zendesk: solved and closed tickets.
• GOV.UK: measuring completion rate — updated 19 February 2021.

Provider definitions and guidance checked on 26 October 2026. The course business and numerical example are illustrative.