When AI Declines to Decide, Count What It Passes On

Imagine a small print studio using AI to sort incoming emails. Routine order enquiries go to the sales inbox; problems with finished orders go to support. When the model is unsure, it leaves the message for a person to route. After a trial, the studio sees that only a small fraction of automatically routed emails went to the wrong place. That sounds encouraging until someone asks what happened to the messages the model left behind.

A system that can decline to decide gives you a choice about which cases to automate. It also gives someone else the cases you set aside. The right question is whether the two routes together serve customers better. A good score for the automated route can coexist with a long wait for everybody else.

Read the two numbers together

In a classification task, a model chooses from a set of labels. A selective classifier can withhold that choice on some inputs. C. K. Chow’s 1970 paper examined the trade-off between making recognition errors and rejecting uncertain cases. In 2017, Yonatan Geifman and Ran El-Yaniv studied how to apply a selection rule to a trained neural network. Their work describes two quantities worth keeping together: coverage, the share of cases the system accepts, and selective risk, the average loss among those accepted cases. If every wrong label counts equally, that second number is the error rate among the cases the model handles.

Take an invented batch of 100 studio emails with independently checked labels. With a permissive rule, the model routes 90 and sends nine of them to the wrong inbox. Coverage is 90%, and the error rate among the routed cases is 10%. With a stricter rule, it routes 60 and gets three wrong. Coverage drops to 60%, and the error rate among routed cases drops to 5%. That second setting leaves 40 messages for review, compared with ten at the first setting.

Neither row tells you how many of those reviews will be correct or how soon they will happen. If staff can reliably route 15 messages in a morning and the stricter rule sends 40 at once, some customers will wait. The lower error rate describes the model’s retained work. It leaves open whether the service as a whole has improved.

See which cases are being handed back

Suppose the print studio’s web form sends neat, labelled enquiries. Customers who reply to an old email thread may include several requests in one message. If those replies are usually deferred, two customers asking the same thing can experience different service depending on how they got in touch. Check coverage and errors separately for meaningful groups, such as form submissions, replies and messages with attachments. Include the counts alongside rates so that a result based on a handful of cases does not look more settled than it is.

Review some accepted cases as well. The classifier may confidently treat ‘another copy of my order’ as a new sale when the customer actually wants a replacement. A higher threshold will not necessarily catch an error the model already scores highly. Our guide to temperature scaling and confidence explains why improving how scores reflect past outcomes is a separate task from correcting the label itself. Test the score against real routing mistakes before treating it as a reliable filter.

Test the handover before you switch it on

Start with a trial in which the system proposes routes but a person still makes the actual decision. Use a set of past messages to choose a few candidate thresholds, then assess the chosen rule on different messages. Include busy periods and examples with ambiguous labels. If reviewers disagree on whether an email belongs with sales or support, write down why. The categories may need a better definition before a new threshold can help.

• Record how many messages the model would route, how many of those routes would be wrong, and how many messages would go to review.
• Time a sample of real reviews. Note what context reviewers need, who covers the queue when the usual owner is away, and what happens if a message sits untouched.
• Separate mistakes by consequence. Sending a routine catalogue request to support may cost a little time. Leaving an urgent production correction in the wrong inbox can cost a print run.
• After launch, sample both routes again when the mix of work changes, such as during a product launch or after redesigning the enquiry form.

Give the reviewer enough information to act without starting from scratch: the original message, the order reference if it is available, and the reason the case needs attention if you can state it accurately. If the model is withholding a label because the order reference is missing, asking the customer for it may be the quickest next step. If the message mentions a deadline and the team cannot tell which order it concerns, someone needs to see it promptly.

The most useful threshold is the one your service can sustain. Decide how many wrong routes you can tolerate, how much review you can handle and what customers should experience while they wait. Then check all three in the same trial. Giving the model permission to defer is a sound design choice when the next step is as carefully designed as the prediction.

Sources

• C. K. Chow, On Optimum Recognition Error and Reject Tradeoff (1970)
• Yonatan Geifman and Ran El-Yaniv, Selective Classification for Deep Neural Networks (2017)