The Public-Service AI Test: Can People Complete the Ordinary Task?

A public-service AI demonstration may answer questions elegantly while leaving the underlying service difficult to use. The relevant question for a resident is often much simpler: can I complete the task I came here to do?

This is a policy proposal, not a description of current legal requirements or a report on a named public body. It argues for testing the complete experience before treating a conversational interface as evidence of better service.

Test a journey with an ending

Choose an ordinary task: finding the right department, reporting a missed collection or understanding what information an application requires. State what successful completion means. Reaching an attractive answer screen may not be the same as submitting a valid request.

Include moments when the resident has incomplete information, uses unfamiliar wording or needs an accessible format. Test whether the system explains what happens next, how long the person should expect to wait and where an unresolved case can go.

Keep another route visible

A proposed service standard should include a usable alternative for people who cannot or do not want to use the AI interface. Its existence should be apparent before the person has spent twenty minutes trying to persuade a chatbot to understand.

This is also a staffing question. If the alternative route reaches an overloaded team without the case history, the handover has merely moved the difficulty. A complete evaluation should examine what the receiving worker actually sees.

Publish measures people can interpret

Suggested measures include task completion, incorrect routing, unresolved cases, time to resolution and accessibility problems. Report denominators and the conditions of the test. A satisfaction score from people who finished can overlook those who abandoned the process.

NIST’s voluntary AI Risk Management Framework supports examining risks in the context of use. The service measures here are this article’s proposed application, not mandatory NIST requirements. Read the framework.

The MaryChuks perspective: public technology earns its place by helping people reach a dependable outcome. An impressive conversation should be the beginning of that assessment, not the final result.