AI Explanations: Check What You Can Explain Yourself

An AI explanation can make an unfamiliar subject feel manageable in a few paragraphs. The terms fit together, the example seems clear, and you finish reading with a welcome sense of progress. Then someone asks why one step follows from another. You remember the conclusion but cannot reconstruct the connection. That gap deserves attention before the explanation becomes something you teach, publish or use to guide a decision.

There are two questions to ask: is the explanation accurate, and what have you understood from it? Checking the source addresses the first. Producing an account of your own gives you something concrete to examine for the second.

The illusion has a history

In a study published on 1 September 2002, Leonid Rozenblit and Frank Keil investigated the illusion of explanatory depth: believing that your understanding is more detailed than it is. Participants rated their understanding of familiar mechanisms, attempted explanations and then rated themselves again. Their estimates fell after trying to explain. The work concerned ordinary devices, including zippers and toilets, rather than AI. It established a useful distinction between an initial feeling of understanding and what someone encounters when trying to supply the details.

Matthew Fisher, Mariel Goddu and Frank Keil extended the question to information access in research published online on 30 March 2015. Across nine experiments, searching online raised participants’ assessments of their own explanatory knowledge. Some assessments concerned topics unrelated to the earlier searches. This was research on internet search, so it cannot establish how a contemporary chatbot affects every learner. It does make the boundary between accessible information and personal knowledge worth examining.

What the 2026 comparison tested

A 2026 Cognitive Science Society conference paper by Ivan Aslanov, Patricio Felmer and Ernesto Guerra compared three groups in a final sample of 102 university students. One consulted a customised ChatGPT interface; another read identical texts without being told their source; a control received no supporting material. Participants then predicted their explanatory ability, wrote explanations and evaluated them.

The chatbot group’s self-ratings fell more after writing than the text-reading group’s. Independent coders rated the reading group’s explanations higher than the chatbot group’s; the unsupported control scored lowest. Receiving assistance and accurately judging what you can explain were therefore separate outcomes.

The chatbot supplied fixed responses generated with ChatGPT-4.5, with no follow-up questions allowed. The authors acknowledge modest effects, a limited sample and extra instruction demands in the chatbot condition. This controlled comparison does not establish permanent skill loss, identify the causal mechanism or represent every conversational learning experience.

Explain the link between steps

For a hypothetical creator, imagine a workshop booking arrangement with twelve places. Under its stated rules, each paid booking occupies one place. An unpaid reservation holds one place until its deadline; expiry releases that place. Assume eight paid bookings and two unexpired reservations. The remaining availability is two places. This is an invented arrangement for illustration, not a description of any booking product.

Someone might repeat “there are two places left” after reading an explanation without understanding the role of reservations. Ask why the answer is not four. A useful account connects each reservation to a temporarily unavailable place. Then change the example: both reservations expire without payment, while the eight paid bookings remain. Under these rules, availability becomes four.

The changed case gives you another part of the explanation to inspect. Can you say what changed, which rule applied and why the paid bookings stayed relevant? If your account jumps from “reservations expired” straight to a number you cannot justify, you have identified a specific link to revisit. An AI-generated explanation of that link should still be checked against the stated rules.

Use a short comparison routine

For a concept you intend to explain to someone else, try the following editorial routine. It is a suggested working practice, not an intervention validated by the studies above.

• Choose a narrow question. Replace “I understand booking systems” with “I can explain how an unpaid reservation changes the available places under these rules.”
• After reading, put the explanation aside briefly and make your own account. Write, speak, sketch or use another format that lets you show the relevant connections.
• Mark the point where your account becomes a label instead of a reason. “The system updates” leaves unanswered what information changes and which rule causes it.
• Change one condition and predict what follows. Keep the other conditions explicit so that you can inspect the reason for your prediction.
• Compare your account with a reliable source or qualified reviewer. Correct the missing link and try explaining that part again.

A fluent account of your own can still be mistaken. The comparison step matters because independently producing an explanation and establishing its accuracy are different checks. Asking the same chatbot to declare your answer excellent supplies feedback, but it does not by itself provide independent confirmation.

Let the task set the depth

The necessary depth depends on what you plan to do. Introducing a general concept in a newsletter requires a different level of understanding from teaching a detailed process or approving a change to a working system. Decide which connections you must be able to explain, and which require further expertise.

Keep the exercise proportionate and accessible. A pause, awkward wording or difficulty speaking on demand does not settle whether someone understands. Allow time, notes where appropriate, diagrams or another means of expression. The useful object of review is the account and its connections, rather than the confidence of the performance.

For teaching and revision tasks, MaryChuks.com’s exercise on explaining an AI-assisted edit offers a related approach: preserve the original, examine a suggestion and explain the decision about it. That makes the learner’s choices available for discussion alongside the finished wording.

Keep access and understanding visible

For work you will revisit, keep a short note of the question, the source and the connection you can now explain. Add the unresolved point that still needs checking. You can continue using reference material while being precise about where your own explanation stops.

The useful outcome is a clearer next step. You may be ready to communicate the idea, need a better source, or need someone with relevant expertise to review it. An AI explanation has served the work when you can examine what it helped you understand and identify what remains unresolved.

Sources

Primary research: the full papers below informed this article. The workshop arrangement and working routine are original illustrations.

• Rozenblit and Keil (2002), The misunderstood limits of folk science: an illusion of explanatory depth. Cognitive Science, published 1 September 2002; author-hosted paper.
• Fisher, Goddu and Keil (2015), Searching for Explanations: How the Internet Inflates Estimates of Internal Knowledge. Journal of Experimental Psychology: General; online publication 30 March 2015.
• Aslanov, Felmer and Guerra (2026), Overconfidence without Understanding? Evidence for an Illusion of Understanding in AI-Assisted Explanations. Proceedings of the 48th Annual Conference of the Cognitive Science Society.