When AI Sounds Profound, Ask What It Actually Claims

An independent designer asks an AI assistant why clients keep requesting changes after they have approved a brief. The answer says the studio needs to ‘realign creative expectations across the decision journey’. It sounds like an explanation. But does the assistant mean that the wrong people approve the work, that the brief never defines what approval covers, or that clients see the finished design too late? Each explanation suggests a different fix. The impressive phrase lets all three pass without choosing among them.

That is a useful moment to slow down. A sentence can feel insightful because it brings familiar ideas together in an unexpected way. It can also feel insightful because it lets the reader supply whatever meaning fits. Before taking advice, see whether the answer says enough to distinguish one interpretation from another.

What the research tested

In a paper from the November 2015 issue of Judgment and Decision Making, Gordon Pennycook and colleagues asked participants to rate the profundity of statements. Some were made by arranging buzzwords into grammatical sentences without a discernible message. The researchers also used comparison material, including ordinary statements and more conventional quotations. Some participants gave high profundity ratings to the constructed statements. The study offers evidence that an impression of depth can arise even when the language gives readers little concrete meaning to work with.

The study did not test modern AI assistants or establish that ornate language is generally empty. A poem, a metaphor and a business diagnosis do different jobs. A metaphor can be valuable because it gives people room to explore. The problem begins when you need a reason to act and the answer supplies a phrase broad enough to fit almost any outcome.

Try changing the example

Return to the designer. Suppose the same person signs off every stage and the brief clearly describes what approval covers. Would the assistant still offer exactly the same diagnosis? Now suppose two people take turns approving, and one has never seen the brief. Would its answer change? A useful diagnosis should tell you why those situations differ. If the phrase fits them equally well, it may be a starting point for questions rather than an explanation.

Ask for a plain sentence that names a cause and a consequence. For example: ‘People who can request changes see the work only after the person who approved the brief has signed off.’ That is a specific hypothesis. You could look at who attended the review, who sent the late requests and whether those people had received the brief. If the records show everyone was involved from the start, the hypothesis loses force. A sentence that can turn out to be wrong gives you something to investigate.

Sometimes the problem is an undefined term rather than a sweeping claim. Our article on checking what AI means by your project shorthand covers words that have a specific local meaning. Here, the risk runs the other way: the assistant may introduce a phrase that no one has defined, while everyone assumes someone else knows what it means.

Keep metaphor, advice and evidence separate

A vivid metaphor can help you notice a pattern. If someone says ‘the brief has become a revolving door’, you might ask which decisions keep returning and who brings them back. The metaphor has done useful work when it points you towards a question. It cannot tell you how often the problem occurs or whether a new approval step will fix it.

Advice also deserves its own check. ‘Invite the final approver to the first review’ is actionable, but whether it is sensible depends on the cause. If that person already attends, the recommendation may add nothing. If different departments disagree about what they can change, it may be more useful to write down which choices are still open. Ask what the recommendation assumes and what evidence would make you choose a different action.

A quick check before acting

• Underline the main claim and rewrite it without the impressive phrase. If you cannot do that without adding information, ask which interpretation the assistant meant.
• Ask for one example where the claim would apply and one where it would not. Look for a difference you could actually observe.
• Decide what would count as evidence. Check the brief, notes or records yourself, or treat the answer as a hypothesis until you can.

Apply that check in proportion to the stakes. You do not need to interrupt a brainstorming session every time someone uses a broad phrase. Write it down if it opens up possibilities. When you are about to change a team’s process, publish an explanation or spend time fixing a problem, ask for more precision. In the design studio, a short review of the last few change requests could do more than another page of strategy language.

An answer can be elegant and specific. The standard is whether it gives you something you can understand well enough to question: a distinction you can explain, a cause you can check or an action you can justify. If the wording is doing all the work, keep asking until the idea can stand on its own.

Source

• Pennycook and colleagues (2015), On the reception and detection of pseudo-profound bullshit, Judgment and Decision Making, 10(6), 549–563.