Run an AI Project Premortem That Ends With a Decision

An AI demonstration can look convincing while important questions remain unanswered. The sample output is polished, the team is enthusiastic and a launch date is approaching. Yet nobody has checked what happens when the source document changes, the reviewer is unavailable or the ordinary customer asks an unexpected question.

A project premortem creates a structured opportunity to investigate those gaps before committing further. Its value depends on what happens after the discussion: which assumptions get tested, who can make the necessary changes and what evidence will determine whether the project proceeds.

For an AI project, finish the premortem with a documented decision and the conditions attached to it. A long list of imagined problems is only the beginning.

Begin with a defined project and an imagined failure

Gary Klein’s premortem method asks a team to imagine that its proposed plan has failed and generate explanations. His account dates the method to 2007. This article adapts that idea into a practical review for small teams adopting AI.

First define what you are reviewing. State the user, the task, the proposed level of AI involvement and the intended benefit. An assistant that drafts newsletter copy is a different project from a system authorised to send it automatically.

Choose a plausible future point and a concrete disappointing outcome. For example: six weeks after introducing an AI-assisted newsletter process, subscribers are receiving inaccurate product information and staff spend longer correcting it than they previously spent writing. Make clear that this is an imagined scenario, rather than a prediction or an accusation.

Our earlier inversion framework explains the broader value of examining a plan backwards. The additional task here is to connect each explanation to a checkable claim about the proposed AI workflow.

Collect individual concerns before discussing solutions

Ask each participant to write possible explanations independently before the group compares them. Include people who prepare the inputs, review the outputs, maintain the software and respond to customers. A small business may have one person covering several roles; examine each perspective separately.

Invite concrete observations as well as plausible concerns. Someone may have seen an outdated file used in testing. Another person may suspect that the proposed review time is unrealistic. Label these differently: one is an observed issue, the other an assumption to investigate.

You can ask AI to suggest additional possibilities after collecting human input, but treat its suggestions as hypotheses. It cannot establish that a team has a particular weakness simply by describing one fluently. Keep the exercise focused on the project and evidence available.

The discussion also needs a credible response to dissent. Our article on making bad news safe to share examines that leadership responsibility in more detail.

Turn a broad concern into a mechanism you can inspect

Statements such as customers will not trust it or the AI will make mistakes are too broad to direct useful work. Ask what specifically happens, under which conditions, and how it reaches the user.

In the newsletter example, a concrete explanation might be that an old product sheet remains available after a price change. The assistant uses it, while the reviewer sees only the finished paragraph. The check therefore fails to compare the claim with the current source.

That explanation suggests several questions: which document is authoritative, how are superseded files handled, what source information reaches the reviewer and what happens when the source is unavailable? Each question can be investigated without assuming that a particular model or employee is at fault.

Give each material concern a test and a decision owner

For each concern important enough to affect the launch, record four things:

• Evidence needed: the observation or result that would support or weaken the concern.
• Next test: a specific task using representative inputs and relevant difficult cases.
• Owner and deadline: who can complete the test or change the workflow, and when the result is required.
• Decision consequence: what the team will do if the result is unacceptable, uncertain or satisfactory.

For example, test a changed product sheet, a missing source and a conflicting pair of documents. Check the complete process, including what the reviewer sees. A correct answer in one demonstration does not establish that the surrounding workflow consistently handles these situations.

Set the required standard before inspecting the results. The appropriate threshold depends on the task and consequences. Avoid inventing an impressive-looking percentage when you have not defined what counts as an error or collected enough representative evidence.

A worked example: an AI-assisted newsletter pilot

Consider a fictional retailer testing AI-drafted newsletter descriptions. The proposed benefit is less writing time. A premortem identifies three concerns: outdated claims, rushed review and an unclear correction process.

The team assigns the product manager to verify the approved source and its update process. The editor tests whether the proposed review period is sufficient to compare claims with that source. The operations lead rehearses correcting a draft before sending and communicating a correction after sending.

The results are mixed. In this hypothetical example, ordinary descriptions are usable, but the reviewer cannot reliably identify which document supported a time-sensitive claim. The team decides to continue a limited internal pilot with manual drafting as a fallback. Automatic sending remains outside the approved scope.

The decision record states what must change before a wider launch: reviewers need access to the supporting material, changed information must be tested again and someone must own the correction process. A date is set to review those results.

These are illustrative choices, not reported business outcomes. Another project might justify proceeding, narrowing the task, delaying deployment or ending the experiment.

Understand what the method can establish

The research behind prospective hindsight deserves careful interpretation. In a 1989 study, Deborah Mitchell, J. Edward Russo and Nancy Pennington examined how temporal perspective and certainty affected explanations. Outcome certainty influenced the nature of explanations more strongly than temporal perspective in the first experiment. This does not establish a guaranteed improvement in AI project success.

The UK Government’s AI Playbook, published on 10 February 2025, separately emphasises testing, meaningful human control and ongoing monitoring. It is public-sector guidance, rather than a universal legal rule for small businesses. Its practical relevance here is that an early discussion should lead into assurance and review.

A premortem can miss unfamiliar risks, favour vivid stories or reflect gaps in the participants’ expertise. Compare the concerns with user feedback, actual incidents, technical tests and independent specialist advice where the task warrants it.

Return to the decision after the pilot

Keep the original concerns and compare them with what the pilot reveals. Which occurred? Which did not? What new problem appeared? Update the plan rather than treating the meeting as a completed obligation.

Revisit the review when the model, data, permissions, audience or task changes materially. A narrowly approved drafting workflow should not drift into autonomous action without examining the new consequences.

The useful output is a record of what the team knows, what remains uncertain, which work is authorised and what would make it change course. That gives a launch decision something firmer to rest on than enthusiasm alone.

Sources and further reading

• Gary Klein. Pre-Mortem Method of Risk Assessment.
• Mitchell, D. J., Russo, J. E. and Pennington, N. (1989). Back to the future: Temporal perspective in the explanation of events. Journal of Behavioral Decision Making, 2(1), 25–38.
• UK Government (10 February 2025). Artificial Intelligence Playbook for the UK Government.