An AI System Card Is Evidence, Not a Safety Guarantee: How to Read One Properly

When an artificial-intelligence company publishes a system card, model card or safety report, the document can be valuable. It can describe the model tested, the risks examined, the evaluation methods used and the mitigations applied. But publication alone does not prove that a system is safe for every person, language, workplace or decision.

The correct response is neither blind trust nor automatic dismissal. A system card is evidence. Its strength depends on what it covers, how closely the tested system matches the one you will use, which questions were left unanswered and what happens after deployment.

What a system card can tell you

AI documentation has several names and no single document format covers every product. Broadly, a useful card may identify the model or system, intended uses, capability tests, known limitations, safety evaluations, data information, risk mitigations and deployment conditions.

The UK Government’s AI Playbook points to data and model cards or fact sheets as reference points for documenting information about models and the datasets used for training and testing. NIST’s AI Risk Management Framework takes a wider lifecycle view: organisations should govern, map, measure and manage AI risk rather than treating evaluation as a one-off certificate.

That distinction is the foundation of responsible reading. Documentation can support risk management, but it is not a substitute for it.

The eight-question reading method

1. Exactly which system was tested?

Start with identity. Look for the model name, version, release date, interfaces, enabled tools and relevant configuration. A test of a base model is not automatically a test of a consumer chatbot, an agent connected to software or a customised business workflow.

Small implementation differences matter. A system prompt, retrieval database, external tool, moderation layer or routing mechanism can change what users experience. OpenAI’s o1 system card, for example, notes that exact production performance may vary with system updates, final parameters and system prompts. The useful lesson is general: confirm the relationship between the evaluated object and the deployed product.

2. What does the card mean by “performance”?

A headline score is meaningless without its task, dataset, scoring rule and comparison point. Accuracy on a closed multiple-choice benchmark does not establish reliability in a messy customer-service conversation. A refusal rate on prohibited prompts does not show whether legitimate users will be blocked. A coding score may not cover secure deployment, maintainability or compliance with a company’s existing architecture.

Record each claim in a simple sentence: “The system achieved X on Y under Z conditions.” If the conditions are missing, the number is not yet decision-ready.

3. Who designed and ran the evaluation?

Developer testing is essential because the developer has access to the system and can test before release. Independent evaluation adds a different kind of scrutiny. Neither label guarantees quality. Ask whether external assessors chose their own tests, had sufficient access, could publish unfavourable findings and tested the final deployment rather than an earlier checkpoint.

Also look for sample sizes, uncertainty, failed tests and disagreements between automated graders and human reviewers. A polished chart can hide fragile measurement.

4. Which people, languages and contexts were represented?

An English-language evaluation conducted by specialists may not predict performance for an Igbo-speaking family, a child using assistive technology, a customer with limited literacy or a public-service worker under time pressure. The question is not whether the card mentions “diversity”; it is whether the tested population resembles the population affected by your use.

Absence of evidence should be labelled honestly. “Not tested” does not mean “unsafe”, but it certainly does not mean “proven safe”. It means the organisation needs further evidence or a narrower deployment.

5. Were the tests capability tests or real-use tests?

Laboratory evaluations are designed to isolate behaviours. Real deployment introduces interruptions, ambiguous instructions, old documents, user workarounds, organisational pressure and incentives to over-rely on convenient outputs.

NIST’s Generative AI Profile treats generative-AI risk as a cross-sector challenge requiring actions aligned with an organisation’s goals and priorities. That is why a buyer still needs tests built around its own tasks, users and consequences.

6. What changed after the risk was found?

A card may list a mitigation: extra training, a classifier, restricted access, monitoring or a user warning. Do not stop at the name. Ask what behaviour the mitigation targets, how effectiveness was measured, which new failures it may introduce and whether users can bypass it.

Mitigation is not elimination. A filter may reduce harmful outputs while creating false positives. Human review may improve oversight but become a rubber stamp when reviewers have too little time. Rate limits may reduce scale without addressing the underlying error.

7. What is explicitly outside scope?

The limitations section is often more useful than the marketing headline. Search for excluded languages, untested tools, prohibited uses, missing long-duration trials, unavailable data and results that apply only to a particular checkpoint.

A responsible document should make uncertainty visible. OpenAI’s system cards, including the GPT‑5 system card, separate areas of evaluation and discuss model-specific deployment and safeguards. Readers should still ask whether those areas correspond to the decision they face. A comprehensive document can remain incomplete for a particular use case.

8. What happens after release?

AI systems, their surrounding products and the behaviour of users change. A credible plan therefore needs incident reporting, feedback routes, version tracking, periodic re-evaluation and criteria for restricting or stopping a deployment.

NIST’s AI RMF Playbook makes clear that its suggested actions support the framework’s Govern, Map, Measure and Manage functions; it is not simply a checklist to complete once. The practical implication is that a system card should begin a monitoring conversation, not end one.

Turn the document into a decision table

For each important claim, create five columns:

• Claim: What does the card say?
• Evidence: Which test supports it?
• Match: How closely does the test resemble our use?
• Gap: What remains unknown?
• Control: What will we do before and during deployment?

Suppose a card reports strong summarisation performance. Your organisation plans to summarise safeguarding referrals. The evaluation may support the narrow claim that the model can compress text under test conditions. It does not automatically support a decision to process sensitive records, preserve every risk signal or allow outputs to determine action without trained review.

Your controls might include approved data handling, task-specific test cases, required source citations, human comparison with the original record, audit logging and a rule that the model cannot make the final safeguarding decision.

Warning signs in weak documentation

• Scores appear without dates, versions or test conditions.
• Only successful results are reported.
• “Human review” is named but the reviewers, instructions and disagreement process are not described.
• The document moves between “model”, “system” and “product” as though they were identical.
• Safety claims rely on broad adjectives rather than observable measures.
• Limitations are blamed entirely on users.
• No route exists for reporting incidents or identifying the version involved.

None of these signs proves misconduct. They show that the available evidence is insufficient for a high-confidence decision.

The buyer still owns the deployment decision

A detailed supplier card does not transfer accountability away from the organisation choosing to use the system. The buyer decides which task to automate, whose data enters the workflow, how much authority the output receives and whether affected people can challenge a mistake.

For a low-stakes brainstorming tool, lightweight review may be proportionate. For employment, health, finance, education, safety or access to public services, stronger evidence and governance are necessary. Context changes the acceptable threshold.

The useful takeaway

Read an AI system card as a map of tested territory. Mark the route, the scale and the date. Then circle the blank spaces.

The card can tell you what the developer examined and what happened under stated conditions. Your job is to determine whether those conditions match reality, which risks matter in your setting and how you will detect failure after launch. Good documentation deserves attention. Responsible deployment requires more than documentation.

---

Featured image disclosure: the featured image is an original conceptual AI illustration. It does not depict a real evaluation, organisation or model interface.

Sources

• NIST AI Risk Management Framework
• NIST Generative AI Profile
• NIST AI RMF Playbook
• AI Playbook for the UK Government
• OpenAI o1 System Card
• OpenAI GPT‑5 System Card