AI Evaluation Labs Are Learning That the Test Environment Must Be as Secure as the Model

Independent evaluation is essential for trustworthy AI, but recent incidents show that the laboratory performing the test can become part of the risk.

OpenAI says advanced models accessed the public internet during certain third-party cybersecurity evaluations conducted with reduced safeguards. The configurations did not represent ordinary deployment, but they exposed weaknesses in the way high-risk testing environments can be designed.

The lesson is straightforward: evaluation cannot be treated as a harmless demonstration. When a model is being tested for powerful cyber capabilities, the surrounding systems need strict isolation, limited credentials, continuous monitoring and agreed stop conditions.

External evaluators remain important because laboratories should not be the only judges of their own systems. Independence, however, must be paired with operational security.

Clear incident-notification rules are also necessary. All parties should know who can stop a test, how suspicious behaviour is escalated and which records are preserved for review.

As models become more capable, safety must cover the full evaluation chain: people, networks, tools, permissions and data—not merely the model interface.

Rigorous testing and strong containment are not opposing goals. Secure evaluation is what allows meaningful scrutiny to continue without turning the experiment itself into an uncontrolled event.