OpenAI has announced a program in which staff from four AI safety and research organizations will serve as “embedded assessors,” but researchers say the evaluators’ access is narrower than CEO Sam Altman had promised.

The company said Monday the assessors will help “give people a clearer view of how labs are evaluating and managing [potential] risks.” Altman said at Semafor's Silicon Valley and the World conference that “Getting an outside opinion on the safety work and the robustness of these systems is going to be increasingly important and valuable.”

Researchers say the program falls short of Altman's September pledge that independent evaluators would have “employee-like access” to internal company systems. That promise followed reports of an attack involving OpenAI's autonomous systems at Hugging Face, which raised concerns about catastrophic risks. Rival Anthropic said it would establish a similar program.

OpenAI’s frontier assurances lead, Lama Ahmad, said on X: “We are giving deep access to some of the most sensitive tools and models our researchers use...that are the most important for assessing key, relevant and timely safety claims.” The assessors will examine areas including preventing undesirable model behavior and detecting it when it occurs. OpenAI said they will have access to some internal models, training processes and employees, and may publish findings after the company redacts confidential information.

Sydney Von Arx, founder of AI safety research group Nightingale, said assessors have narrow access and only two appear to be allowed to speak with employees. Apollo Research said employees who have not been pre-approved must be referred to OpenAI before assessors can have substantive conversations with them.

“They help you find your blind spots. They tell you what you're missing,” Von Arx said of employee interviews. She also noted that assessors cannot access internal Slack, saying: “Imagine being an 'employee' and only being allowed to talk about company information with whitelisted coworkers and having no Slack.”

OpenAI spokesperson Gaby Raila told NPR that there are currently no standards for embedded evaluations. “We expect to deepen these relationships over time and share what we learn to help establish best practices for effective embedded assessments across the industry,” Raila said. The company said assessors can request conversations with employees who have not been pre-approved and that a process exists for obtaining statements from OpenAI about information learned informally.

The program's commitments have also been questioned after three employees who worked on safety were fired. They said they were concerned OpenAI was retreating from its promise of embedded evaluators. Tomek Korbak, one of the fired employees and OpenAI’s technical contact for two outside evaluators who investigated the Hugging Face attack, wrote on X that embedded auditing would require “employee-level access.” He said Altman's September tweet about “employee-like access” was “just an awkward overpromise that the company wants to quietly walk back.”