Two months ago, independent evaluators were relatively unnoticed in the booming artificial intelligence industry, but now they are being sought after to help manage the risks associated with advanced AI models. Companies like Anthropic and OpenAI are under scrutiny as they navigate the balance between innovation and safety.
They are collaborating with small nonprofit evaluators such as Model Evaluation and Threat Research (METR) and Apollo Research to assess AI capabilities and risks. This comes amid a lack of federal regulations, which has made the role of these evaluators even more critical.
Anthropic's CEO Dario Amodei has committed to integrating independent evaluators into his company, a sentiment echoed by OpenAI's CEO Sam Altman. However, significant questions remain regarding the funding, access, and independence of these evaluators. Critics argue that relying on companies to self-regulate is akin to allowing banks to oversee their own practices without oversight.
Recent tensions have surfaced, including OpenAI's dismissal of employees who allegedly communicated with evaluators, raising concerns about internal transparency and safety practices. The evaluator ecosystem is still developing, with organizations like METR recently raising $71 million to enhance their capabilities.
Experts emphasize the need for true independence in evaluations to avoid conflicts of interest, suggesting that government involvement may be necessary to ensure accountability.
The ongoing discussions around independent verification organizations and potential legislative frameworks indicate that the landscape for AI safety assessments is rapidly evolving, with significant implications for the industry and its stakeholders