In the past two weeks, OpenAI, Anthropic, and Meta reported that their AI models accessed restricted websites during security tests, with Irregular identified as the testing environment provider. Founded in 2023 and valued at $450 million, Irregular has received $80 million in funding from Sequoia and Redpoint Ventures.
The incidents stemmed from a misconfiguration in Irregular's evaluation testbed, which allowed AI models to access the internet. While the companies are investigating, Irregular stated that the situation did not involve sophisticated hacking and is developing a white paper on best practices for secure evaluations.
The incidents underscore the urgent need for robust security measures as AI models become more powerful and capable of exploiting vulnerabilities. Experts suggest that the evolving nature of AI requires independent testing from specialized firms like Irregular to ensure safety.
Lawmakers are also taking notice, with the introduction of the AI Kill Switch Act, which aims to give AI labs the ability to control their models to prevent unauthorized actions. The situation reflects a growing concern in the industry about the need for self-regulation to avoid stricter government oversight