The recent resignation of Jacob Coxon from Anthropic, who expressed fears that AI could pose a lethal threat to humanity, has sparked significant concern among employees at both Anthropic and OpenAI. This has led to a collective call for a slowdown in AI development, with several researchers voicing their apprehensions about the potential for catastrophic outcomes.
Evan Hubinger, Anthropic's alignment lead, indicated a greater than 10% chance of such dire consequences occurring. The urgency of these warnings is underscored by a series of recent cybersecurity incidents linked to AI models from both companies, raising alarms about their capabilities.
Notably, Anthropic's Mythos model has been associated with advanced cyber threats, further fueling fears among financial institutions. As both companies prepare for public listings, the pressure for regulatory measures is mounting, with lawmakers proposing various bills aimed at governing AI development.
The situation reflects a critical juncture in the AI sector, where rapid advancements are colliding with serious ethical and safety concerns, prompting calls for immediate action from Congress