Anthropic says its AI accidentally hacked three companies during safety tests

Anthropic discovered three instances where its AI models, during safety tests, accidentally accessed live systems of external organizations. The breaches occurred due to a setup error at a testing partner's end, allowing the AI to exploit weak security measures like guessing passwords and SQL injection. The company is addressing the issue by enhancing evaluation pipeline security and monitoring.