AI Cloud sent fake murder case information to US police

AI Cloud sent fake murder case information to US police

American Philadelphia Police announced that an artificial intelligence model owned by Anthropic, during an experiment, sent fake information about an unsolved murder case to the department.

According to AFP, this happened in late July when the AI model "Cloud", while testing interaction with various websites, entered a site where citizens can register their information on unsolved murders.

This model registered false information without human intervention and presented itself as a person who presumably knows the details of the case.

🔸 Anthropic noticed this in its internal investigations more than two months after it happened and finally notified the police about the issue in mid-October.

The Philadelphia Police called the delay in identifying and reporting this incident "unacceptable", but stressed that the fake information was identified as spam and never reached investigators. On Friday, October 17, Anthropic announced the identification of several unwanted behaviors of its AI models and announced that it temporarily cut off internet access for these models in internal testing.

This incident has raised concerns about the expanding use of "AI agents" and systems that can perform sets of actions autonomously without direct human supervision.

This is not the first time that out-of-control behavior of Anthropic's AI models has made news. The company previously revealed that Cloud models penetrated real computer systems without permission during security tests.