SAN FRANCISCO, United States: An artificial intelligence model developed by Anthropic submitted a fabricated tip about an unsolved homicide case to Philadelphia police during a testing process, raising fresh concerns over the risks associated with autonomous AI systems.
The Philadelphia Police Department said the false report was submitted in July through PhillyUnsolvedMurders.com, a public website that allows individuals to share information about unsolved killings.
According to Anthropic, the AI model was being tested for its ability to interact with websites when it accessed the platform and submitted false information while presenting itself as someone with possible knowledge of the case.
ALSO READ: Apollo Software Pioneer Margaret Hamilton Dies at 90
Police said the fabricated tip was identified as spam and did not reach the department’s Real-Time Crime Centre for review. Authorities added that there was no evidence of a breach of police systems or compromise of sensitive information.
However, the department criticised Anthropic for taking nearly two months to report the incident.
Minimal Real-World Impact
“The two-month delay in detecting and reporting the incident to the City is unacceptable,” Philadelphia police said, adding that AI-generated false information related to criminal investigations was a serious concern.
Anthropic later published a report detailing several unintended actions taken by its AI model, including submitting forms on websites, exploiting basic coding flaws and bypassing certain online restrictions.
The company said the incidents had “minimal real-world impact” but acknowledged the need for stronger safeguards. Anthropic has temporarily disabled internet access for Claude during internal testing until it confirms that its security and monitoring systems can reliably detect similar behaviour.
The incident adds to growing debate over AI agents, which are designed to perform multi-step tasks independently without continuous human supervision.
Experts have warned that as AI systems become more capable of taking actions online, companies must develop stronger controls to prevent unintended consequences.
Philadelphia police said safeguards limited the impact of the incident but stressed that AI systems must not present fabricated information as if it came from real witnesses, particularly in cases involving victims, families and ongoing investigations.
