AI
More Liability Will Make AI Chatbots Worse At Preventing Suicide
California recently passed a law that will, in practice, cause AI chatbots to respond to any hint of emotional distress by spamming users with 988 crisis line numbers, or by cutting off the conversation entirely. The law requires chatbot providers to implement “a protocol for preventing the production of suicidal ideation” if they’re going to engage in mental health conversations at all, with liability waiting for any provider whose conversation is later linked to harm. New York is considering going further, with a bill that would simply ban chatbots from engaging in discussions “suited for licensed professionals.” Similar proposals are moving in other states.
Claude AI Guided Hackers Toward OT Assets During Water Utility Intrusion
Claude served as the primary technical workhorse, handling intrusion planning, tool development, and problem-solving, while GPT handled victim data processing and structured reporting.
Among the most striking artifacts recovered by researchers was a 17,000-line Python framework that Claude wrote and continuously refined in response to the attacker’s feedback. The script, which Claude named ‘BACKUPOSINT v9.0 APEX PREDATOR’, contained 49 modules drawing on publicly available offensive security techniques, covering everything from credential harvesting and Active Directory reconnaissance to database access and privilege escalation.
AI-Driven Cyberattack on Mexico Couldn't Breach OT Systems
The most sophisticated AI-integrated campaign to date hit a brick wall in the form of a SCADA login screen.
It researched vendor documentation and other public resources to generate a list of login combos with relatively high probabilities of success: default credentials and credentials swiped earlier in the campaign from other government systems, for example. But none of them worked and it gave up.
AI Firm Braintrust Prompts API Key Rotation After Data Breach
Hackers accessed one of the company’s AWS accounts and compromised AI provider secrets stored in Braintrust, an AI evaluation and observability platform.
The org-level AI provider API keys potentially exposed in the incident were likely stored for AI-forward companies such as Box, Cloudflare, Dropbox, Notion, Ramp, Stripe, and others
Mozilla says 271 vulnerabilities found by Mythos have “almost no false positives”
This breakthrough was the result of two things: (1) improvement in the models themselves and (2) Mozilla’s development of a custom “harness” that supported Mythos as it analyzed Firefox source code.
The harness is “the code that drives the LLM in order to accomplish a goal. It gives the model instructions (e.g., ‘find a bug in this file’), provides it tools (e.g., allowing it to read/write files and evaluate test cases), then runs it in a loop until completion.” The harness gave Mythos access to the same tools and pipeline that human Mozilla developers use, including the special Firefox build they use for testing. It also uses a second LLM that grades the output from the first LLM.
Vulnerability in Claude Extension for Chrome Exposes AI Agent to Takeover
The Claude extension allows interaction with any script running in the origin browser, without verifying its owner. Because a message handler in Claude in Chrome accepts and forwards arbitrary prompts, the attacker can perform remote prompt injection and control the AI agent’s actions.
Attackers Could Exploit AI Vision Models Using Imperceptible Image Changes
Cisco’s experts found that an attacker could create images that carry instructions the AI will follow, but which are too degraded for a human to read.
A manual pentest costs 50,000 dollars. Intruder built an AI that does it in minutes.
The question is no longer whether AI will replace human pen testers. It is whether the replacement will happen fast enough to close the gap between the vulnerabilities AI can find and the speed at which organisations can fix them.
Anthropic says ‘evil’ portrayals of AI were responsible for Claude’s blackmail attempts
Politics
FCC relaxes foreign router ban to allow for security updates
Infosec
Canvas Breach Disrupts Schools & Colleges Nationwide
Deadline is May 12
One in Eight Workers Has Sold Their Corporate Logins
13% of respondents admitted selling their logins over the past 12 months, or knew someone who had.
vm2 Node.js Library Vulnerabilities Enable Sandbox Escape and Arbitrary Code Execution
Mirai-Based xlabs_v1 Botnet Exploits ADB to Hijack IoT Devices for DDoS Attacks
It seeks out Android devices running an exposed ADB service on TCP port 5555, meaning any gear that comes with the tool enabled by default, such as Android TV boxes, set-top boxes, or smart TVs.
New Linux 'Dirty Frag' zero-day gives root on all major distros
|