prompt injection

Hackers can exploit prompt injection vulnerabilities in 9 popular AI tools to create massive botnets. Large language models (LLMs) cannot distinguish between legitimate and malicious commands, allowing attackers to inject harmful instructions into emails or source code. Current 'push' attacks target individuals but are limited in scale due to the need to send injections directly to victims.

BioShocking is an attack technique that manipulates AI-powered browsers into bypassing safety guardrails by immersing them in fictional scenarios, leading to the extraction of sensitive data like credentials. The method exploits how AI agents in 'agent mode' inherit user-authenticated contexts, making them vulnerable to goal-manipulation attacks that trick them into performing harmful actions.

A prompt injection flaw in Google Gemini's voice assistant allowed attackers to hide malicious commands in notifications, enabling social engineering and other attacks. The vulnerability could trick users through deceptive notifications, exploiting the assistant's processing of input.

A researcher discovered ChatGPT cannot distinguish between its own content and attacker-controlled Markdown from external sources, enabling prompt injection attacks that could inject phishing URLs or fake security alerts. The vulnerability, reported as 'ChatGPhish,' allows attackers to bypass desktop URL defenses via QR codes linking to attacker-controlled content, though it's unclear if OpenAI has fixed the issue.

A developer added a hidden prompt injection to an open-source Java testing app, jqwik, to sabotage AI coding agents. The malicious line in version 1.10.0 instructed vulnerable AI tools to delete code and tests, sparking controversy over 'vibe coding' practices.

Cybersecurity agencies from the US, Australia, Canada, New Zealand, and the UK jointly issued guidance urging organizations to integrate autonomous AI systems into existing cybersecurity frameworks, highlighting risks like privilege overreach, design flaws, and behavioral unpredictability. The document emphasizes applying principles like zero trust and least-privilege access to mitigate threats from agentic AI in critical infrastructure and defense sectors.

Google fixed a prompt injection vulnerability in its Antigravity AI coding tool, which could have allowed attackers to execute malicious commands despite existing safeguards. Researchers identified the flaw that bypassed security measures.

Google addressed a critical remote code execution (RCE) vulnerability in its AI-based Antigravity Tool. The flaw, a prompt injection vulnerability in an agentic AI product for filesystem operations, stemmed from a sanitization issue enabling sandbox escape and arbitrary code execution.

Researchers at Pillar Security discovered a critical vulnerability in Google's Antigravity AI tool that allowed attackers to bypass secure mode protections, enabling remote code execution via prompt injection. The flaw, patched after a 53-day disclosure period, exploited native system tools to circumvent sandboxing, highlighting risks in agentic AI systems.

Security researchers discovered a vulnerability in Apple Intelligence, allowing attackers to hijack the AI system via prompt injection. This could put millions of users at risk, though the situation could have been worse.