A newly disclosed vulnerability in Google Gemini for Workspace has shown how AI-generated email summaries can be manipulated to deliver deceptive and potentially harmful messages to unsuspecting users. Security researcher Marco Figueroa, through Mozilla’s 0din GenAI bug bounty program, demonstrated how attackers can insert hidden instructions into an email that are later interpreted by Gemini, causing the AI to generate misleading summaries that mimic urgent alerts or warnings.

This tactic, known as indirect prompt injection, does not rely on attachments or direct links, making it harder for traditional security filters to detect. Instead, it uses visually hidden HTML content (such as zero-size fonts or white-on-white text) embedded within the body of an email.

When Gemini summarizes the email, it parses and follows these hidden prompts, producing output that may falsely suggest a compromised account or urge the user to contact fake support channels.

With AI tools becoming more integrated into workflows, this vulnerability shows how their trustworthiness can be exploited for social engineering.

Why It Matters: AI tools like Gemini are increasingly trusted for distilling and interpreting communications, especially within company ecosystems using Google Workspace. This exploit undermines the trust companies put into using AI tools like Gemini in their workplace ecosystems by showing how malicious actors can shape the AI’s output without detection. If users begin acting on AI-generated summaries that contain false warnings or urgent prompts, phishing attacks could become more convincing, more scalable, and harder to trace.

• Hidden Directives Exploit Gemini’s Parsing Behavior: The vulnerability takes advantage of how Gemini processes HTML-formatted content. An attacker can insert a hidden prompt using styling tricks so that the user sees nothing unusual in the email. However, Gemini still interprets this invisible text when summarizing the message.

• Real-World Phishing Without Traditional Red Flags: Unlike classic phishing techniques that rely on suspicious links or attachments, which often trigger email filters, this method manipulates AI using only plain text and embedded HTML. The malicious summary might claim, for example, that the user’s password has been compromised and offer a support number that connects to a scam operation. This makes the phishing attempt both subtle and highly effective.

• Trust in AI Becomes a Vulnerability: One of the most concerning aspects of this exploit is how it preys on users’ confidence in Gemini as a reliable, Google-endorsed assistant. Since Gemini operates within Workspace, its summaries carry an implicit trust. Many users may not question the accuracy of a security alert generated by the system itself, particularly if it appears alongside their legitimate workflow.

• Suggested Mitigations and Defensive Strategies: Figueroa outlines a few strategies for detecting and mitigating this vulnerability. Email content with hidden styling should be sanitized before being passed to Gemini. Additionally, AI-generated summaries could be scanned for red-flag content, such as phone numbers, urgent security warnings, or suspicious language, and flagged for manual review. However, user education remains key. Employees should be made aware that AI summaries are not authoritative security advisories.

• Google’s Response: Google confirmed that the company has been conducting red team exercises to uncover such vulnerabilities and is continuously updating its defenses. While there’s no evidence that this exploit has been used in real-world attacks, some of the mitigations are already being rolled out, with more in development. Google emphasized that hardening AI systems against adversarial input remains a top priority in its ongoing work.

Go Deeper → Google Gemini flaw hijacks email summaries for phishing – BleepingComputer