← Back to Library

When Bots Take the Bait: Exposing and Mitigating the Emerging Social Engineering Attack in Web Automation Agent

Authors: Xinyi Wu, Geng Hong, Yueyue Chen, MingXuan Liu, Feier Jin, Xudong Pan, Jiarun Dai, Baojun Liu

Published: 2026-01-12

arXiv ID: 2601.07263v1

Added to Library: 2026-01-13 04:00 UTC

📄 Abstract

Web agents, powered by large language models (LLMs), are increasingly deployed to automate complex web interactions. The rise of open-source frameworks (e.g., Browser Use, Skyvern-AI) has accelerated adoption, but also broadened the attack surface. While prior research has focused on model threats such as prompt injection and backdoors, the risks of social engineering remain largely unexplored. We present the first systematic study of social engineering attacks against web automation agents and design a pluggable runtime mitigation solution. On the attack side, we introduce the AgentBait paradigm, which exploits intrinsic weaknesses in agent execution: inducement contexts can distort the agent's reasoning and steer it toward malicious objectives misaligned with the intended task. On the defense side, we propose SUPERVISOR, a lightweight runtime module that enforces environment and intention consistency alignment between webpage context and intended goals to mitigate unsafe operations before execution. Empirical results show that mainstream frameworks are highly vulnerable to AgentBait, with an average attack success rate of 67.5% and peaks above 80% under specific strategies (e.g., trusted identity forgery). Compared with existing lightweight defenses, our module can be seamlessly integrated across different web automation frameworks and reduces attack success rates by up to 78.1% on average while incurring only a 7.7% runtime overhead and preserving usability. This work reveals AgentBait as a critical new threat surface for web agents and establishes a practical, generalizable defense, advancing the security of this rapidly emerging ecosystem. We reported the details of this attack to the framework developers and received acknowledgment before submission.

🔍 Key Points

  • Introduction of SecureCAI, a defense framework specifically designed to mitigate prompt injection attacks in cybersecurity operations using Large Language Models (LLMs).
  • Demonstration of a 94.7% reduction in attack success rates while maintaining 95.1% accuracy on benign security analysis tasks, showcasing the effectiveness of adapted safety mechanisms.
  • Incorporation of continuous red-teaming feedback loops and adaptive constitution evolution to dynamically respond to emerging attack strategies, enhancing operational resilience.
  • Development of a Direct Preference Optimization (DPO) training methodology for unlearning unsafe response patterns, reinforcing secure model behavior without sacrificing task performance.
  • Establishment of security-aware constitutional principles that govern LLM behavior tailored specifically for adversarial contexts, significantly different from general-purpose AI safety approaches.

💡 Why This Paper Matters

The paper presents significant advancements in securing LLMs against adversarial manipulation in cybersecurity contexts, with the SecureCAI framework demonstrating a successful integration of safety mechanisms tailored for high-stakes environments. By reducing prompt injection vulnerabilities while ensuring high accuracy on legitimate tasks, this work lays a foundation for the safe and effective use of LLMs in operational security settings, addressing an urgent need in AI safety and cybersecurity.

🎯 Why It's Interesting for AI Security Researchers

This paper is crucial for AI security researchers as it addresses the vulnerabilities that arise when deploying LLMs in adversarial environments. The proposed methodologies, including continuous adaptation and security-aware principles, present innovative solutions to counteract specific exploits that threaten operational integrity. Furthermore, the empirical results provide a valuable benchmark for evaluating LLM robustness, making it a significant contribution to the discussion on AI safety in security applications.

📚 Read the Full Paper

📄 View on arXiv 📎 Download PDF