← Back to Library

Security Considerations for Artificial Intelligence Agents

Authors: Ninghui Li, Kaiyuan Zhang, Kyle Polley, Jerry Ma

Published: 2026-03-12

arXiv ID: 2603.12230v1

Added to Library: 2026-03-13 03:04 UTC

Red Teaming

📄 Abstract

This article, a lightly adapted version of Perplexity's response to NIST/CAISI Request for Information 2025-0035, details our observations and recommendations concerning the security of frontier AI agents. These insights are informed by Perplexity's experience operating general-purpose agentic systems used by millions of users and thousands of enterprises in both controlled and open-world environments. Agent architectures change core assumptions around code-data separation, authority boundaries, and execution predictability, creating new confidentiality, integrity, and availability failure modes. We map principal attack surfaces across tools, connectors, hosting boundaries, and multi-agent coordination, with particular emphasis on indirect prompt injection, confused-deputy behavior, and cascading failures in long-running workflows. We then assess current defenses as a layered stack: input-level and model-level mitigations, sandboxed execution, and deterministic policy enforcement for high-consequence actions. Finally, we identify standards and research gaps, including adaptive security benchmarks, policy models for delegation and privilege control, and guidance for secure multi-agent system design aligned with NIST risk management principles.

🔍 Key Points

  • The paper identifies unique security threats and vulnerabilities that AI agent systems face, especially regarding the blurring of code and data boundaries, which exacerbate traditional risks like code injection and data leakage.
  • It proposes a layered defense strategy against attacks, notably indirect prompt injection attacks and cascading failures in workflows, outlining input-level, model-level, and system-level defenses that must work together for effective security.
  • The paper discusses the need for new security standards and guidelines tailored for multi-agent systems, emphasizing the importance of understanding the operational contexts and deployment models of AI agents.

💡 Why This Paper Matters

This paper provides essential insights into the evolving security landscape of AI agent systems, highlighting new vulnerabilities, the complexity of multi-agent interactions, and the necessity for novel security frameworks. Its recommendations for layered defenses and new standards are crucial for safeguarding the integrity, confidentiality, and availability of AI-driven applications.

🎯 Why It's Interesting for AI Security Researchers

Given the increasing integration of AI agents in diverse applications across industries, this paper is vital for AI security researchers who must address emerging vulnerabilities and develop robust security mechanisms. Its focus on practical implications and defense strategies is particularly relevant as organizations seek to implement secure AI systems.

📚 Read the Full Paper