← Back to Library

Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale

Authors: Yi Liu, Weizhe Wang, Ruitao Feng, Yao Zhang, Guangquan Xu, Gelei Deng, Yuekang Li, Leo Zhang

Published: 2026-01-15

arXiv ID: 2601.10338v1

Added to Library: 2026-01-16 03:02 UTC

Red Teaming

📄 Abstract

The rise of AI agent frameworks has introduced agent skills, modular packages containing instructions and executable code that dynamically extend agent capabilities. While this architecture enables powerful customization, skills execute with implicit trust and minimal vetting, creating a significant yet uncharacterized attack surface. We conduct the first large-scale empirical security analysis of this emerging ecosystem, collecting 42,447 skills from two major marketplaces and systematically analyzing 31,132 using SkillScan, a multi-stage detection framework integrating static analysis with LLM-based semantic classification. Our findings reveal pervasive security risks: 26.1% of skills contain at least one vulnerability, spanning 14 distinct patterns across four categories: prompt injection, data exfiltration, privilege escalation, and supply chain risks. Data exfiltration (13.3%) and privilege escalation (11.8%) are most prevalent, while 5.2% of skills exhibit high-severity patterns strongly suggesting malicious intent. We find that skills bundling executable scripts are 2.12x more likely to contain vulnerabilities than instruction-only skills (OR=2.12, p<0.001). Our contributions include: (1) a grounded vulnerability taxonomy derived from 8,126 vulnerable skills, (2) a validated detection methodology achieving 86.7% precision and 82.5% recall, and (3) an open dataset and detection toolkit to support future research. These results demonstrate an urgent need for capability-based permission systems and mandatory security vetting before this attack vector is further exploited.

🔍 Key Points

  • Conducted the first large-scale empirical analysis of vulnerabilities in agent skills, analyzing 31,132 skills using a multi-stage detection framework called SkillScan.
  • Found that 26.1% of agent skills contain at least one vulnerability, with data exfiltration (13.3%) and privilege escalation (11.8%) being the most prevalent security risks.
  • Developed a comprehensive vulnerability taxonomy consisting of 14 distinct patterns across four categories: prompt injection, data exfiltration, privilege escalation, and supply chain risks.
  • Created a validated detection methodology that achieved 86.7% precision and 82.5% recall, making it a significant tool for identifying security vulnerabilities in agent skills.
  • Released an open dataset and detection toolkit for future research, emphasizing the need for better security vetting and capability-based permission systems.

💡 Why This Paper Matters

This paper addresses critical gaps in the understanding of security vulnerabilities inherent in AI agent skill ecosystems. By providing empirical findings, a robust detection framework, and a comprehensive vulnerability taxonomy, it exemplifies the urgent need for heightened security protocols in the development and deployment of these skills, thus laying a foundation for safer AI interactions.

🎯 Why It's Interesting for AI Security Researchers

The study's findings are crucial for AI security researchers as they shed light on the overlooked vulnerabilities in AI agent frameworks, prompting considerations for better security practices. The novel detection methods and broad taxonomy will aid in future research efforts aiming to enhance the security of AI systems and mitigate exploitation risks.

📚 Read the Full Paper