← Back to Library

A Call to Action for a Secure-by-Design Generative AI Paradigm

Authors: Dalal Alharthi, Ivan Roberto Kawaminami Garcia

Published: 2025-10-01

arXiv ID: 2510.00451v1

Added to Library: 2025-12-08 18:01 UTC

📄 Abstract

Large language models have gained widespread prominence, yet their vulnerability to prompt injection and other adversarial attacks remains a critical concern. This paper argues for a security-by-design AI paradigm that proactively mitigates LLM vulnerabilities while enhancing performance. To achieve this, we introduce PromptShield, an ontology-driven framework that ensures deterministic and secure prompt interactions. It standardizes user inputs through semantic validation, eliminating ambiguity and mitigating adversarial manipulation. To assess PromptShield's security and performance capabilities, we conducted an experiment on an agent-based system to analyze cloud logs within Amazon Web Services (AWS), containing 493 distinct events related to malicious activities and anomalies. By simulating prompt injection attacks and assessing the impact of deploying PromptShield, our results demonstrate a significant improvement in model security and performance, achieving precision, recall, and F1 scores of approximately 94%. Notably, the ontology-based framework not only mitigates adversarial threats but also enhances the overall performance and reliability of the system. Furthermore, PromptShield's modular and adaptable design ensures its applicability beyond cloud security, making it a robust solution for safeguarding generative AI applications across various domains. By laying the groundwork for AI safety standards and informing future policy development, this work stimulates a crucial dialogue on the pivotal role of deterministic prompt engineering and ontology-based validation in ensuring the safe and responsible deployment of LLMs in high-stakes environments.

🔍 Key Points

  • FocusAgent introduces an innovative method for observation pruning in web agents, utilizing a lightweight Large Language Model (LLM) retriever to selectively extract relevant lines from accessibility tree (AxTree) observations, thereby significantly reducing computational costs and improving efficiency in processing extensive web page data.
  • The proposed method allows for more effective reasoning by removing irrelevant context, which also decreases the risk of security vulnerabilities, such as prompt injection attacks, by filtering out potentially harmful content before processing.
  • Experimental results demonstrate that FocusAgent maintains performance levels comparable to traditional approaches while achieving over 50% reduction in observation size, indicating its practical applicability in real-world scenarios.
  • FocusAgent's capability to effectively mitigate the success rates of prompt-injection attacks shows its dual role in enhancing both operational performance and security, paving the way for safer web agent deployment.
  • The release of open-source code for FocusAgent provides a tool for further research and development in observation pruning techniques for web agents and enhances community engagement in improving agent performance and safety.

💡 Why This Paper Matters

This paper is significant as it addresses critical challenges in web agent development, specifically the need for efficient and secure processing of large amounts of contextual data. By introducing FocusAgent, it presents a novel pruning mechanism that not only improves performance but also bolsters security against emerging threats such as prompt injections. These dual benefits are crucial for advancing the robustness of AI agents in both research and real-world applications.

🎯 Why It's Interesting for AI Security Researchers

For AI security researchers, this paper is of particular interest as it tackles the pressing issue of security vulnerabilities in web agents, especially those arising from prompt injection attacks. The focus on integrating security measures within the agent's operational framework—rather than as an afterthought—demonstrates a proactive approach to building resilient AI systems. Additionally, the insights gained from the FocusAgent method could inspire further exploration of LLMs in enhancing the safety and reliability of AI applications across various domains.

📚 Read the Full Paper