Philipp Zimmermann
← Back to Newsletter

Paper Library

Collection of AI Security research papers

Showing 947 papers total

May 19 - May 25, 2025

1 paper

EVA: Red-Teaming GUI Agents via Evolving Indirect Prompt Injection

Yijie Lu, Tianjie Ju, Manman Zhao, Xinbei Ma, Yuan Guo, ZhuoSheng Zhang
2025-05-20
red teaming
2505.14289v1

May 12 - May 18, 2025

2 papers

CAPTURE: Context-Aware Prompt Injection Testing and Robustness Enhancement

Gauri Kholkar, Ratinder Ahuja
2025-05-18
red teaming
2505.12368v2

WebInject: Prompt Injection Attack to Web Agents

Xilong Wang, John Bloch, Zedian Shao, Yuepeng Hu, Shuyan Zhou, Neil Zhenqiang Gong
2025-05-16
red teaming
2505.11717v4

May 05 - May 11, 2025

1 paper

System Prompt Poisoning: Persistent Attacks on Large Language Models Beyond User Injection

Zongze Li, Jiawei Guo, Haipeng Cai
2025-05-10
red teaming
2505.06493v3

April 28 - May 04, 2025

3 papers

OET: Optimization-based prompt injection Evaluation Toolkit

Jinsheng Pan, Xiaogeng Liu, Chaowei Xiao
2025-05-01
red teaming
2505.00843v1

CachePrune: Neural-Based Attribution Defense Against Indirect Prompt Injection Attacks

Rui Wang, Junda Wu, Yu Xia, Tong Yu, Ruiyi Zhang, Ryan Rossi, Lina Yao, Julian McAuley
2025-04-29
red teaming
2504.21228v1

Prompt Injection Attack to Tool Selection in LLM Agents

Jiawen Shi, Zenghui Yuan, Guiyao Tie, Pan Zhou, Neil Zhenqiang Gong, Lichao Sun
2025-04-28
red teaming
2504.19793v3

April 21 - April 27, 2025

2 papers

Adversarial Attacks on LLM-as-a-Judge Systems: Insights from Prompt Injections

Narek Maloyan, Dmitry Namiot
2025-04-25
red teaming
2504.18333v1

WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks

Ivan Evtimov, Arman Zharmagambetov, Aaron Grattafiori, Chuan Guo, Kamalika Chaudhuri
2025-04-22
red teaming
2504.18575v3

April 14 - April 20, 2025

4 papers

Breaking the Prompt Wall (I): A Real-World Case Study of Attacking ChatGPT via Lightweight Prompt Injection

Xiangyu Chang, Guang Dai, Hao Di, Haishan Ye
2025-04-20
red teaming
2504.16125v1

DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks

Yupei Liu, Yuqi Jia, Jinyuan Jia, Dawn Song, Neil Zhenqiang Gong
2025-04-15
red teaming
2504.11358v4

DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks

Yupei Liu, Yuqi Jia, Jinyuan Jia, Dawn Song, Neil Zhenqiang Gong
2025-04-15
red teaming
2504.11358v3

Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systems

William Hackett, Lewis Birch, Stefan Trawicki, Neeraj Suri, Peter Garraghan
2025-04-15
red teaming
2504.11168v3

April 07 - April 13, 2025

1 paper

Defense against Prompt Injection Attacks via Mixture of Encodings

Ruiyi Zhang, David Sullivan, Kyle Jackson, Pengtao Xie, Mei Chen
2025-04-10
2504.07467v1

March 24 - March 30, 2025

1 paper

Defeating Prompt Injections by Design

Edoardo Debenedetti, Ilia Shumailov, Tianqi Fan, Jamie Hayes, Nicholas Carlini, Daniel Fabian, Christoph Kern, Chongyang Shi, Andreas Terzis, Florian Tramèr
2025-03-24
2503.18813v2

March 10 - March 16, 2025

1 paper

Prompt Injection Detection and Mitigation via AI Multi-Agent NLP Frameworks

Diego Gosmar, Deborah A. Dahl, Dario Gosmar
2025-03-14
red teaming
2503.11517v1

February 17 - February 23, 2025

2 papers

Can Indirect Prompt Injection Attacks Be Detected and Removed?

Yulin Chen, Haoran Li, Yuan Sui, Yufei He, Yue Liu, Yangqiu Song, Bryan Hooi
2025-02-23
red teaming
2502.16580v5

UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models

Huawei Lin, Yingjie Lao, Tong Geng, Tan Yu, Weijie Zhao
2025-02-18
red teaming
2502.13141v1

January 20 - January 26, 2025

1 paper

PromptShield: Deployable Detection for Prompt Injection Attacks

Dennis Jacob, Hend Alzahrani, Zhanhao Hu, Basel Alomair, David Wagner
2025-01-25
2501.15145v2

December 02 - December 08, 2024

1 paper

Trust No AI: Prompt Injection Along The CIA Security Triad

Johann Rehberger
2024-12-08
red teaming
2412.06090v1

October 28 - November 03, 2024

4 papers

Defense Against Prompt Injection Attack by Leveraging Attack Techniques

Yulin Chen, Haoran Li, Zihao Zheng, Yangqiu Song, Dekai Wu, Bryan Hooi
2024-11-01
red teaming
2411.00459v6

Attention Tracker: Detecting Prompt Injection Attacks in LLMs

Kuo-Han Hung, Ching-Yun Ko, Ambrish Rawat, I-Hsin Chung, Winston H. Hsu, Pin-Yu Chen
2024-11-01
red teaming
2411.00348v2

InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models

Hao Li, Xiaogeng Liu
2024-10-30
2410.22770v3

Embedding-based classifiers can detect prompt injection attacks

Md. Ahsan Ayub, Subhabrata Majumdar
2024-10-29
red teaming
2410.22284v1