Philipp Zimmermann
← Back to Newsletter

Paper Library

Collection of AI Security research papers

Showing 770 papers total

April 14 - April 20, 2025

4 papers

Breaking the Prompt Wall (I): A Real-World Case Study of Attacking ChatGPT via Lightweight Prompt Injection

Xiangyu Chang, Guang Dai, Hao Di, Haishan Ye
2025-04-20
red teaming
2504.16125v1

DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks

Yupei Liu, Yuqi Jia, Jinyuan Jia, Dawn Song, Neil Zhenqiang Gong
2025-04-15
red teaming
2504.11358v4

DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks

Yupei Liu, Yuqi Jia, Jinyuan Jia, Dawn Song, Neil Zhenqiang Gong
2025-04-15
red teaming
2504.11358v3

Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systems

William Hackett, Lewis Birch, Stefan Trawicki, Neeraj Suri, Peter Garraghan
2025-04-15
red teaming
2504.11168v3

April 07 - April 13, 2025

1 paper

Defense against Prompt Injection Attacks via Mixture of Encodings

Ruiyi Zhang, David Sullivan, Kyle Jackson, Pengtao Xie, Mei Chen
2025-04-10
2504.07467v1

March 24 - March 30, 2025

1 paper

Defeating Prompt Injections by Design

Edoardo Debenedetti, Ilia Shumailov, Tianqi Fan, Jamie Hayes, Nicholas Carlini, Daniel Fabian, Christoph Kern, Chongyang Shi, Andreas Terzis, Florian Tramèr
2025-03-24
2503.18813v2

March 10 - March 16, 2025

1 paper

Prompt Injection Detection and Mitigation via AI Multi-Agent NLP Frameworks

Diego Gosmar, Deborah A. Dahl, Dario Gosmar
2025-03-14
red teaming
2503.11517v1

February 17 - February 23, 2025

2 papers

Can Indirect Prompt Injection Attacks Be Detected and Removed?

Yulin Chen, Haoran Li, Yuan Sui, Yufei He, Yue Liu, Yangqiu Song, Bryan Hooi
2025-02-23
red teaming
2502.16580v5

UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models

Huawei Lin, Yingjie Lao, Tong Geng, Tan Yu, Weijie Zhao
2025-02-18
red teaming
2502.13141v1

January 20 - January 26, 2025

1 paper

PromptShield: Deployable Detection for Prompt Injection Attacks

Dennis Jacob, Hend Alzahrani, Zhanhao Hu, Basel Alomair, David Wagner
2025-01-25
2501.15145v2

December 02 - December 08, 2024

1 paper

Trust No AI: Prompt Injection Along The CIA Security Triad

Johann Rehberger
2024-12-08
red teaming
2412.06090v1

October 28 - November 03, 2024

7 papers

Defense Against Prompt Injection Attack by Leveraging Attack Techniques

Yulin Chen, Haoran Li, Zihao Zheng, Yangqiu Song, Dekai Wu, Bryan Hooi
2024-11-01
red teaming
2411.00459v6

Attention Tracker: Detecting Prompt Injection Attacks in LLMs

Kuo-Han Hung, Ching-Yun Ko, Ambrish Rawat, I-Hsin Chung, Winston H. Hsu, Pin-Yu Chen
2024-11-01
red teaming
2411.00348v2

InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models

Hao Li, Xiaogeng Liu
2024-10-30
2410.22770v3

Embedding-based classifiers can detect prompt injection attacks

Md. Ahsan Ayub, Subhabrata Majumdar
2024-10-29
red teaming
2410.22284v1

Systematically Analyzing Prompt Injection Vulnerabilities in Diverse LLM Architectures

Victoria Benjamin, Emily Braca, Israel Carter, Hafsa Kanchwala, Nava Khojasteh, Charly Landow, Yi Luo, Caroline Ma, Anna Magarelli, Rachel Mirin, Avery Moyer, Kayla Simpson, Amelia Skawinski, Thomas Heverin
2024-10-28
red teaming
2410.23308v1

Palisade -- Prompt Injection Detection Framework

Sahasra Kokkula, Somanathan R, Nandavardhan R, Aashishkumar, G Divya
2024-10-28
2410.21146v1

Fine-tuned Large Language Models (LLMs): Improved Prompt Injection Attacks Detection

Md Abdur Rahman, Fan Wu, Alfredo Cuzzocrea, Sheikh Iqbal Ahamed
2024-10-28
red teaming
2410.21337v2

October 14 - October 20, 2024

2 papers

Enhancing Prompt Injection Attacks to LLMs via Poisoning Alignment

Zedian Shao, Hongbin Liu, Jaden Mu, Neil Zhenqiang Gong
2024-10-18
red teaming
2410.14827v3

SPIN: Self-Supervised Prompt INjection

Leon Zhou, Junfeng Yang, Chengzhi Mao
2024-10-17
red teaming
2410.13236v1

October 07 - October 13, 2024

3 papers

Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems

Donghyun Lee, Mo Tiwari
2024-10-09
red teaming
2410.07283v1

SecAlign: Defending Against Prompt Injection with Preference Optimization

Sizhe Chen, Arman Zharmagambetov, Saeed Mahloujifar, Kamalika Chaudhuri, David Wagner, Chuan Guo
2024-10-07
red teaming
2410.05451v3

A test suite of prompt injection attacks for LLM-based machine translation

Antonio Valerio Miceli-Barone, Zhifan Sun
2024-10-07
red teaming
2410.05047v1

September 23 - September 29, 2024

1 paper

GenTel-Safe: A Unified Benchmark and Shielding Framework for Defending Against Prompt Injection Attacks

Rongchang Li, Minjie Chen, Chang Hu, Han Chen, Wenpeng Xing, Meng Han
2024-09-29
red teaming
2409.19521v1