← Back to Library

Reimagining Safety Alignment with An Image

Authors: Yifan Xia, Guorui Chen, Wenqian Yu, Zhijiang Li, Philip Torr, Jindong Gu

Published: 2025-11-01

arXiv ID: 2511.00509v1

Added to Library: 2025-11-05 05:01 UTC

📄 Abstract

Large language models (LLMs) excel in diverse applications but face dual challenges: generating harmful content under jailbreak attacks and over-refusal of benign queries due to rigid safety mechanisms. These issues are further complicated by the need to accommodate different value systems and precisely align with given safety preferences. Moreover, traditional methods like SFT and RLHF lack this capability due to their costly parameter tuning requirements and inability to support multiple value systems within a single model. These problems are more obvious in multimodal large language models (MLLMs), especially in terms of heightened over-refusal in cross-modal tasks and new security risks arising from expanded attack surfaces. We propose Magic Image, an optimization-driven visual prompt framework that enhances security while reducing over-refusal. By optimizing image prompts using harmful/benign samples, our method enables a single model to adapt to different value systems and better align with given safety preferences without parameter updates. Experiments demonstrate improved safety-effectiveness balance across diverse datasets while preserving model performance, offering a practical solution for deployable MLLM safety alignment.

🔍 Key Points

  • Introduction of Intent Shift Attack (ISA) as a new method for jailbreaking LLMs through minimal linguistic modifications, significantly changing the perception of intent.
  • Development of a comprehensive taxonomy of intent transformations to categorize various strategies for obfuscating harmful intents into benign-seeming queries.
  • Demonstration through experiments that ISA achieves over 70% improvement in attack success rate compared to traditional harmful prompts, with nearly 100% success when models are fine-tuned exclusively on benign ISA data.
  • Evaluation of existing defense mechanisms reveals their inadequacies against ISA, highlighting critical gaps in current safety strategies for LLMs.
  • Exploration of potential defenses shows a trade-off between safety and utility, emphasizing the need for more sophisticated safety mechanisms that can accurately assess intent.

💡 Why This Paper Matters

This paper is highly relevant as it sheds light on the vulnerabilities of large language models (LLMs) to sophisticated jailbreaking attacks via the Intent Shift Attack. The work not only introduces a novel method that is both effective and subtle but also exposes fundamental weaknesses in LLMs' ability to infer user intent, raising important questions about the integrity of safety mechanisms in AI systems. Moreover, by challenging existing defenses and proposing enhancements, the paper sets the stage for future research aimed at improving AI safety, thus contributing to the broader discourse on responsible AI deployment.

🎯 Why It's Interesting for AI Security Researchers

AI security researchers will find this paper particularly significant as it highlights new attack vectors against LLMs that bypass traditional defenses, thereby pushing the boundaries of current AI security practices. Understanding ISA's methodology and its implications for intent misinterpretation will be crucial for developing more resilient models and refining defensive strategies. Furthermore, the findings prompt deeper investigation into the intersection of AI safety and human-like interpretative capabilities in language models, a topic of increasing significance in the field.

📚 Read the Full Paper

📄 View on arXiv 📎 Download PDF