← Back to Library

The Blessing and Curse of Dimensionality in Safety Alignment

Authors: Rachel S. Y. Teo, Laziz U. Abdullaev, Tan M. Nguyen

Published: 2025-07-27

arXiv ID: 2507.20333v1

Added to Library: 2025-07-29 04:00 UTC

Red Teaming

📄 Abstract

The focus on safety alignment in large language models (LLMs) has increased significantly due to their widespread adoption across different domains. The scale of LLMs play a contributing role in their success, and the growth in parameter count follows larger hidden dimensions. In this paper, we hypothesize that while the increase in dimensions has been a key advantage, it may lead to emergent problems as well. These problems emerge as the linear structures in the activation space can be exploited, in the form of activation engineering, to circumvent its safety alignment. Through detailed visualizations of linear subspaces associated with different concepts, such as safety, across various model scales, we show that the curse of high-dimensional representations uniquely impacts LLMs. Further substantiating our claim, we demonstrate that projecting the representations of the model onto a lower dimensional subspace can preserve sufficient information for alignment while avoiding those linear structures. Empirical results confirm that such dimensional reduction significantly reduces susceptibility to jailbreaking through representation engineering. Building on our empirical validations, we provide theoretical insights into these linear jailbreaking methods relative to a model's hidden dimensions. Broadly speaking, our work posits that the high dimensions of a model's internal representations can be both a blessing and a curse in safety alignment.

🔍 Key Points

  • The paper analyzes the impact of high dimensionality in large language models (LLMs) on safety alignment, suggesting that while increased dimensions enhance model capabilities, they also introduce vulnerabilities such as susceptibility to jailbreaking through activation engineering.
  • The authors present empirical evidence of the 'curse of dimensionality', postulating that linear structures in the activation space can be exploited to bypass safety measures during model deployment.
  • Two novel fine-tuning methods are proposed: the Fast Johnson–Lindenstrauss Transform (FJLT) and a Bottleneck method, both aimed at projecting hidden representations onto lower-dimensional subspaces to improve resilience against representation-engineering attacks without sacrificing safety alignment.
  • Experiments conducted confirm that both methods significantly reduce the effectiveness of jailbreaking attacks while preserving sufficient information for maintaining model utility and safety.
  • The work emphasizes the need for a deeper understanding of how dimensionality relates to safety alignment, calling for further research to establish more effective, semantics-based projection methods.

💡 Why This Paper Matters

This paper is significant in the landscape of AI research as it addresses the dual nature of high-dimensional representations in LLMs, elucidating both their advantages and the emergent risks they pose concerning safety alignment. By proposing practical solutions to mitigate these risks, the authors contribute to the ongoing discourse on responsible AI deployment.

🎯 Why It's Interesting for AI Security Researchers

AI security researchers would find this paper highly relevant as it explores critical vulnerabilities in large language models that could be exploited in real-world applications. The insights gained about jailbreaking mechanisms and the proposed defenses not only advance academic knowledge but also provide practical methodologies that can be utilized to enhance the security of AI systems against adversarial attacks.

📚 Read the Full Paper

📄 View on arXiv 📎 Download PDF