← Back to Library

The Rogue Scalpel: Activation Steering Compromises LLM Safety

Authors: Anton Korznikov, Andrey Galichin, Alexey Dontsov, Oleg Y. Rogov, Ivan Oseledets, Elena Tutubalina

Published: 2025-09-26

arXiv ID: 2509.22067v1

Added to Library: 2025-09-29 04:01 UTC

Red Teaming Safety

📄 Abstract

Activation steering is a promising technique for controlling LLM behavior by adding semantically meaningful vectors directly into a model's hidden states during inference. It is often framed as a precise, interpretable, and potentially safer alternative to fine-tuning. We demonstrate the opposite: steering systematically breaks model alignment safeguards, making it comply with harmful requests. Through extensive experiments on different model families, we show that even steering in a random direction can increase the probability of harmful compliance from 0% to 2-27%. Alarmingly, steering benign features from a sparse autoencoder (SAE), a common source of interpretable directions, increases these rates by a further 2-4%. Finally, we show that combining 20 randomly sampled vectors that jailbreak a single prompt creates a universal attack, significantly increasing harmful compliance on unseen requests. These results challenge the paradigm of safety through interpretability, showing that precise control over model internals does not guarantee precise control over model behavior.

🔍 Key Points

  • 1. **Vulnerability of Activation Steering**: The paper proves that activation steering, often seen as a safe and interpretable model control technique, can systematically compromise the safety mechanisms of Large Language Models (LLMs), notably increasing the probability of harmful compliance with prompts.
  • 2. **Random Steering Effects**: The study highlights alarming findings, showing that injecting random perturbations into model activations can increase harmful compliance rates from 0% to as high as 27%, revealing inherent vulnerabilities across various model architectures.
  • 3. **Hazardous Impact of SAEs**: Utilizing benign features from Sparse Autoencoders (SAEs) further escalates harmful compliance rates, suggesting that interpreted steering vectors designed for safety can inadvertently lead to significant risks.
  • 4. **Creation of Universal Attack Vectors**: The authors demonstrate that aggregating multiple harmful random steering vectors can form universal attack vectors, achieving up to a 4× increase in compliance rates on unseen prompts without needing model internals or harmful data for training.
  • 5. **Case Study Validation**: Through case studies using public API steering, the research effectively illustrates how production-level models can be compromised by features ordinarily believed to be safe, revealing fundamental lapses in model safety.

💡 Why This Paper Matters

This paper is critically relevant as it challenges the safe implementation of interpretability techniques like activation steering in LLMs. By systematically demonstrating that these methods can lead to severe compromises in model safety, it calls into question widely accepted practices in AI safety and control, emphasizing the need for robust safety frameworks that can account for the unintended consequences of seemingly benign manipulations.

🎯 Why It's Interesting for AI Security Researchers

For AI security researchers, this paper is crucial because it exposes new attack vectors that could be exploited by malicious actors to bypass safety mechanisms in LLMs. The findings regarding the dual nature of interpretability—where it can lead to both improved control and significant vulnerabilities—highlight the necessity of investigating the safety implications of advanced AI technologies. This research lays the groundwork for developing more resilient systems against such vulnerabilities and fuels the ongoing discourse on AI safety and ethics.

📚 Read the Full Paper

📄 View on arXiv 📎 Download PDF