← Back to Library

Jailbreaking Large Language Models Through Content Concretization

Authors: Johan Wahréus, Ahmed Hussain, Panos Papadimitratos

Published: 2025-09-16

arXiv ID: 2509.12937v1

Added to Library: 2025-09-17 04:00 UTC

Red Teaming

📄 Abstract

Large Language Models (LLMs) are increasingly deployed for task automation and content generation, yet their safety mechanisms remain vulnerable to circumvention through different jailbreaking techniques. In this paper, we introduce \textit{Content Concretization} (CC), a novel jailbreaking technique that iteratively transforms abstract malicious requests into concrete, executable implementations. CC is a two-stage process: first, generating initial LLM responses using lower-tier, less constrained safety filters models, then refining them through higher-tier models that process both the preliminary output and original prompt. We evaluate our technique using 350 cybersecurity-specific prompts, demonstrating substantial improvements in jailbreak Success Rates (SRs), increasing from 7\% (no refinements) to 62\% after three refinement iterations, while maintaining a cost of 7.5\textcent~per prompt. Comparative A/B testing across nine different LLM evaluators confirms that outputs from additional refinement steps are consistently rated as more malicious and technically superior. Moreover, manual code analysis reveals that generated outputs execute with minimal modification, although optimal deployment typically requires target-specific fine-tuning. With eventual improved harmful code generation, these results highlight critical vulnerabilities in current LLM safety frameworks.

🔍 Key Points

  • Introduction of Content Concretization (CC) as a new jailbreaking technique for LLMs, enhancing the ability to transform abstract requests into executable code.
  • Empirical evaluations showed a substantial increase in jailbreak success rates, from 7% without refinements to 62% after three refinement iterations.
  • A/B testing confirmed that additional refinement steps yield higher-quality, more malicious outputs, with consistent preferences across multiple LLM evaluators.
  • Cost analysis indicated that CC remains economically viable, with costs averaging only 7.5 cents per prompt, despite increased token consumption with refinements.
  • The findings highlight critical vulnerabilities in current LLM safety mechanisms, revealing how iterative processes can be exploited for generating harmful code.

💡 Why This Paper Matters

This paper presents a significant advancement in understanding the security vulnerabilities of large language models by introducing a novel jailbreaking technique, Content Concretization (CC). It effectively demonstrates how these models can be manipulated to produce potentially harmful outputs while maintaining cost-effectiveness. The implications of CC are profound, as they not only raise concerns about LLM safety measures but also underscore the necessity for enhanced security protocols to counteract such exploitation methods.

🎯 Why It's Interesting for AI Security Researchers

The research is vital for AI security researchers as it uncovers the weaknesses in the safety mechanisms of LLMs, providing insights into how adversaries can bypass existing safeguards. The novel methodologies and empirical results prompt further investigation into developing robust countermeasures, making it a critical contribution to the ongoing discourse on AI safety and the ethical deployment of language models.

📚 Read the Full Paper