← Back to Library

CoCoTen: Detecting Adversarial Inputs to Large Language Models through Latent Space Features of Contextual Co-occurrence Tensors

Authors: Sri Durga Sai Sowmya Kadali, Evangelos E. Papalexakis

Published: 2025-08-05

arXiv ID: 2508.02997v1

Added to Library: 2025-08-14 23:04 UTC

Red Teaming

📄 Abstract

The widespread use of Large Language Models (LLMs) in many applications marks a significant advance in research and practice. However, their complexity and hard-to-understand nature make them vulnerable to attacks, especially jailbreaks designed to produce harmful responses. To counter these threats, developing strong detection methods is essential for the safe and reliable use of LLMs. This paper studies this detection problem using the Contextual Co-occurrence Matrix, a structure recognized for its efficacy in data-scarce environments. We propose a novel method leveraging the latent space characteristics of Contextual Co-occurrence Matrices and Tensors for the effective identification of adversarial and jailbreak prompts. Our evaluations show that this approach achieves a notable F1 score of 0.83 using only 0.5% of labeled prompts, which is a 96.6% improvement over baselines. This result highlights the strength of our learned patterns, especially when labeled data is scarce. Our method is also significantly faster, speedup ranging from 2.3 to 128.4 times compared to the baseline models. To support future research and reproducibility, we have made our implementation publicly available.

🔍 Key Points

  • Introduction of CoCoTen, a novel method based on latent space features derived from Contextual Co-occurrence Matrices and Tensors, for detecting adversarial and jailbreak inputs in LLMs.
  • The method demonstrates high effectiveness, achieving an F1 score of 0.83 with only 0.5% of labeled data, signifying a 96.6% improvement over existing baseline models.
  • CoCoTen exhibits significant computational efficiency, with processing speedups ranging from 2.3 to 128.4 times compared to baseline methods, emphasizing its practicality for real-world applications.
  • The method's performance indicates robustness in label-scarce environments, providing a promising approach for enhancing LLM security against evolving adversarial techniques.
  • The paper supports future research and reproducibility by publicly releasing the implementation of CoCoTen.

💡 Why This Paper Matters

The paper presents a significant advancement in the field of AI security, specifically addressing the vulnerabilities of Large Language Models (LLMs) to adversarial attacks. By leveraging innovative techniques involving Contextual Co-occurrence Matrices and Tensor decomposition, CoCoTen demonstrates superior performance even in data-scarce settings, providing a reliable solution for detecting harmful inputs and increasing the trustworthiness of AI systems. This work is particularly relevant given the growing concerns about the misuse of LLMs and sets a foundation for future exploration in this area.

🎯 Why It's Interesting for AI Security Researchers

This paper would be of particular interest to AI security researchers as it addresses a critical challenge in the safety and reliability of Large Language Models by proposing a new detection method for adversarial prompts. With the growing prevalence of jailbreaking and adversarial attacks on AI systems, understanding and mitigating these risks is vital. The effectiveness of CoCoTen in data-scarce environments and its substantial efficiency improvements over existing models make it a valuable contribution to the ongoing discourse on AI safety.

📚 Read the Full Paper

📄 View on arXiv 📎 Download PDF