← Back to Library

WaveSP-Net: Learnable Wavelet-Domain Sparse Prompt Tuning for Speech Deepfake Detection

Authors: Xi Xuan, Xuechen Liu, Wenxin Zhang, Yi-Cheng Lin, Xiaojian Lin, Tomi Kinnunen

Published: 2025-10-06

arXiv ID: 2510.05305v1

Added to Library: 2025-11-17 01:01 UTC

📄 Abstract

Modern front-end design for speech deepfake detection relies on full fine-tuning of large pre-trained models like XLSR. However, this approach is not parameter-efficient and may lead to suboptimal generalization to realistic, in-the-wild data types. To address these limitations, we introduce a new family of parameter-efficient front-ends that fuse prompt-tuning with classical signal processing transforms. These include FourierPT-XLSR, which uses the Fourier Transform, and two variants based on the Wavelet Transform: WSPT-XLSR and Partial-WSPT-XLSR. We further propose WaveSP-Net, a novel architecture combining a Partial-WSPT-XLSR front-end and a bidirectional Mamba-based back-end. This design injects multi-resolution features into the prompt embeddings, which enhances the localization of subtle synthetic artifacts without altering the frozen XLSR parameters. Experimental results demonstrate that WaveSP-Net outperforms several state-of-the-art models on two new and challenging benchmarks, Deepfake-Eval-2024 and SpoofCeleb, with low trainable parameters and notable performance gains. The code and models are available at https://github.com/xxuan-acoustics/WaveSP-Net.

🔍 Key Points

  • Introduction of Adversarial Reinforcement Learning for Agent Safety (ARLAS), a framework that co-trains an attacker and agent to enhance LLM agent safety against indirect prompt injections.
  • Utilization of a population-based learning strategy to ensure the agent learns robust defenses against a diverse set of attacks generated by the attacker model.
  • Validation of ARLAS on BrowserGym and AgentDojo benchmarks, demonstrating a significant reduction in attack success rates and improved task completion rates compared to baseline models.
  • ARLAS reduces reliance on manually crafted attack patterns by employing an autonomous adversarial training method, leading to more varied and challenging attacks over time.
  • Analysis of the sentence embeddings of generated attacks confirms that ARLAS produces increasingly diverse prompt injections throughout training.

💡 Why This Paper Matters

This paper presents a significant advancement in the safety and security of Large Language Model (LLM) agents through a novel adversarial reinforcement learning framework called ARLAS. By co-training an agent with an autonomous attacker, the framework addresses the critical challenge of indirect prompt injections, ultimately leading to more robust and capable agents that can maintain performance while minimizing the risk of compromise. The innovative approach not only automates the generation of diverse attack patterns but also ensures that agents can adapt to evolving threats, making it a crucial step toward enhancing LLM deployment in real-world scenarios.

🎯 Why It's Interesting for AI Security Researchers

This paper is highly relevant to AI security researchers as it tackles the pressing issue of LLM vulnerabilities, specifically regarding indirect prompt injections that can lead to critical failures and security breaches. The introduction of ARLAS offers a fresh perspective on creating resilient AI systems through adversarial training, presenting potentially impactful methodologies for improving the security posture of AI agents in various applications. Additionally, the framework's potential for reducing manual effort in crafting attack strategies aligns with significant trends towards automating security assessment and enhancing the robustness of AI-driven systems.

📚 Read the Full Paper

📄 View on arXiv 📎 Download PDF