← Back to Library

Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services

Authors: Fabrizio Dimino, Bhaskarjit Sarmah, Stefano Pasquali

Published: 2026-03-11

arXiv ID: 2603.10807v1

Added to Library: 2026-03-12 02:00 UTC

Red Teaming

📄 Abstract

The rapid adoption of large language models (LLMs) in financial services introduces new operational, regulatory, and security risks. Yet most red-teaming benchmarks remain domain-agnostic and fail to capture failure modes specific to regulated BFSI settings, where harmful behavior can be elicited through legally or professionally plausible framing. We propose a risk-aware evaluation framework for LLM security failures in Banking, Financial Services, and Insurance (BFSI), combining a domain-specific taxonomy of financial harms, an automated multi-round red-teaming pipeline, and an ensemble-based judging protocol. We introduce the Risk-Adjusted Harm Score (RAHS), a risk-sensitive metric that goes beyond success rates by quantifying the operational severity of disclosures, accounting for mitigation signals, and leveraging inter-judge agreement. Across diverse models, we find that higher decoding stochasticity and sustained adaptive interaction not only increase jailbreak success, but also drive systematic escalation toward more severe and operationally actionable financial disclosures. These results expose limitations of single-turn, domain-agnostic security evaluation and motivate risk-sensitive assessment under prolonged adversarial pressure for real-world BFSI deployment.

🔍 Key Points

  • Introduction of the Risk-Adjusted Harm Score (RAHS) to quantify operational severity of harmful disclosures in regulated BFSI settings.
  • Development of a domain-specific benchmark (FinRedTeamBench) to assess LLM security failures related to financial misconduct.
  • Implementation of an automated multi-turn red teaming framework that adapts adversarial prompts based on previous model responses, exposing dynamic vulnerabilities.
  • Findings that higher decoding stochasticity and sustained adaptive interaction lead to an increase in severe, actionable financial disclosures under adversarial settings.
  • Demonstration of limitations in existing binary success rate measures and the need for more nuanced, risk-sensitive evaluation methods.

💡 Why This Paper Matters

This paper proposes a novel framework for evaluating large language models (LLMs) in the context of financial services, where regulatory and operational risks are paramount. By introducing the Risk-Adjusted Harm Score and a domain-specific benchmark, the authors provide a sophisticated approach to red teaming that can expose critical vulnerabilities in LLMs when deployed in high-stakes environments. The findings underscore the importance of dynamic evaluation methods that accurately reflect risks in practical applications, making this work highly relevant for both researchers and practitioners in AI safety.

🎯 Why It's Interesting for AI Security Researchers

This paper is of significant interest to AI security researchers because it addresses the gaps in existing evaluation frameworks for LLMs, particularly in regulated sectors like finance. The introduction of a risk-aware scoring system tailored for specific harms associated with financial misconduct adds a layer of depth to security assessments that is often overlooked. The automated multi-turn red teaming methodology offers a blueprint for future research on adaptive adversarial strategies and their implications on model vulnerabilities, which are essential considerations for the safety of AI systems.

📚 Read the Full Paper

📄 View on arXiv 📎 Download PDF