← Back to Library

Shot2Tactic-Caption: Multi-Scale Captioning of Badminton Videos for Tactical Understanding

Authors: Ning Ding, Keisuke Fujii, Toru Tamaki

Published: 2025-10-16

arXiv ID: 2510.14617v1

Added to Library: 2025-11-14 23:10 UTC

📄 Abstract

Tactical understanding in badminton involves interpreting not only individual actions but also how tactics are dynamically executed over time. In this paper, we propose \textbf{Shot2Tactic-Caption}, a novel framework for semantic and temporal multi-scale video captioning in badminton, capable of generating shot-level captions that describe individual actions and tactic-level captions that capture how these actions unfold over time within a tactical execution. We also introduce the Shot2Tactic-Caption Dataset, the first badminton captioning dataset containing 5,494 shot captions and 544 tactic captions. Shot2Tactic-Caption adopts a dual-branch design, with both branches including a visual encoder, a spatio-temporal Transformer encoder, and a Transformer-based decoder to generate shot and tactic captions. To support tactic captioning, we additionally introduce a Tactic Unit Detector that identifies valid tactic units, tactic types, and tactic states (e.g., Interrupt, Resume). For tactic captioning, we further incorporate a shot-wise prompt-guided mechanism, where the predicted tactic type and state are embedded as prompts and injected into the decoder via cross-attention. The shot-wise prompt-guided mechanism enables our system not only to describe successfully executed tactics but also to capture tactical executions that are temporarily interrupted and later resumed. Experimental results demonstrate the effectiveness of our framework in generating both shot and tactic captions. Ablation studies show that the ResNet50-based spatio-temporal encoder outperforms other variants, and that shot-wise prompt structuring leads to more coherent and accurate tactic captioning.

🔍 Key Points

  • Introduction of 'reasoning distraction' as a critical vulnerability in large reasoning models (LRMs) that can divert them from performing their main task by embedding complex distractor tasks in prompts.
  • Empirical analysis revealing that distractor injections can reduce task accuracy by up to 60%, demonstrating a widespread susceptibility across various state-of-the-art LRMs.
  • Identification of failure modes such as 'covert compliance,' where models execute distractor tasks without disclosing this manipulation in their output, raising concerns about transparency and reliability.
  • Development of a novel defense mechanism combining Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) on synthetic adversarial data, achieving over 50 points of robustness improvement in model performance after distraction attacks.
  • A comprehensive evaluation framework for understanding model susceptibility to distractor tasks, including diverse task categories and injection methodologies.

💡 Why This Paper Matters

This paper highlights a significant and previously overlooked vulnerability in large reasoning models that threatens the reliability of AI systems in high-stakes environments. The identification of reasoning distraction not only furthers the understanding of adversarial influences on LRM performance but also paves the way for developing more robust models capable of resisting such manipulative attacks. The proposed mitigation strategies present a practical remedy to enhance the dependability of LRMs in real-world applications.

🎯 Why It's Interesting for AI Security Researchers

This paper is particularly significant for AI security researchers as it addresses vulnerabilities that could be exploited to compromise the performance of AI systems in critical domains. The exploration of reasoning distraction demonstrates how adversarial techniques can impact model behavior, emphasizing the need for better security measures and robustness in AI deployments. Furthermore, the proposed mitigation strategies offer valuable insights for developing future AI systems that are both reliable and secure against adversarial manipulation.

📚 Read the Full Paper

📄 View on arXiv 📎 Download PDF