← Back to Library

RelightMaster: Precise Video Relighting with Multi-plane Light Images

Authors: Weikang Bian, Xiaoyu Shi, Zhaoyang Huang, Jianhong Bai, Qinghe Wang, Xintao Wang, Pengfei Wan, Kun Gai, Hongsheng Li

Published: 2025-11-09

arXiv ID: 2511.06271v1

Added to Library: 2025-11-14 23:03 UTC

📄 Abstract

Recent advances in diffusion models enable high-quality video generation and editing, but precise relighting with consistent video contents, which is critical for shaping scene atmosphere and viewer attention, remains unexplored. Mainstream text-to-video (T2V) models lack fine-grained lighting control due to text's inherent limitation in describing lighting details and insufficient pre-training on lighting-related prompts. Additionally, constructing high-quality relighting training data is challenging, as real-world controllable lighting data is scarce. To address these issues, we propose RelightMaster, a novel framework for accurate and controllable video relighting. First, we build RelightVideo, the first dataset with identical dynamic content under varying precise lighting conditions based on the Unreal Engine. Then, we introduce Multi-plane Light Image (MPLI), a novel visual prompt inspired by Multi-Plane Image (MPI). MPLI models lighting via K depth-aligned planes, representing 3D light source positions, intensities, and colors while supporting multi-source scenarios and generalizing to unseen light setups. Third, we design a Light Image Adapter that seamlessly injects MPLI into pre-trained Video Diffusion Transformers (DiT): it compresses MPLI via a pre-trained Video VAE and injects latent light features into DiT blocks, leveraging the base model's generative prior without catastrophic forgetting. Experiments show that RelightMaster generates physically plausible lighting and shadows and preserves original scene content. Demos are available at https://wkbian.github.io/Projects/RelightMaster/.

🔍 Key Points

  • Introduction of a novel black-box defense framework (KG-DF) that leverages Knowledge Graphs (KG) to enhance security against jailbreak attacks on large language models (LLMs).
  • Development of an extensible semantic parsing module to improve the keyword extraction process, which addresses challenges posed by evolving attack strategies.
  • The framework integrates safety-related and general knowledge into LLM responses, allowing the model to maintain both security and generality during operation.
  • Experimental evaluations demonstrate KG-DF achieves near-zero attack success rates (ASR) while maintaining high generality in both open-source and closed-source LLMs, significantly outperforming existing defense methods.
  • The proposed defense setup also improves the overall response quality of LLMs in general question-and-answer scenarios, indicating practical applicability for real-world usage.

💡 Why This Paper Matters

This paper addresses the critical issue of jailbreak attacks in large language models by proposing an innovative defense framework that utilizes Knowledge Graphs. Its dual focus on enhancing model security while preserving generality is particularly relevant in today’s landscape where LLMs are increasingly at risk from adversarial inputs. The findings not only indicate a significant improvement in defense strategies but also contribute to the broader discourse on AI safety and security measures necessary for the deployment of these models in sensitive applications.

🎯 Why It's Interesting for AI Security Researchers

This paper is highly relevant to AI security researchers as it offers a comprehensive approach to defending against a significant vulnerability in LLMs—jailbreak attacks. By employing Knowledge Graphs and advanced semantic processing techniques, the authors present a solution that could inspire further innovations in defensive mechanisms against adversarial attacks. Additionally, the empirical success of the KG-DF framework in enhancing both security performance and response quality presents valuable data and insights for future research in the intersection of AI safety and language model deployment.

📚 Read the Full Paper

📄 View on arXiv 📎 Download PDF