← Back to Library

VFXMaster: Unlocking Dynamic Visual Effect Generation via In-Context Learning

Authors: Baolu Li, Yiming Zhang, Qinghe Wang, Liqian Ma, Xiaoyu Shi, Xintao Wang, Pengfei Wan, Zhenfei Yin, Yunzhi Zhuge, Huchuan Lu, Xu Jia

Published: 2025-10-29

arXiv ID: 2510.25772v1

Added to Library: 2025-11-14 23:06 UTC

📄 Abstract

Visual effects (VFX) are crucial to the expressive power of digital media, yet their creation remains a major challenge for generative AI. Prevailing methods often rely on the one-LoRA-per-effect paradigm, which is resource-intensive and fundamentally incapable of generalizing to unseen effects, thus limiting scalability and creation. To address this challenge, we introduce VFXMaster, the first unified, reference-based framework for VFX video generation. It recasts effect generation as an in-context learning task, enabling it to reproduce diverse dynamic effects from a reference video onto target content. In addition, it demonstrates remarkable generalization to unseen effect categories. Specifically, we design an in-context conditioning strategy that prompts the model with a reference example. An in-context attention mask is designed to precisely decouple and inject the essential effect attributes, allowing a single unified model to master the effect imitation without information leakage. In addition, we propose an efficient one-shot effect adaptation mechanism to boost generalization capability on tough unseen effects from a single user-provided video rapidly. Extensive experiments demonstrate that our method effectively imitates various categories of effect information and exhibits outstanding generalization to out-of-domain effects. To foster future research, we will release our code, models, and a comprehensive dataset to the community.

🔍 Key Points

  • Agent Skills are a newly introduced framework that allows agents to dynamically utilize knowledge based on markdown files, which presents a risk for prompt injections.
  • The authors demonstrate how malicious instructions can be hidden within Agent Skills to exfiltrate sensitive data, indicating a significant security vulnerability in such frameworks.
  • A key finding is the ability to bypass system-level guardrails by exploiting benign actions, which can be dangerous if users select options that allow actions without further prompts.
  • Experiments revealed that malicious scripts can be executed without user confirmation if the 'Don't ask again' feature is enabled, showcasing an exploitation pathway for attackers.
  • The paper emphasizes the importance of more robust defenses and alerts users against third-party Agent Skills that are not vetted for security.

💡 Why This Paper Matters

This paper is relevant as it exposes significant security vulnerabilities in the Agent Skills framework for LLMs, a critical aspect of ongoing developments in AI. By highlighting the ease with which malicious actions can be implemented and the potential consequences of such vulnerabilities, the paper serves as a call for improved security measures and oversight in AI applications that utilize similar architectures.

🎯 Why It's Interesting for AI Security Researchers

The paper would be of interest to AI security researchers as it uncovers a novel attack vector related to prompt injections, particularly in the context of continually learning models. The findings prompt further investigation into the security implications of dynamic knowledge integration in LLMs and underline the necessity for improved safeguarding mechanisms against even simple injections, which can have far-reaching impacts in practice.

📚 Read the Full Paper