← Back to Library

AnchorDrive: LLM Scenario Rollout with Anchor-Guided Diffusion Regeneration for Safety-Critical Scenario Generation

Authors: Zhulin Jiang, Zetao Li, Cheng Wang, Ziwen Wang, Chen Xiong

Published: 2026-03-03

arXiv ID: 2603.02542v1

Added to Library: 2026-03-04 03:00 UTC

Safety

📄 Abstract

Autonomous driving systems require comprehensive evaluation in safety-critical scenarios to ensure safety and robustness. However, such scenarios are rare and difficult to collect from real-world driving data, necessitating simulation-based synthesis. Yet, existing methods often exhibit limitations in both controllability and realism. From a capability perspective, LLMs excel at controllable generation guided by natural language instructions, while diffusion models are better suited for producing trajectories consistent with realistic driving distributions. Leveraging their complementary strengths, we propose AnchorDrive, a two-stage safety-critical scenario generation framework. In the first stage, we deploy an LLM as a driver agent within a closed-loop simulation, which reasons and iteratively outputs control commands under natural language constraints; a plan assessor reviews these commands and provides corrective feedback, enabling semantically controllable scenario generation. In the second stage, the LLM extracts key anchor points from the first-stage trajectories as guidance objectives, which jointly with other guidance terms steer the diffusion model to regenerate complete trajectories with improved realism while preserving user-specified intent. Experiments on the highD dataset demonstrate that AnchorDrive achieves superior overall performance in criticality, realism, and controllability, validating its effectiveness for generating controllable and realistic safety-critical scenarios.

🔍 Key Points

  • Proposed AnchorDrive framework integrates LLMs and diffusion models for generating safety-critical driving scenarios, achieving both controllability and realism.
  • Utilizes a two-stage approach: the first stage generates scenarios through an LLM-based driver agent, while the second stage refines trajectories using a diffusion model anchored by key points derived from the first stage.
  • Experiments on the highD dataset show AnchorDrive significantly outperforms existing methods in criticality, realism, and semantic controllability metrics, demonstrating its effectiveness in generating complex driving scenarios.
  • Introduces a plan assessor mechanism that provides feedback to the LLM-based driver agent to ensure realistic behavior and adherence to user instructions during scenario generation.
  • First-of-its-kind integration of LLMs for textual instruction processing with advanced trajectory generation using diffusion models, pushing the boundaries of scenario synthesis in autonomous driving.

💡 Why This Paper Matters

The AnchorDrive framework represents a significant advancement in the field of safety-critical scenario generation for autonomous vehicles. By combining the strengths of language modeling and diffusion processes, the study showcases a novel approach that balances user-defined controllability with realistic driving patterns. The demonstrated effectiveness through empirical evaluation establishes its relevance in improving safety evaluations for autonomous driving systems.

🎯 Why It's Interesting for AI Security Researchers

This paper is particularly relevant to AI security researchers as it tackles the critical challenge of generating safe and effective testing scenarios for autonomous driving systems. By enhancing the realism and controllability of generated scenarios, the framework can help identify vulnerabilities and edge cases in self-driving algorithms, thereby contributing to the development of more robust and secure AI-driven transportation systems.

📚 Read the Full Paper