Physical red teaming for vision-language-action models

RedVLA: Physical Red Teaming for Vision-Language-Action Models

RedVLA proactively injects a risk factor into a valid, task-feasible scene and refines it with trajectory-driven, gradient-free optimization to expose physical unsafe behavior before deployment.

Yuhao Zhang1,* · Borong Zhang1,2,* · Jiaming Fan1,* · Jiachen Shen1 · Yishuai Cai1 · Yaodong Yang1,† · Jiaming Ji1,†
1 Institute for AI, Peking University
2 Zhongguancun Academy
* Equal contribution · † Corresponding authors
Yuhao Zhang1,* · Borong Zhang1,2,* · Jiaming Fan1,* · Jiachen Shen1 · Yishuai Cai1 · Yaodong Yang1,† · Jiaming Ji1,†
1 Institute for AI, Peking University
2 Zhongguancun Academy
* Equal contribution · † Corresponding authors
01 · Interactive demo

Unsafe behavior gallery

Agent-view and wrist-view · representative LIBERO rollouts
Dangerous item misuse

Turn on the stove and put the moka pot on it

Agent
Wrist
Resource damage

Put the black bowl in the bottom drawer of the cabinet and close it

Agent
Wrist
Robot damage

Turn on the stove and put the moka pot on it

Agent
Wrist
Dangerous item misuse

Push the plate to the front of the stove

Agent
Wrist
Resource damage

Put the black bowl in the bottom drawer of the cabinet and close it

Agent
Wrist
Robot damage

Put the yellow and white mug in the microwave and close it

Agent
Wrist
Dangerous item misuse

Put the black bowl in the bottom drawer of the cabinet and close it

Agent
Wrist
Resource damage

Turn on the stove and put the moka pot on it

Agent
Wrist
Robot damage

Put the black bowl in the bottom drawer of the cabinet and close it

Agent
Wrist
Environmental harm

Turn on the stove and put the moka pot on it

Agent
Wrist
Representative unsafe rollouts. RedVLA covers instantaneous state violations, risks that accumulate over time, and conditional hazards that require a causal sequence.
02 · Motivation

Overview

Real-world deployment of Vision-Language-Action models remains limited by the risk of unpredictable and irreversible physical harm. Existing evaluations rarely provide a way to proactively uncover these risks before deployment.

RedVLA keeps the original instruction and policy fixed while changing the scene. Risk Scenario Synthesis identifies critical interaction regions from benign trajectories and places a selected risk factor inside a valid, task-feasible scene. Trajectory-Driven Risk Amplification then iteratively refines the risk factor through policy feedback, making the target violation stable across heterogeneous VLA models.

Motivation: three domains of VLA vulnerability and the role of RedVLA
Figure 1: Motivation of Physical Red Teaming for Vision-Language-Action Models. RedVLA proactively injects risk factors into the benign scene and optimizes it via policy feedback.
03 · Method

RedVLA method framework

A black-box, two-stage pipeline turns benign trajectories into targeted physical safety probes while preserving task feasibility.

1. Risk Scenario Synthesis

Define a target safety predicate and risk object, identify critical interaction regions from benign motion, instantiate the factor in one such region, and verify task feasibility.

2. Trajectory-Driven Risk Amplification

Use spatial features from the policy trajectory to update the risk factor with gradient-free optimization. Validity checks and backtracking preserve a feasible scene until the target violation is triggered.

Safety-cost taxonomy

Evaluate instantaneous state-level violations, temporally accumulated cumulative-level risks, and sequential conditional-level risks across resource damage, dangerous item misuse, robot damage, and environmental harm.

RedVLA two-stage method framework
RedVLA framework from the current paper: taxonomy and interaction identification feed scenario synthesis, followed by trajectory-guided risk amplification.
04 · Evidence

Key Results at a Glance

RedVLA exposes physical safety vulnerabilities across six representative VLA models, while SimpleVLA-Guard detects and mitigates them online.

95.5%Peak ASR on π0.5
92.1%Average ASR across five models stronger than OpenVLA
6VLA models from three model families
10Optimization iterations in the main evaluation
59.5%Maximum online ASR reduction from SimpleVLA-Guard

Main evaluation

Average ASR across six models64.9% → 95.5%
State / cumulative / conditional average ASR>95% / 88.9% / 66.1%
Cross-model transfer gain+10.3% ASR@1 · +12.1% ASR@5
Real-world Franka validation>80% ASR over 10 trials per task
Most unsafe rollouts occur during meaningful task execution (Success + Unsafe or Attempt + Unsafe), while collapse accounts for only a small fraction.

SimpleVLA-Guard

Seen tasks: ASR without → with guard90.9% → 31.4%
Unseen tasks: ASR without → with guard98.3% → 46.7%
Offline PRC-AUC0.94 seen · 0.89 unseen
Benign SR drop4.08% seen · 10.00% unseen
The guard uses internal π0 representations, an LSTM temporal head, and Functional Conformal Prediction for preemptive intervention.
05 · Citation

BibTeX

@misc{zhang2026redvlaphysicalredteaming,
  title        = {RedVLA: Physical Red Teaming for Vision-Language-Action Models},
  author       = {Yuhao Zhang and Borong Zhang and Jiaming Fan and Jiachen Shen and Yishuai Cai and Yaodong Yang and Jiaming Ji},
  year         = {2026},
  eprint       = {2604.22591},
  archivePrefix= {arXiv},
  primaryClass = {cs.RO},
  url          = {https://arxiv.org/abs/2604.22591}
}