Abstract
Aerial mobile manipulation requires unmanned aerial vehicles (UAVs) to traverse heterogeneous environments such as indoor and outdoor domains, and connect navigation with subsequent physical interaction within the same trajectories. Existing aerial benchmarks are typically confined to single-environment settings, limiting evaluation of agents across continuous environmental transitions. When interaction is evaluated, agents are often initialized near the target, leaving the transition from navigation to interaction largely unassessed. This work introduces CrossFly, a benchmark for long-horizon interaction-oriented aerial mobility across heterogeneous environments. CrossFly contains 15K trajectories across more than 20 reconstructed scenes under a unified simulation and evaluation framework, covering Indoor, Outdoor, and Indoor↔Outdoor trajectories. It further introduces Interaction Readiness Rate (IRR) to assess whether terminal navigation states provide suitable conditions for downstream interaction without assuming a specific manipulation policy. Experiments with five representative aerial vision-language-action (VLA) agents and the proposed CrossFly-Agent show that Indoor↔Outdoor settings are more challenging than single-environment settings, with the highest success rate reaching 9.7% on unseen scenes. A downstream manipulation case study further validates the relevance of interaction-oriented arrival to subsequent physical interaction. We hope CrossFly helps advance aerial embodied intelligence toward unified mobility and physical interaction in heterogeneous environments.
Task Demonstrations
Outdoor
Benchmark Statistics
The dataset contains synchronized egocentric RGB observations, 6-DoF flight states, language instructions, waypoints, and subtask annotations. Cross-environment trajectories account for 51.2% of the dataset.
We train agents on trajectories from 18 scenes and evaluate them on two splits: test trajectories from those same 18 scenes (seen-scene evaluation), and trajectories from five additional scenes not used during training (unseen-scene evaluation).
Construction Pipeline
1. Prepare
Reconstruct scenes from multi-view RGB and point clouds, then annotate target objects, waypoints, and instructions.
2. Execute
Ego-Planner and PD tracking generate rollouts under UAV dynamics. Collision and waypoint checks reject infeasible trajectories.
3. Annotate
Waypoint-defined segments and visual context produce fine-grained, human-verified subtask supervision.
Benchmark Results
| Model | Indoor | Outdoor | Indoor↔Outdoor |
|---|---|---|---|
| π₀ | 10.8 | 13.5 | 2.4 |
| π₀-FAST | 13.2 | 15.6 | 3.7 |
| OpenFly-Agent | 10.6 | 13.4 | 7.2 |
| OpenVLA-OFT | 11.9 | 14.1 | 3.1 |
| GR00T-N1.7 | 13.8 | 16.2 | 4.2 |
| CrossFly-Agent | 17.3 | 18.5 | 9.7 |
| Model | Indoor | Outdoor | Indoor↔Outdoor |
|---|---|---|---|
| π₀ | 17.7 | 20.4 | 13.2 |
| π₀-FAST | 18.5 | 22.7 | 15.8 |
| OpenFly-Agent | 15.1 | 18.1 | 13.9 |
| OpenVLA-OFT | 17.2 | 19.5 | 14.9 |
| GR00T-N1.7 | 18.9 | 21.3 | 11.7 |
| CrossFly-Agent | 23.5 | 25.5 | 19.8 |
CrossFly-Agent achieves 17.3% SR indoors, 18.5% outdoors, and 9.7% across environments on unseen scenes. A controlled comparison matching trajectory length, subtask count, and instruction length also retains a substantial cross-environment performance gap.