NCS-LVTF: A Layered Vision–Temporal Fusion Framework for Continuous 6-DoF Tracking of Non-Cooperative Satellites in Close-Range Proximity
- We propose NCS-LVTF (Non-Cooperative Satellite Layered Vision–Temporal Fusion), a layered pipeline for continuous 6-DoF tracking of non-cooperative satellites in close-range proximity that combines multimodal vision, dynamics-based temporal priors, B-ESKF (temporally-biased ESKF) fusion, and learned-gated ICP refinement, substantially outperforming external baselines on the NCS-Prox Tier 3 test set.
- We develop a robust close-range visual module with OoV fine-tuning (visibility-masked heatmap supervision and distance/visibility sample reweighting) and depth-consistency gating (DCR) that rejects unreliable PnP under collapsing visibility and geometric inconsistency, yielding significantly lower close-range error than the standalone backbone and prior architecture.
- We introduce the NCS-Prox three-tier dataset to fill the public benchmark gap for sustained close-range non-cooperative tracking: Tier 1 provides a large-scale static visual corpus (~100k frames) for module training, Tier 2 a tumbling-dynamics corpus (1000 trajectories) for temporal-model validation, and Tier 3 comprises 70 continuous approach scenarios with synchronized RGB-D, point clouds, IMU, and trajectory ground truth for end-to-end evaluation. The full configuration achieves eR=1.72° and et=4.57 cm on the held-out Tier 3 test split.
Abstract
Continuous 6-DoF pose tracking is essential for non-cooperative satellite proximity operations and on-orbit servicing; nevertheless, dedicated methods for the close-range rendezvous phase remain scarce. At close range, neither modality alone is reliable: out-of-view (OoV) conditions can cause catastrophic visual failures, while dynamics-only propagation inevitably accumulates drift under model mismatch and long-horizon extrapolation. Most existing fusion pipelines inject temporal information through filtering alone and do not exploit the physical structure implicit in motion history. We propose NCS-LVTF (Non-Cooperative Satellite Layered Vision–Temporal Fusion), a layered vision–temporal fusion framework for continuous 6-DoF tracking of non-cooperative satellites in close-range proximity. OoV-robust, depth-gated multimodal visual observations form the core sensing stream, while physics-based tumbling propagation supplies a temporal prior periodically assimilated through B-ESKF (temporally-biased ESKF) with adaptive observation noise, along with learned-gated ICP geometric refinement and a series of supporting modules and system-level optimizations. We also introduce NCS-Prox, a unified three-tier dataset for close-range non-cooperative proximity tracking: Tier 1 provides a large-scale static visual corpus for module training, Tier 2 a tumbling-dynamics corpus for temporal-model validation, and Tier 3 comprises 70 continuous approach scenarios with synchronized RGB-D imagery, point clouds, IMU, and trajectory ground truth for end-to-end system evaluation. Under the NCS-Prox Tier 3 evaluation protocol, experiments show that NCS-LVTF consistently outperforms vision-only and dynamics-only baselines, achieving eR=1.72° and et=4.57 cm on the held-out Tier 3 test split.