Collin Phillips, Virginia Tech
https://doi.org/10.64861/XURI2644
Keywords: Reinforcement Learning, Space Domain Awareness, Autonomous Sensor Tasking, Continuous Boresight Tasking, Deep Neural Networks
Abstract:
Space sensor tasking is an inherently difficult problem with complex decision metrics and trade-offs which grow ever more intractable as increasing number of resident space objects (RSOs) enter the domain. Solutions often ignore long-horizon planning, or the potential for multiple observations per tasking-period. Deep reinforcement learning (DRL) is a promising solution for complex decision problems like this one, and recent work has explored its application to the space domain awareness (SDA) problem. However, the general state-of-the-art restricts the action space to a discrete choice in either grid or target index selection. Additionally, these algorithms reduce the observation space to either a pooled matrix of RSO features on the sensing-hemisphere, or a fixed-sized vector representing a subset of the catalog. These restrictions are helpful for policy convergence, but necessarily restrict how precisely a policy can represent the environment. In this work we introduce a novel approach to observation space encoding on arbitrary catalog sizes, and train an agent to produce boresight tasks on the continuous unit sphere._x000D_
_x000D_
Our environment has been created for this effort as an extension of the open source RESONAATE toolkit, a high-fidelity orbit propagator and tasking simulator. We train specifically for a single-sensor multiple-target environment, using randomly generated sets of geosynchronous targets created by sampling feasible orbital elements in the field of regard. We describe a flexible feature-vector representation for each RSO, which we gather into an unordered set for the tasking agent. Our environment reward is a proxy for covariance reduction which uses the observation age as a confidence metric; while less exact than propagating each UKF, our metric is a reasonable approximation that saves considerable time during training. Our architecture ingests an arbitrary set of unordered RSO feature vectors, and embeds a fixed-width context vector to encode relevant features of the environment state. Using a combination of self-attention and cross-attention blocks, our architecture encodes geometric features of the relationship between all RSOs in the field of regard, and then selects a pointing vector from the set of inputs. We then demonstrate a behavior cloning approach that uses a naïve greedy algorithm to generate an offline dataset, and supervised pre-training to transfer the greedy policy to our network. Early results show successful transfer with greedy performance within margin of error. _x000D_
_x000D_
While the greedy algorithm is generally optimal for short-horizons in the single-agent case, there is potential reward to be gained by allowing the boresight to vary continuously; because, by adjusting the boresight in a neighborhood around any given target, it is often possible to observe multiple targets in a single collect. We implement a fine-tuning routine using proximal policy optimization (PPO) as an attempt to surpass the naïve greedy algorithm used to construct the offline dataset. Our study also investigates the application of multiple reward metrics, and compares their performance for both the tabula-rasa (random weights) and behavior cloned training routines. Our trained policy is compared against both random, sweeping, pretrained, and greedy tasking baselines. Additionally, we study the robustness of our trained policy against new ground-site locations and varying catalog sizes without retraining, demonstrating that the policy has effectively learned a geometric solution, independent of the ground-site location. _x000D_
_x000D_
This work represents a novel way to encode the SDA problem space for DRL tasking solutions. We demonstrate the scalability of our approach to large catalogs, and validate performance against near-optimal baselines. Furthermore, we believe this work represents a path towards more advanced applications of deep learning for the problem domain including deep multi-agent RL, inference on large catalogs, and a robust neural-network backbone for individual RSO feature embeddings._x000D_
Date of Conference: September 15-18, 2026
Track: AI for SDA Applications