Kevin Vanslette, Raytheon BBN; Alexander Lay, Raytheon BBN; David Kusterer, Virginia Tech Institute for National Security; Kevin Schroeder, Virginia Tech Institute for National Security
Keywords: Artificial Intelligence, Orbital Propagation, Uncertainty
Abstract:
We present a technique for rapid and uncertainty quantified orbital propagation using “uncertainty-aware” deep neural networks, based on our prior work [1]. This “far faster than real-time” propagation method will enable scalable higher-level track forecasting analysis such as maneuver detection, observation correlation, multi-hypothesis and counterfactual forecasting, sensor scheduling, and conjunction screening at unprecedented scales. This work leverages advances in uncertainty-aware artificial intelligence (AI) [1-4] to reliably predict accurate tracks and multi-time track covariance matrices from high fidelity simulation data of both “expected inlier” and “unexpected outlier” track scenarios.
Due to the proliferation of Resident Space Objects (RSOs) in Low Earth Orbit (LEO), the task of real-time orbital tracking and propagation over the entire LEO belt is computationally expensive using traditional physics-based methods. Physics-based methods are used for propagation and tracking due to their high fidelity, reliability, explainability, control and maneuver modeling, and their ability to quantify and forecast track uncertainty. Due to the nonlinearity of orbital dynamics, the primary computational expense comes from accurately approximating the propagation of track uncertainty. Common methods, such as the unscented or particle Kalman Filter, require repeated sampling of the propagator, which is only partially parallelizable. Further, once new observations are obtained, track and uncertainty forecasts need to be repropagated.
Despite producing state-of-the-art results in terms of prediction accuracy, traditional deep learning AI methods can still fail to meet the high reliability and robustness standards that would allow for their adoption into critical space domain applications. This is to say, it is not good enough to make accurate predictions over “inlier” training and testing datasets if the model then fails to generalize appropriately in real world applications that may include “outliers”. If new data is sufficiently different from the data used to train the model, i.e. outlier data, traditional AI methods often make overly confident, yet incorrect, predictions [4]. This is a major problem for space applications because not only is the prediction wrong, it is wrong without warning of possible errors, which could compromise the integrity of downstream processing and decision making. Outlier situations arise when applying an AI model outside of the domain of the training data. This problem can be readily observed by training on one orbit but testing on another, training on station keeping maneuvers but testing on large maneuvers, or training on low-fidelity simulation data but testing on real data.
If instead the AI model was trained to produce properly quantified predictive uncertainties (i.e. orbital uncertainties), the method would gracefully handle outlier situations by stating it “doesn’t know” through the prediction of large uncertainties. These large uncertainties can be used to “tip-off” downstream processes and decision makers to defer to other models (such as traditional physics-based models) to ensure the robustness of the application. The subfield of AI that aims to get AI to accurately predict its own uncertainty is called uncertainty-aware, or probabilistic, AI [1-4].
To combat scalability challenges that arise when trying to track and propagate RSOs in LEO, we present an uncertainty-aware deep neural network approach that forecasts both orbital trajectories and their corresponding joint covariance matrices over the trajectory’s joint multi-time configuration space. This approach has two distinct advantages over traditional physics-based orbital propagation. First, in desktop experiments and only using the CPU, this approach yielded a greater than 1000 times speed-up in terms of the number of propagations per second over Orekit, which is a leading open-source physics-based propagation tool (applied to 40,000 simulated RSOs in LEO). Because our method forecasts trajectories and covariance in the joint configuration space, the second advantage is that updating forecasted tracks with new observations (or counterfactual conditions) simply requires updating the joint multi-time mean trajectory vector and covariance matrix, which can be done in parallel across the trajectory rather than requiring serial repropagation. On a challenging data set and for a related problem [1], our method was shown to predict accurate (error quantifying) covariance matrices within a few percentage points of the ground truth error distribution, which means the predicted heteroskedastic uncertainties from the model were well calibrated. Further, the method reliably predicts large uncertainties in outlier situations by design, which suppressed Kalman filter track aberrations due to outliers compared to existing methods in the literature.
In lieu of real data, we will explore two methods of training our uncertainty-aware AI model using the REsponsive Space ObservatioN Analysis and Autonomous Tasking Engine (RESONAATE), which has a higher fidelity than Orekit. RESONAATE will be used to generate high fidelity physics-based orbital track data and uncertainties with station keeping maneuvers for training and testing. The AI model will train using the same input parameters as RESONAATE such as spacecraft parameters (e.g. mass, drag coefficient, and reflectivity) and environmental parameters (solar radiation pressure, Earth precession and nutation, atmospheric drag, and gravity) as well as the prior trajectory and covariance information. Following [1], we will explore single mode deep Gaussian ensembles as well as dynamic deep multivariate mixture ensembles to capture the non-Gaussian deviations (e.g. covariance non-realism [5]) that occur in the long-time propagation horizon due to the nonlinearity of the propagation dynamics.
To demonstrate the robustness of the approach and its ability to gracefully generalize outside of the “inlier” training station keeping data, we will also test and verify substantial increases in forecast uncertainty when it is presented with large, non-station keeping, “outlier” maneuvers.
[1] Vanslette, K., Lay, A., et al. “Dynamic Deep Multivariate Mixture Ensembles with Applications to Overwater Emitter Localization”. In progress.
[2] Lakshminarayanan, B., et al. “Simple and scalable predictive uncertainty estimation using deep ensembles.” Advances in neural information processing systems 30 (2017).
[3] Russell, R. L., et al. “Multivariate uncertainty in deep learning.” IEEE Transactions on Neural Networks and Learning Systems 33.12 (2021): 7937-7943.
[4] Ovadia, Y., et al. “Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift.” Advances in neural information processing systems 32 (2019).
[5] Poore, Audrey B., et al. “Covariance and uncertainty realism in space surveillance and tracking.” Report of the Air Force Space Command Astrodynamics Innovation Committee 27 (2016).
Date of Conference: September 17-20, 2024
Track: Machine Learning for SDA Applications