Michael Klonowski, University of Colorado at Boulder; Naomi Owens Fahrner, Ball Aerospace; Casey Heidrich, University of Colorado at Boulder; Marcus Holzinger, University of Colorado at Boulder
Keywords: Cislunar architecture design, cislunar, monte carlo tree search, reinforcement learning, sda, ssa, multi-objective optimization
Abstract:
Robust Cislunar Architecture Design Optimization for Cooperative Agents
Michael Klonowski, Marcus Holzinger
University of Colorado Boulder
Naomi Owens Fahrner
Ball Aerospace
As launch capabilities and interest in Cislunar missions expand for both cooperative and uncooperative actors, so grows the necessity for a robust architecture in the Cislunar region for conducting Space Domain Awareness (SDA). Earth-based SDA architecture is limited in many ways in conducting Cislunar SDA. This is notably seen in terms of capability: the ever-increasing congestion of Low Earth Orbit (LEO) requires urgent attention from Earth-based sensors. In constructing an effective Cislunar SDA architecture, significant difficulties arise due to the highly nonlinear and chaotic dynamics governing the region. Furthermore, there exist a multitude of nonlinear, often competing objectives which must be considered in any architecture design. The result is a multi-objective (MO) mixed-integer nonlinear programming problem (MINLP) [1], the solution of which consists of a set Pareto-optimal solutions defined along the Pareto front [2]. Of significant interest among hypothetical objective functions in this problem is the minimum delta-v a cooperative agent needs to maintain some threshold of persistent custody within a Cislunar architecture while satisfying other mission objectives. This paper builds on [3] and presents a Cislunar architecture design process as a MO-MINLP problem that seeks to minimize the cost for a cooperative agent to make sufficient use of the architecture, while minimizing the cost of the architecture, and maximizing proper Cislunar catalogue maintenance objectives.
Previous work [3] served as a proof of concept for modeling the Cislunar architecture design problem as a MO-MINLP problem. This work was validated using multi-objective Monte Carlo Tree Search (MO-MCTS) [4] as the optimization algorithm to solve this problem. This proposed work is an intuitive continuation of [3] and introduces an important metric to the architecture design problem, namely, ease of use by an intelligent cooperative agent (hereby referred to as the user). At each step (tree walk) of MO-MCTS, the user, with full knowledge of the current architecture state, solves a multi-objective optimization (MOO) problem to determine a trajectory that maximizes the time spent within reach of the architecture while minimizing the total cost of the trajectory (total delta-v, payload mass, duration, etc.). The so-called minimum delta-v at each iteration corresponds to the trajectory chosen by the user that maximizes its time spent within reach of the architecture, within a reasonable set of constraints. The user is intelligent in the sense that it is presumed to be acting optimally with respect to the state of the architecture and its own controls. Through this process, the MO-MCTS algorithm guiding the architecture design learns an approximate mapping of states to minimum user cost to remain in the architecture. This is obtained through the chosen user trajectory, control policy, payload mass, and transfer duration. This mapping serves as a guide to the architecture design process to explore states that minimize this cost for the user, thereby returning a robust set of architecture designs that span the trade-space between user cost, architecture cost, and architecture performance goals.
Naturally, through the MOO problem performed by the user at each step of MO-MCTS, a library of optimal trajectories is created with intrinsic characteristics of these so-called exploiting trajectories informed by a specific Cislunar architectural layouts. Work by Bosanac [5] uses clustering methods on Poincare maps generated through surface of section (SOS) crossings of ballistic Cislunar trajectories. The SOS, defined as occurrences of periapsis with respect to a primary body, represent defining characteristics of the geometry of these trajectories. By using clustering methods, these characteristics are distinctly categorized by their geometric properties. This paper uses this method to similarly categorize evader trajectories based on their geometric properties. By geometrically categorizing optimal user trajectories informed by Cislunar architecture layouts, we can gain insight into the underlying chaotic dynamics that govern the performance of a Cislunar architecture.
Intelligent and optimal architecture is critical for the safety and sustainability of Cislunar space for all parties. Such an architecture must be robust not only for performing regularly scheduled catalogue maintenance tasks or transfer assistance but must also provide guarantees of total cost for agents using the architecture. So, a realistic, optimal architecture be affordable to use, but also must provide sufficient opportunities over time for such use. Furthermore, through generation of optimal architectures, data-mining analysis of the characteristics of exploiting trajectories can be leveraged to gain insight into the fundamental dynamics guiding optimal Cislunar architecture design.
[1] Bussieck, Michael R., and Armin Pruessner. “Mixed-integer nonlinear programming.” SIAG/OPT Newsletter: Views & News 14.1 (2003): 19-22.
[2] Marler, R. Timothy, and Jasbir S. Arora. “Survey of multi-objective optimization methods for engineering.” Structural and multidisciplinary optimization 26 (2004): 369-395.
[3] Klonowski, Michael, Marcus J. Holzinger, and Naomi Owens Fahrner. “Optimal Cislunar Architecture Design Using Monte Carlo Tree Search Methods.” (2022).
[4] Wang, Weijia, and Michele Sebag. “Multi-objective monte-carlo tree search.” Asian conference on machine learning. PMLR, 2012.
[5] Bosanac, Natasha. “Data-mining approach to poincare maps in multi-body trajectory design.” Journal of Guidance, Control, and Dynamics 43.6 (2020): 1190-1200.
Date of Conference: September 19-22, 2023
Track: Cislunar SDA