Steven Rossland, University of Colorado Boulder; Greg Furlich, University of Colorado Boulder; Tanner Gill, University of Colorado Boulder; Brian Sweeney, University of Colorado Boulder
https://doi.org/10.64861/LHXJ5779
Keywords: Transformer Models; Photometric Time Series; Satellite Characterization; Space Domain Awareness; Fourier Analysis; Machine Learning
Abstract:
Space Domain Awareness (SDA) as a mission processing and analysis system relies on the methods for detection, orbit determination, and characterizing to interpret the nature of Resident Space Objects (RSOs) under limited and often ambiguous measurement conditions. While detection and orbit determination have matured with robust matched filtering and statistical filtering algorithms over the history of SDA, characterization methods have lagged behind. Characterization presents a data-starved challenge dependent on the subtle temporal structure within photometric light curves, and the derived kinematic state estimates. Photometric variability may reflect attitude motion, operational pointing modes, stabilization states, or anomalous behavior, yet extracting this information requires modeling time-dependent structure rather than isolated measurements. This work investigates the use of transformer-based sequence models as a general tool for extracting behavioral structures from sequential physical measurements, with emphasis on representation design and characterization inference.
We present two independent but complementary approaches that apply attention-based sequence modeling to RSO photometric data. The two models operate in different domains: one in the time domain and the other in the frequency domain. Both approaches treat transformers not as language models, but as temporal inference architectures whose effectiveness depends critically on representation of physically meaningful observables. As transformers in Large Language Models demonstrate the ability to learn grammatical rules of language, our approach is grounded in training transformers to learn the laws of nature such as orbital mechanics for use in their analysis of RSOs.
Because these models capture sequence likelihood, deviations from learned transition structure or from predicted light curve evolution can be flagged as anomalous behavior. Prototype training of characterization was tested on datasets from the Space Debris Light Curve Database (SDLCD) and Mini-MegaTORTORA (MMT-9). These models characterized RSOs into operational, mission stare, and tumbling and yielded accuracy on the order of 70%. Application to publicly available photometric data sets further suggests that harmonic embeddings provide a compact and informative representation for transformer-based behavioral inference, particularly when periodic structure dominates the observed signal.
Taken together, these two approaches illustrate distinct domain philosophies for applying transformer architecture to physical systems: direct temporal embeddings of photometric and kinematic observables, and frequency embeddings through Fourier decomposition with grammar-based sequence modeling. In both cases, representation design emerges as the central determinant of performance. Transformers themselves provide a flexible mechanism for modeling temporal dependencies; however, physically meaningful embedding is required to ensure interpretability, stability, and generalization.
Another key feature of the transformer exploit is the next token prediction as a means to forecast future states and behaviors of the RSO. This allows these methods to distill behavior and intent characterizations from current observations as well as predict future intent and behavior. These applications demonstrate the transformer as a powerful tool for battlespace awareness.
This work demonstrates the feasibility of attention-based sequence models for RSO photometric characterization and behavioral regime inference; however, it highlights the importance of physics-informed representation in extending transformer methods beyond language modeling into dynamical systems analysis.
Date of Conference: September 15-18, 2026
Track: Satellite Characterization