Sven Müller, Institute of Space Systems, Technische Unviversität Braunschweig; Enrico Stoll, Institute of Space Systems, Technische Unviversität Braunschweig
Keywords: data streams, real-time, measurements, filtering, SST
Abstract:
With increasing amounts of raw sensor measurements of resident space objects and the tendency to go to near-real-time processing, it becomes more and more important to discard measurements of certain objects quickly. This way, computationally intense processing can be constrained to a subset of the measurements, for example to objects on a whitelist. This lowers the burden to install processing chains that automatically work on new measurement data as soon as it is acquired. If such a filtering is done in near-real-time, it may enable new usage concepts of sensor data, e.g. to provide whitelisted measurements to certain institutions, corporations or the public instantly, without the need for human intervention. Near-real-time filtering also allows rapid sensor tasking. As soon as measurements are produced that pass the filter, the sensor (or another one) could be tasked to track the detected objects. This makes near-real-time filtering a potentially valuable mechanism for Space Surveillance and Tracking. The output side of such a filtering mechanism can be regarded as a virtual sensor, since it outputs the same kind of measurement data with roughly the same timing, only filtered. Thus, existing interfaces to process measurement data do not need to be changed for integrating this approach.
By utilizing so-called Data Stream Management Systems (DSMS), continuous near-real-time filtering can be achieved. DSMS are essentially databases transformed for the purpose of near-real-time processing of data that streams in continuously. They are not meant to be used for saving data persistently, though, and thus, do not save processed data on HDD or SSD, but only in RAM – and only as long as the data is still being processed. Because of the processing being done in RAM only, DSMS are able to cope with very high input rates of raw measurement data. Furthermore, these systems follow the data-driven processing paradigm, meaning that the input of each single data point triggers the data processing and new data results are produced. The processing steps done by a DSMS in this case, filtering – can be modified easily by changing the data flow definition sent to the DSMS. These definitions often are saved in simple ASCII text files. Some systems even allow changes in the data flow, when the DSMS is still processing data. Data fusion is also possible with these types of systems, since they provide mechanisms for branching and joining data streams. In order to organize processing of potentially infinite incoming data streams, DSMS create windows on the streams, for example windows with all data elements that streamed in in the last five minutes. Data elements that drop from such a window are dismissed. Additionally, DSMS come with general processing mechanics necessary for achieving high performance, such as the scheduling of multiple parallel internal processing chains. One of the reasons, they have been invented is to be able to handle uncertain data – like sensor measurements.
This paper elaborates on DSMS and their capabilities. It is shown how a specific DSMS was adjusted for the purpose of measurement filtering. A test scenario and its execution are described, the results are discussed.
Date of Conference: September 11-14, 2018
Track: Poster