Julian Gamboa, Northwestern University; Xi Shen, Northwestern University; Tabassom Hamidfar, Northwestern University; Selim Shahriar, Northwestern University
Keywords: Image Recognition, Surveillance, Shift Invariance, Scale Invariance, Rotation Invariance, Field Programmable Gate Array, Digital Holography, Hybrid Opto-electronic Correlator, Polar Mellin Transform
Abstract:
Space situational awareness requires the monitoring of numerous terrestrial sites and spatial bodies. However, current state-of-the-art target recognition systems are severely limited in their operational speed, largely due to technological limitations when dealing with the large amounts of data inherent to image processing. New tools such as machine learning have undoubtedly helped alleviate these problems, but themselves face speed issues when using images with higher resolutions. Optical systems are largely independent of the resolution of the source images, as they primarily rely on analog physical processes for operation. Optical correlators, for example, use a combination of lenses and holograms to perform Fourier transforms (FTs) and optically multiply them together, producing a correlation signal. This technology has existed for many decades, yet it has remained impractical due to performance limitations of the required dynamic nonlinear materials. We recently proposed [J. Opt. Soc. Am. A 31, 41-47, 2014] and demonstrated [Applied Optics 56, Issue 10, 2754, 2017] a hybrid opto-electronic correlator (HOC) that overcomes these material limitations by using spatial light modulators (SLMs), phase stabilization circuits, and VLSI chips to record the required amplitude and phase information on focal plane arrays (FPAs) through interference with reference plane-waves, thus replacing the analog holographic technique with a digital one. While the HOC architecture is inherently shift-invariant, we proposed [J. Opt. Soc. Am. A 31 No. 6, 1259, 2014] and demonstrated [Optics Express 27, 16507, 2019] that the platform can perform shift, scale, and rotation invariant (SSRI) correlation by pre-processing the input images using the polar Mellin transform (PMT). This correlator can theoretically operate at speeds on the microsecond scale but is limited by the availability of high-speed SLMs to project the query images into the optical domain. As such, we further investigated the use of holographic memory devices (HMDs) as all-optical databases of PMT reference images within the HOC, demonstrating that they can eliminate the need for ultrafast SLMs in SSRI target recognition [Optics Express 29, 40194, 2021]. Additionally, we have recently eliminated the requirement for optical phase stabilization in this architecture through the use of an off-axis technique, removing perhaps the largest practical limitation of this technology [Optics Express 31, 5990, 2023].
Briefly, the HOC functions as follows. First, a laser beam is expanded and split into an auxiliary-beam arm, an input arm, and an output arm. The input arm consists of two identical segments, one for the reference and another for the query, wherein an HMD or SLM projects an image towards a lens that produces the FT at its output plane. One FPA detects the intensity of the FT, another detects the intensity of the interference between the FT and an auxiliary beam from the corresponding arm, and a third FPA detects the intensity of the auxiliary beam itself. The detected signals are processed electronically via a field programmable gate array (FPGA) and projected on an output SLM to be FT’d again, producing the final correlation signal that can be captured on an FPA or detector, depending on the application. In order to avoid having to stabilize the optical phases of the auxiliary beams, it is necessary to place the reference and query HMD/SLM such that they are shifted in opposite directions along their projection planes. For SSRI operation, it is necessary to pre-process the input images using the PMT. For the reference arm, these pre-processed images can be stored directly on the HMD and accessed at high speed. However, this is not possible for the query arm, as the images are unknown at the time of construction. For this reason, it is necessary to develop a technique that can generate the PMT at speeds compatible with the rest of the HOC.
The PMT can be calculated as the log-polar transform (LPT) of the magnitude of the FT of an image. Because of this, optics can be used for the FT part of the operation, while an FPGA can be used for the log-polar component. An opto-electronic PMT pre-processor (OPP) based on this approach functions as follows. First, the original image is projected on an SLM and directed towards a lens that will produce the FT at its output plane. An FPA can detect the intensity of the FT and transmit the signal to an FPGA, performing circular DC blocking to eliminate the DC component. Next, the FPGA reorders the signal pixels according to the LPT, which is pre-calculated and stored in memory to avoid unnecessary real-time repetitions. This is possible because the LPT only depends on the coordinate system, unlike the FT which depends on the value at every possible coordinate. Finally, the PMT’d signal is projected on an output SLM, which can itself function as the input SLM for the query arm of the HOC. Here we report on the development and demonstration of the OPP for use in an automated high-speed HOC, with an operating speed of 500 frames per second (2 ms per operation).
Demonstration of the operation of the OPP at this speed establishes the feasibility of operating an SSRI HOC essentially at the same speed, using only components that are currently available on the market. This operating speed represents nearly an order of magnitude enhancement in the speed of operation for the task of SSRI image recognition, compared to the leading computational methods, which are typically limited to ~12 ms per operation [IFAC-PapersOnLine, 51, 76, 2018]. To date, this is the only optics-based PMT processor ever developed, enabling the fastest SSRI implementation in a correlator. In addition to enhancing the HOC, the OPP can be easily implemented in existing optical image processors to enable them to perform at high speed in an SSRI manner. In principle, the speed could be improved to operate in the microsecond regime, although further development is needed to reach that stage. The incorporation of the OPP and the automation of the HOC has resulted in a high-speed and robust image recognition system for space situational awareness. This work has been supported by AFOSR Grant No. FA9550-18-01-0359.
Date of Conference: September 19-22, 2023
Track: SDA Systems & Instrumentation