Xi Shen, Northwestern University; Julian Gamboa, Northwestern University; Tabassom Hamidfar, Northwestern University; Shamima Mitu, Northwestern University; Selim Shahriar, Northwestern University
Keywords: Image Recognition, Surveillance, Shift Invariance, Scale Invariance, Rotation Invariance, High-speed, Real-time, Digital Holography, Hybrid Opto-electronic Correlator, Joint Transform Correlator, Polar Mellin Transform
Abstract:
Real-time monitoring of a target is a fundamental part of space situational awareness that poses many challenges due to the large amounts of image data. Computational techniques are commonly implemented for this task due to their size, widespread availability, and built-in re-programmability. However, this comes at the cost of speed due to the difficulty in processing two-dimensional data. Recent developments in machine learning have helped to mitigate these issues, yet they too are limited by the low scalability of the digital domain. Optical signal processing systems have an inherent advantage in this regard, as the size of an image has little impact on the processing time thanks to the use of analog physical mechanisms to perform the required computations. Optical correlators are of particular interest for space situational awareness. These systems use converging lenses to produce the Fourier transform (FT) of two input images, which can then be multiplied together and FT’d to produce a cross-correlation. Because the correlation is obtained by the mere propagation of light, these systems are unaffected by the content or complexity of the images, operating at the speed it takes for light to propagate through them. Unfortunately, these all-optical correlators are typically limited by the speed at which the inputs can be updated. Additionally, all-optical correlators use holographic filters or nonlinear materials for the multiplication step, which make real-world implementations difficult. We have shown how opto-electronic correlators may overcome this issue by replacing these fragile materials with focal plane arrays (FPAs) and electronic processing [J. Opt. Soc. Am. A 31, 41-47, 2014] [Applied Optics 56, Issue 10, 2754, 2017]. While these types of correlators are limited by the use of slow spatial light modulators (SLMs), we recently demonstrated a technique to improve the operating speed by incorporating a holographic image database, eliminating the need for ultrafast SLMs [Optics Express 29, 40194, 2021].
The work reported here uses an opto-electronic joint transform correlator (OEJTC) which functions as follows. First, a single SLM projects the query and reference images into the optical domain, directing them towards a biconvex lens that produces the two overlapping FTs at its output plane. An FPA then detects the intensity of the interference between the FTs of the two input images. The resulting digital electronic signal contains the product of one FT with the conjugate of the other. This is then projected on an output SLM and FT’d using another biconvex lens. An FPA at the output plane detects a signal that contains the two-dimensional cross-correlation of the input images.
Two-dimensional correlations are inherently shift invariant because of the FT, yet they are not able to detect targets that have been scaled or rotated relative to the reference. This impediment can be overcome by first pre-processing the input images to convert them into signatures that contain the same information albeit represented in a shift, scale, and rotation invariant (SSRI) manner. We recently showed how the polar Mellin transform (PMT) is an excellent candidate for such a pre-processing step [Optics Express 27, 16507, 2019], as it can be computed as the log-polar coordinate transform (LPT) of the magnitude of the FT, and so can also take advantage of Fourier optics to enhance its speed. It is thus possible to construct an opto-electronic PMT pre-processing (OPP) stage that operates at the same speed as the rest of the correlation system. Briefly, the OPP functions as follows. First, the original image is projected on a high-speed SLM and directed towards a lens that produces the FT at its output plane. A high-speed FPA connected to a computer via a PCIe interface can detect the intensity of the FT. The computer captures the FT frame and uses a parallelized script to reorder the signal pixels according to the LPT, which is pre-calculated and stored in memory to avoid unnecessary real-time computations. This functionality is possible because the LPT depends exclusively on the coordinate system. We recently demonstrated that such a pipelined architecture can produce the LPT at a speed of around one millisecond per image [AMOS proceedings, 102, 2023]. Finally, the PMT’d signal is projected on a high-speed output SLM which itself functions as the input SLM for an OEJTC. Here, we report on an experimental implementation of a complete real-time SSRI OEJTC using this technique.
The demonstration of a real-time SSRI OEJTC represents a significant step in opto-electronic target recognition, overcoming the limitations of all-optical correlators while maintaining an operational speed that is an order of magnitude faster than leading computational techniques [IFAC-PapersOnLine, 51, 76, 2018]. In addition, the use of off-the-shelf components makes this a viable solution that may be implemented today for space situational awareness. Furthermore, the continuous improvements to high-speed SLMs and FPAs will only improve the functionality of these devices, offering an enticing future in robust opto-electronic high-speed target recognition systems. The work presented here has received support from AFOSR Grant No. FA9550-18-01-0359.
Date of Conference: September 17-20, 2024
Track: SDA Systems & Instrumentation