Anomalous Detection in Hyperspectral Imagery Using Autoencoder Technology

Leveraging spectral and spatial information to improve autoencoders, enhancing the identification of abnormal pixels

Hyperspectral imaging (HSI) from spaceborne, airborne, and drone platforms has emerged as a transformative Earth observation technology and provides spatially distributed detailed spectral information across potentially hundreds of contiguous wavelength bands. By capturing the unique spectral signatures of materials, hyperspectral sensors enable the precise identification and characterization of vegetation, soils, minerals, water quality, and man-made objects. Satellite-based systems offer large-scale and repetitive monitoring for global environmental and climate studies, while airborne sensors provide higher spatial resolution and flexibility for regional applications. Drone-mounted HSI cameras further enhance observation capabilities by delivering ultra-high-resolution data tailored to local-scale investigations. Together, these platforms support a wide range of applications, including precision agriculture, ecosystem monitoring, biodiversity assessment, mineral exploration, environmental pollution detection, disaster management, urban mapping, and infrastructure inspection. The synergy between space, air, and drone observations enables multi-scale analyses, fostering improved understanding and sustainable management of natural and built environments.

A hyperspectral image is arranged by a three-dimensional (3D) data cube (Fig. 1) acquired by using imaging and spectroscopy techniques.^2,3 HSI provides 2D spatial information along with comprehensive spectral information of objects mapped to the third dimension. Thus, spectral-spatial information can explore the fine-detailed properties of objects in the images. The purpose of anomaly detection is to find abnormal samples whose distributions are inconsistent with most instances in the dataset.⁶ Hyperspectral anomaly detection (HAD), which is used to identify spectrally distinct (abnormal) pixels, is crucial in many applications, such as environmental monitoring, precision agriculture, and surveillance. In case of environmental monitoring, HAD enables early detection of forest stress, pollution, and oil spills. In precision agriculture, they facilitate the early detection of crop diseases, nutrient deficiencies, and water stress, and in surveillance, they help to identify camouflaged vehicles and concealed installations. The combination of spatial and spectral information gives complementary benefits for better HAD. Many methods have been developed for HAD; these can be classified into probability-distribution-based, representation-based, and deep learning-based ones.⁵

Enhancing autoencoders for HAD

Deep learning methods can be grouped into supervised learning and unsupervised learning frameworks. Supervised learning methods are distinguished by their reliance on labeled training data for model development. In contrast, unsupervised learning methods do not need to know anything about the target pixels a priori. A well-known unsupervised feature extraction model is autoencoder (AE), and it has been widely applied for HAD.^2,6 The idea of AE is to consider that most of the pixels in HSI are background, while the anomalous pixels are only a small part of the image. Reasonably, when a hyperspectral image is used in the AE model as input, the model generally learns the background. Then AE simultaneously reconstructs both the anomaly targets and the background, but the lack of prior information limits the ability to detect anomalies⁶ and can provide many false positive responses.

Recently, we developed an improved version of AE¹ that combines a 3D Convolutional Autoencoder and Spectral-Spatial Attention (SSA). The new approach employs 3D convolutional layers to create a compact latent space from the input image, while the decoder uses deconvolution layers to recover the input image (Fig. 2). The 3D convolutional layers convolve across spectral and spatial dimensions, thus leveraging both the spectral and spatial hierarchy of image features. 3D convolutional AE preserves spectral-spatial patterns well but focuses mainly on local neighborhoods. It treats all spectral channels (bands) equally over the whole image but with no extra ‘attention’ toward potentially important ones. Therefore, they may fail to focus on informative bands. To focus on important bands, we attach multi-head spectral attention and multi-scale spatial attention, where spectral attention dynamically reweights each spectral band. Hence, the network emphasizes discriminative wavelengths and suppresses noise. Besides, multi-scale spatial attention focuses on where (which pixels or regions) to emphasize or suppress, which enhances local contrast and spatial coherence. Details of the developed algorithm can be found in.¹

Demonstrating performance using the ABU Urban-II dataset

To show the performance of the developed AE algorithm, we build an error map based on the well-known robust Mahalanobis-statistical distance metric. We experimented on several benchmark HSI datasets. Here is an example to show the performance of the new method. The dataset that we used was acquired using the airborne visible/infrared imaging spectrometer (AVIRIS) over the Texas Coast area in the USA. The dataset is known as the ABU Urban-II dataset (Texas Coast-II or ABU-U2⁴). It has 207 spectral bands within 400 to 2500 nm wavelengths, and the spatial resolution is 17.2 m/pixel. The size of the dataset is 100*100 pixels. The buildings are treated as anomalies and comprise only 155 pixels. The false-color image and the ground truth (GT) of this dataset are shown in Fig. 3(a) and 3(b), respectively. We calculate the area under the curve (AUC), which is one of the most widely used metrics to evaluate the performance of HAD algorithms. The new algorithm achieves an AUC of 99.91%.