Automated co-registration of multitemporal series of multispectral uav images for crop monitoring

Abstract and Figures

The technologies and applications of aerial photography have exponentially evolved in the past two decades. The advent of purpose specific micro sensors and unmanned aerial vehicles (UAVs) have gone a long way in revolutionizing the applications of aerial images from crop monitoring to hazard assessment. The use of UAVs has become widespread due to its operational flexibility, ultra-high spatial resolution acquisition, and inexpensiveness. However, the application of multitemporal series of multispectral UAV imagery still suffers a setback of significant misregistration errors, and therefore becoming a concern for applications such as precision agriculture. In addition, the micro sensors have different spectral properties thus an integration and/or comparative analysis is inevitable. Researchers have widely studied the application of UAV imagery; topical issues range from spectral and spatial properties to image registration and multi-sensor integration. In crop monitoring, accurate co-registration of images acquired within and in different epochs remains to be exhaustively researched on; spectral band-to-band alignment is fundamental to precise crop sensing. Although the application of UAV-based sensors is widespread, direct image georeferencing and co-registration is done using GCP; this is usually costly and time consuming. This research proposes a novel approach for automatic co-registration of multitemporal UAV imagery using intensity-based keypoints. The approach is based on multispectral orthophoto alignment. However, photogrammetric products such as Digital Surface Models (DSM) present an interesting challenge to automatically co-register a multitemporal series of such products including the several spectral channels, from three to twelve, captured by the UAV-based micro sensors. This research makes an effort to investigate inherent intra-epoch and inter-epoch co-registration errors in multispectral imagery, and the spatial data quality of photogrammetric products of two UAV-based cameras (Parrot Sequoia and Micro MCA Tetracam). A successful image registration involves four major stages, feature detection, feature description or descriptor extraction, feature matching, and geometric transformation. Estimation of geometric transformation from matched point pairs yields a 2D transformation matrix, in this case a similarity transformation matrix, which maps the inlying matched pairs. The inliers are selected randomly iteratively until sufficient pairs fitting the transformation model is reached. In this study, existing keypoints detection algorithms were assessed to identify the most viable algorithm to be implemented for co-registration of multi-spectral and multitemporal UAV images. Intensity-based registration algorithms (SURF, BRISK, MSER and KAZE) were tested and optimally parameterized. A detailed comparison on the performance of these algorithms was done and an informed decision was reached to pursue further experiments with only SURF and KAZE. The co-registration error analysis shows that optimally parametrized SURF and KAZE algorithms can obtain co-registration accuracies of 0.1 and 0.3 pixels for intra-epoch and inter-epoch images respectively. To obtain better intra-epoch co-registration accuracy, collective band processing is advised. On the other hand, the quality of the DSM per band is evaluated and the results show that the red band is best fit for DSM extraction. In addition, the spatial variability of the UAV-based spectral features and that of Sentinel 2B satellite imageryare compared; the results show that regardless of the differences in spectral bandwidths, and spatial resolution, they are highly correlated. A positive correlation of 0.93 and 0.77 was obtained for the maize field and non-vegetated area respectively. However, the UAVs take pride in the spatial resolution advantage to reveal intrinsic intra-farm variability. On the other hand, a comparison of spectral response of vegetation using Parrot Sequoia and Micro MCA Tetracam show that although both cameras are able to correctly sense active and declining photosynthetic activity in crops, spectral and radiometric calibration is key to achieving optimal response for the Micro MCA camera. In light of the results obtained in this study, descriptor-based methods are fit for coregistration of multispectral imagery for crop monitoring. In addition, UAV based multispectral cameras have different specifications and thus differ in the quality of their respective photogrammetric outputs; using the same system for monitoring purposes is advised.