End-to-end pipeline for simultaneous temperature estimation and super resolution of low-cost uncooled infrared camera frames for precision agriculture applications

Radiometric infrared (IR) imaging is a valuable technique for remote-sensing applications in precision agriculture, such as irrigation monitoring, crop health assessment, and yield estimation. Low-cost uncooled non-radiometric IR cameras offer new implementations in agricultural monitoring. However, these cameras have inherent drawbacks that limit their usability, such as low spatial resolution, spatially variant nonuniformity, and lack of radiometric calibration. In this article, we present an end-to-end pipeline for temperature estimation and super resolution of frames captured by a low-cost uncooled IR camera. The pipeline consists of two main components: a deep-learning-based temperature-estimation module, and a deep-learning-based super-resolution module. The temperature-estimation module learns to map the raw gray level IR images to the corresponding temperature maps while also correcting for nonuniformity. The super-resolution module uses a deep-learning network to enhance the spatial resolution of the IR images by scale factors of x2 and x4. We evaluated the performance of the pipeline on both simulated and real-world agricultural datasets composing of roughly 20,000 frames of various crops. For the simulated data, the results were on par with the real-world data with sub-degree accuracy. For the real data, the proposed pipeline was compared to a high-end radiometric thermal camera, and achieved sub-degree accuracy. The results of the real data are on par with the simulated data. The proposed pipeline can enable various applications in precision agriculture that require high quality thermal information from low-cost IR cameras.

Level Set KSVD

We present a new algorithm for image segmentation - Level-set KSVD. Level-set KSVD merges the methods of sparse dictionary learning for feature extraction and variational level-set method for image segmentation. Specifically, we use a generalization of the Chan-Vese functional with features learned by KSVD. The motivation for this model is agriculture based. Aerial images are taken in order to detect the spread of fungi in various crops. Our model is tested on such images of cotton fields. The results are compared to other methods.

Simultaneous temperature estimation and nonuniformity correction from multiple frames

Infrared (IR) cameras are widely used for temperature measurements in various applications, including agriculture, medicine, and security. Low-cost IR camera have an immense potential to replace expansive radiometric cameras in these applications, however low-cost microbolometer-based IR cameras are prone to spatially-variant nonuniformity and to drift in temperature measurements, which limits their usability in practical scenarios. To address these limitations, we propose a novel approach for simultaneous temperature estimation and nonuniformity correction from multiple frames captured by low-cost microbolometer-based IR cameras. We leverage the physical image acquisition model of the camera and incorporate it into a deep learning architecture called kernel estimation networks (KPN), which enables us to combine multiple frames despite imperfect registration between them. We also propose a novel offset block that incorporates the ambient temperature into the model and enables us to estimate the offset of the camera, which is a key factor in temperature estimation. Our findings demonstrate that the number of frames has a significant impact on the accuracy of temperature estimation and nonuniformity correction. Moreover, our approach achieves a significant improvement in performance compared to vanilla KPN, thanks to the offset block. The method was tested on real data collected by a low-cost IR camera mounted on a UAV, showing only a small average error of $0.27^\circ C-0.54^\circ C$ relative to costly scientific-grade radiometric cameras. Our method provides an accurate and efficient solution for simultaneous temperature estimation and nonuniformity correction, which has important implications for a wide range of practical applications.

Improving temperature estimation in low-cost infrared cameras using deep neural networks

Low-cost thermal cameras are inaccurate (usually $\pm 3^\circ C$) and have space-variant nonuniformity across their detector. Both inaccuracy and nonuniformity are dependent on the ambient temperature of the camera. The main goal of this work was to improve the temperature accuracy of low-cost cameras and rectify the nonuniformity. A nonuniformity simulator that accounts for the ambient temperature was developed. An end-to-end neural network that incorporates the ambient temperature at image acquisition was introduced. The neural network was trained with the simulated nonuniformity data to estimate the object's temperature and correct the nonuniformity, using only a single image and the ambient temperature measured by the camera itself. Results show that the proposed method lowered the mean temperature error by approximately $1^\circ C$ compared to previous works. In addition, applying a physical constraint on the network lowered the error by an additional $4\%$. The mean temperature error over an extensive validation dataset was $0.37^\circ C$. The method was verified on real data in the field and produced equivalent results.