College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, P.R. China, Key Laboratory of Spectroscopy Sensing, Ministry of Agriculture and Rural Affairs, Hangzhou, P.R. China
Abstract:Advancements in unmanned aerial vehicle (UAV) remote sensing with spectral imaging enable efficient assessment of critical agronomic traits. However, existing reflectance calibration or generation methods suffer from limited prediction accuracy and practical flexibility. This study explores reliable and cost-efficient methods for the accurate conversion of digital number values acquired from a multispectral imager into reflectance, leveraging real-time solar spectra as references. To ensure consistent measurements of incident light, an upward gimbal-mounted downwelling spectrometer was attached to the UAV, and a sinusoidal model was developed to correct for solar position variability. Using principal component analysis on the reference solar spectrum for band selection, a multiple linear regression model with four sensitive bands (4-Band MLR) and a 30 nm bandwidth achieved performance comparable to the direct correction method. The root mean square error (RMSE) for reflectance prediction improved by 86.1% compared to the empirical line method under fluctuating cloudy conditions and by 59.6% compared to the downwelling light sensor method averaged across different weather conditions. The RMSE was calculated as 2.24% in a ground-based diurnal validation, and 2.03% in a UAV campaign conducted at various times throughout a sunny day. Implementing the 4-Band MLR model enhanced the consistency of canopy reflectance within a homogeneous vegetation area by 95.0% during spectral imaging in a large rice field under significant cloud fluctuations. Additionally, improvements of 86.0% and 90.3% were noted for two vegetation indices: the normalized difference vegetation index (NDVI; a ratio index) and the difference vegetation index (DVI; a non-ratio index), respectively.
Abstract:Biomass estimation of oilseed rape is crucial for optimizing crop productivity and breeding strategies. While UAV-based imaging has advanced high-throughput phenotyping, current methods often rely on orthophoto images, which struggle with overlapping leaves and incomplete structural information in complex field environments. This study integrates 3D Gaussian Splatting (3DGS) with the Segment Anything Model (SAM) for precise 3D reconstruction and biomass estimation of oilseed rape. UAV multi-view oblique images from 36 angles were used to perform 3D reconstruction, with the SAM module enhancing point cloud segmentation. The segmented point clouds were then converted into point cloud volumes, which were fitted to ground-measured biomass using linear regression. The results showed that 3DGS (7k and 30k iterations) provided high accuracy, with peak signal-to-noise ratios (PSNR) of 27.43 and 29.53 and training times of 7 and 49 minutes, respectively. This performance exceeded that of structure from motion (SfM) and mipmap Neural Radiance Fields (Mip-NeRF), demonstrating superior efficiency. The SAM module achieved high segmentation accuracy, with a mean intersection over union (mIoU) of 0.961 and an F1-score of 0.980. Additionally, a comparison of biomass extraction models found the point cloud volume model to be the most accurate, with an determination coefficient (R2) of 0.976, root mean square error (RMSE) of 2.92 g/plant, and mean absolute percentage error (MAPE) of 6.81%, outperforming both the plot crop volume and individual crop volume models. This study highlights the potential of combining 3DGS with multi-view UAV imaging for improved biomass phenotyping.
Abstract:Non-destructive assessments of plant phenotypic traits using high-quality three-dimensional (3D) and multispectral data can deepen breeders' understanding of plant growth and allow them to make informed managerial decisions. However, subjective viewpoint selection and complex illumination effects under natural light conditions decrease the data quality and increase the difficulty of resolving phenotypic parameters. We proposed methods for adaptive data acquisition and reflectance correction respectively, to generate high-quality 3D multispectral point clouds (3DMPCs) of plants. In the first stage, we proposed an efficient next-best-view (NBV) planning method based on a novel UGV platform with a multi-sensor-equipped robotic arm. In the second stage, we eliminated the illumination effects by using the neural reference field (NeREF) to predict the digital number (DN) of the reference. We tested them on 6 perilla and 6 tomato plants, and selected 2 visible leaves and 4 regions of interest (ROIs) for each plant to assess the biomass and the chlorophyll content. For NBV planning, the average execution time for single perilla and tomato plant at a joint speed of 1.55 rad/s was 58.70 s and 53.60 s respectively. The whole-plant data integrity was improved by an average of 27% compared to using fixed viewpoints alone, and the coefficients of determination (R2) for leaf biomass estimation reached 0.99 and 0.92. For reflectance correction, the average root mean squared error of the reflectance spectra with hemisphere reference-based correction at different ROIs was 0.08 and 0.07 for perilla and tomato. The R2 of chlorophyll content estimation was 0.91 and 0.93 respectively when principal component analysis and Gaussian process regression were applied. Our approach is promising for generating high-quality 3DMPCs of plants under natural light conditions and facilitates accurate plant phenotyping.