Detection method of high temperature stress of tomato at seedling stage based on thermal infrared and RGB images

ZHONG Lujie; WANG Xiaochan; ZHANG Xiaolei; SHI Yinyan

doi:10.7671/j.issn.1001-411X.202203039

Journal of South China Agricultural University > 2023 > 44(1): 110-122. > DOI: 10.7671/j.issn.1001-411X.202203039

ZHONG Lujie, WANG Xiaochan, ZHANG Xiaolei, et al. Detection method of high temperature stress of tomato at seedling stage based on thermal infrared and RGB images[J]. Journal of South China Agricultural University, 2023, 44(1): 110-122. DOI: 10.7671/j.issn.1001-411X.202203039

Citation:

PDF (3416 KB)

Detection method of high temperature stress of tomato at seedling stage based on thermal infrared and RGB images

ZHONG Lujie^1,,
WANG Xiaochan^{1, 2, ,},
ZHANG Xiaolei^{1, 2},
SHI Yinyan^{1, 2}

1.
College of Engineering, Nanjing Agricultural University, Nanjing 210031, China
2.
Engineering Laboratory of Modern Facility Agriculture Technology and Equipment in Jiangsu Province, Nanjing 210031, China

More Information

Received Date: March 20, 2022
Available Online: May 17, 2023

Graphical Abstract

Abstract

Abstract

Objective
Aiming at the problem of high temperature stress encountered in the growth of tomato at seedling stage in actual production scenarios, a method for detecting high temperature stress of tomato at seedling stage based on thermal infrared and RGB images was proposed.

Method
Firstly, the tomato canopy temperature parameters were obtained by inversion through the thermal infrared image of tomato plant at seedling stage, and the canopy temperature characteristic indicators were extracted by the partial least squares (PLS) model. Then, a Mask-RCNN model using three different backbone feature extraction networks was established, and the RGB images of tomato seedlings were input into the Mask-RCNN model by means of transfer learning for instance segmentation of high temperature stress symptoms. The characteristic indicators of tomato stress symptoms at seedling stage were obtained. Finally, the extracted temperature and stress symptom characteristic indicators were used to construct a hierarchical data set and fed into the high temperature stress classification model to obtain the high temperature stress level.

Result
The cumulative contribution rate of the canopy temperature characteristic indicators extracted based on the PLS model reached 95.45%. The high temperature stress symptom segmentation network based on ResNet101+Mask-RCNN had the highest segmentation accuracy for mild and severe stress of tomato at seedling stage, with mean average precision (mAP) of 77.3% and 73.8% respectively. Among the four high temperature stress grading models constructed based on temperature and stress symptom characteristic indicators, the back propagation neural network (BPNN) showed the best high temperature stress grading performance with the grading accuracy rate of 95.6%.

Conclusion
The proposed method in this study achieves better detection performance for high temperature stress of tomato at seedling stage, and provides a technical support for early detection and rapid automatic warning of high temperature stress of tomato at seedling stage.
- Seedling stage,
- Tomato,
- High temperature,
- Thermal infrared image,
- Mask-RCNN,
- Stress detection

FullText(HTML)

References (38)

References

[1]	ZHANG J, JIANG X D, LI T L, et al. Photosynthesis and ultrastructure of photosynthetic apparatus in tomato leaves under elevated temperature[J]. Photosynthetica, 2014, 52(3): 430-436. doi: 10.1007/s11099-014-0051-8
[2]	巩文睿, 金萍, 钟启文. 设施农业物联网技术应用现状与发展建议[J]. 农业科技管理, 2017, 36(4): 20-23. doi: 10.16849/j.cnki.issn1001-8611.2017.04.006
[3]	ZHOU R, KONG L, WU Z, et al. Physiological response of tomatoes at drought, heat and their combination followed by recovery[J]. Physiologia Plantarum, 2019, 165(2): 144-154.
[4]	WEN J Q, JIANG F L, LIU M, et al. Identification and expression analysis of Cathepsin B-like protease 2 genes in tomato at abiotic stresses especially at high temperature[J]. Scientia Horticulturae, 2021, 277(10): 109799.
[5]	TORRES G M, LOLLATO R P, OCHSNER T E. Comparison of drought probability assessments based on atmospheric water deficit and soil water deficit[J]. Agronomy Journal, 2013, 105(2): 428-436. doi: 10.2134/agronj2012.0295
[6]	WISHART J, GEORGE T S, BROWN L K, et al. Field phenotyping of potato to assess root and shoot characteristics associated with drought tolerance[J]. Plant and Soil, 2014, 378(1/2): 351-363.
[7]	ZHOU Z, MAJEED Y, DIVERRES NARANJO G, et al. Assessment for crop water stress with infrared thermal imagery in precision agriculture: A review and future prospects for deep learning applications[J]. Computers and Electronics in Agriculture, 2021, 182: 106019.
[8]	CHANDEL A K, KHOT L R, YU L X. Alfalfa (Medicago sativa L. ) crop vigor and yield characterization using high-resolution aerial multispectral and thermal infrared imaging technique[J]. Computers and Electronics in Agriculture, 2021, 182: 105999.
[9]	HOU M J, TIAN F, ORTEGA-FARIAS S, et al. Estimation of crop transpiration and its scale effect based on ground and UAV thermal infrared remote sensing images[J]. European Journal of Agronomy, 2021, 131: 126389. doi: 10.1016/j.eja.2021.126389
[10]	DAS S, CHAPMAN S, CHRISTOPHER J, et al. UAV-thermal imaging: A technological breakthrough for monitoring and quantifying crop abiotic stress to help sustain productivity on sodic soils: A case review on wheat[J]. Remote Sensing Applications Society and Environment, 2021, 23: 100583. doi: 10.1016/j.rsase.2021.100583
[11]	ZHU W, CHEN H, CIECHANOWSKA I, et al. Application of infrared thermal imaging for the rapid diagnosis of crop disease[J]. IFAC-PapersOnLine, 2018, 51(17): 424-430. doi: 10.1016/j.ifacol.2018.08.184
[12]	LACERDA L N, SNIDER J L, COHEN Y, et al. Using UAV-based thermal imagery to detect crop water status variability in cotton[J]. Smart Agricultural Technology, 2022, 2: 100029. doi: 10.1016/j.atech.2021.100029
[13]	TIAN F, HOU M, QIU Y, et al. Salinity stress effects on transpiration and plant growth under different salinity soil levels based on thermal infrared remote (TIR) technique[J]. Geoderma, 2020, 357: 113961. doi: 10.1016/j.geoderma.2019.113961
[14]	MA S, BI Y, ZHANG Y, et al. Thermal infrared imaging study of water status and growth of arbuscular mycorrhizal soybean (Glycine max) under drought stress[J]. South African Journal of Botany, 2022, 146: 58-65. doi: 10.1016/j.sajb.2021.09.037
[15]	BANERJEE K, KRISHNAN P. Normalized sunlit shaded index (NSSI) for characterizing the moisture stress in wheat crop using classified thermal and visible images[J]. Ecological Indicators, 2020, 110: 47-59.
[16]	SINGH A, GANAPATHYSUBRAMANIAN B, SINGH A K, et al. Machine learning for high-throughput stress phenotyping in plants[J]. Trends in Plant Science, 2016, 21(2): 110-124. doi: 10.1016/j.tplants.2015.10.015
[17]	SINGH A K, SINGH, GANAPATHYSUBRAMANIAN B, SARKAR S, et al. Deep learning for plant stress phenotyping: Trends and future perspectives[J]. Trends in Plant Science, 2018, 23(10): 883-898. doi: 10.1016/j.tplants.2018.07.004
[18]	ANAMI B S, MALVADE N N, PALAIAH S. Deep learning approach for recognition and classification of yield affecting paddy crop stresses using field images[J]. Artificial Intelligence in Agriculture, 2020, 4: 12-20. doi: 10.1016/j.aiia.2020.03.001
[19]	ZHOU J, LI J X, WANG C S, et al. Crop disease identification and interpretation method based on multimodal deep learning[J]. Computers and Electronics in Agriculture, 2021, 189: 106408.
[20]	MOON S K, AMJAD M, QURESHI M, et al. Use of deep learning techniques for identification of plant leaf stresses: A review[J]. Sustainable Computing: Informatics and Systems, 2020, 28: 100443. doi: 10.1016/j.suscom.2020.100443
[21]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. arXiv: 1409. 1556. (2014-01-01) [2022-01-10]. https://arXiv.org/abs/1409.1556.
[22]	SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE. 2016: 2818-2826.
[23]	WU S J, ZHONG S H, LIU Y. Deep residual learning for image steganalysis[J]. Multimedia Tools and Applications, 2018, 77(9): 10437-10453. doi: 10.1007/s11042-017-4440-4
[24]	JI M M, WU Z B. Automatic detection and severity analysis of grape black measles disease based on deep learning and fuzzy logic[J]. Computers and Electronics in Agriculture, 2022, 193: 106718. doi: 10.1016/j.compag.2022.106718
[25]	LIPPMANN R, BABBEN S, MENGER A, et al. Development of wild and cultivated plants under global warming conditions[J]. Current Biology, 2019, 29(24): R1326-R1338. doi: 10.1016/j.cub.2019.10.016
[26]	易平涛, 李伟伟, 郭亚军. 线性无量纲化方法的结构稳定性分析[J]. 系统管理学报, 2014, 23(1): 104-110. doi: 10.3969/j.issn.1005-2542.2014.01.015
[27]	MA J Y, MA Y, LI C. Infrared and visible image fusion methods and applications: A survey[J]. Information Fusion, 2019, 45: 153-178.
[28]	WESTERHUIS J A, KOURTI T, MACGREGOR J F. Analysis of multiblock and hierarchical PCA and PLS models[J]. Journal of Chemometrics, 1998, 12(5): 301-321.
[29]	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask-R-CNN[C]// 2017 IEEE International Conference on Computer Vision. Veniu, Italy: IEEE, 2017: 2961-2969.
[30]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[31]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015: 640-651.
[32]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2015: 1512-1516.
[33]	RUSSELL B C, TORRALBA A, MURPHY K P, et al. LabelMe: A database and web-based tool for image annotation[J]. International Journal of Computer Vision, 2008, 77(1/2/3): 157-173. doi: 10.1007/s11263-007-0090-8
[34]	穆龙涛, 高宗斌, 崔永杰, 等. 基于改进AlexNet的广域复杂环境下遮挡猕猴桃目标识别[J]. 农业机械学报, 2019, 50(10): 24-34. doi: 10.6041/j.issn.1000-1298.2019.10.003
[35]	乔虹, 冯全, 赵兵, 等. 基于Mask R-CNN的葡萄叶片实例分割[J]. 林业机械与木工设备, 2019, 47(10): 15-22. doi: 10.3969/j.issn.2095-2953.2019.10.003
[36]	AMARO E G, CANALES J C, CABRERA J E, et al. Identification of diseases and pests in tomato plants through artificial vision[M]//Intelligent Computing Methodologies. Cham: Springer International Publishing, 2020: 98-109.
[37]	LI Y R, YANG K M, GAO W, et al. A spectral characteristic analysis method for distinguishing heavy metal pollution in crops: VMD-PCA-SVM[J]. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 2021, 255: 119649. doi: 10.1016/j.saa.2021.119649
[38]	ANAMI B S, MALVADE N N, PALAIAH S. Classification of yield affecting biotic and abiotic paddy crop stresses using field images[J]. Information Processing in Agriculture, 2020, 7(2): 272-285.

Cited By

Cited by

Periodical cited type(9)

1.	柯欣，费琪，夏馨蕊，叶建仁，朱丽华. 抗松针褐斑病湿地松胚性愈伤组织诱导及体胚产量影响因素的研究. 南京林业大学学报(自然科学版). 2025(01): 87-94 .
2.	刘阳，郭文冰，薛蕾，王哲，曾明，欧阳曦，伍彩云，车晓亮. 湿地松抗褐变和易褐变无性系胚性愈伤团的生理生化特征. 林业与环境科学. 2025(01): 9-21 .
3.	王斯彤，张柏习，王曼，王浩，孟鹏. 樟子松无性系(GS1)胚性愈伤组织的初步诱导. 分子植物育种. 2023(18): 6088-6095 .
4.	程方，叶建仁. 抗性湿地松胚性愈伤组织的维持与增殖. 绿色科技. 2023(14): 1-7+27 .
5.	程方，孙婷玉，叶建仁. 抗松针褐斑病湿地松未成熟合子胚胚性愈伤组织的诱导. 南京林业大学学报(自然科学版). 2023(06): 175-182 .
6.	高启阳，黄宇龙，郭文冰，赵奋成，刘阳. 碳源等因素对湿地松优良无性系胚性愈伤组织增殖的影响. 植物研究. 2022(01): 21-28 .
7.	胡珊，杨春霞，谷振军，杜强，肖平江，李火根. 火炬松体细胞胚胎发生体系的优化. 林业科学研究. 2022(03): 9-17 .
8.	徐康，程强强，杨春霞，谷振军，丁伟，李火根. 速生湿地松良种胚性愈伤组织诱导与增殖. 广西植物. 2021(02): 283-291 .
9.	程子珊，易敏，宋才玲，程强强，黄若，邓诏磊，张莹莹，张露. 湿地松体细胞胚胎发生胚性愈伤组织诱导条件优化. 江西农业大学学报. 2021(05): 1054-1064 .

Other cited types(5)

Get Citation

PDF

XML

Article views (139) PDF downloads (323) Cited by(14)

Turn off MathJax

Article Contents

Abstract

References

Detection method of high temperature stress of tomato at seedling stage based on thermal infrared and RGB images

Abstract

References

Cited by

Periodical cited type(9)

Other cited types(5)

Catalog

Export File

Citation

Format

Content