Measurement of tomato fruits quantity at different ripening stages based on color point cloud images

ZHANG Xianjie; WANG Xiaochan; SUN Guoxiang; SHI Yinyan; WEI Tianxiang; CHEN Hao

doi:10.7671/j.issn.1001-411X.202105021

Journal of South China Agricultural University > 2022 > 43(2): 105-112. > DOI: 10.7671/j.issn.1001-411X.202105021

ZHANG Xianjie, WANG Xiaochan, SUN Guoxiang, et al. Measurement of tomato fruits quantity at different ripening stages based on color point cloud images[J]. Journal of South China Agricultural University, 2022, 43(2): 105-112. DOI: 10.7671/j.issn.1001-411X.202105021

Citation:

PDF (1236 KB)

Measurement of tomato fruits quantity at different ripening stages based on color point cloud images

1.
College of Engineering, Nanjing Agricultural University, Nanjing 210031, China
2.
Engineering Laboratory of Modern Facility Agriculture Technology and Equipment in Jiangsu Province, Nanjing 210031, China

More Information

Received Date: May 10, 2021
Available Online: May 17, 2023

Graphical Abstract

Abstract

Abstract

Objective
In order to measure the number of tomato fruits at different ripening stages in greenhouse, a method based on color point cloud images was proposed.

Method
The image information of tomato in greenhouse was collected by KinectV2.0 on the mobile platform to synthesize the tomato plant point cloud, then the tomato plant point clouds from two perspectives were synthesized into a point cloud, and the point cloud of nearby tomato plant was obtained by depth information interception. The labeled point cloud data were input into the PointRCNN object detection network to train the prediction model and recognize tomato fruit in the tomato plant point cloud. Finally, support vector machine(SVM) classifier based on feature matrix training was used to classify ripeness of the identified fruits, and the number of tomato fruits at different ripening stages was obtained.

Result
The precision rate of the method based on PointRCNN object detection network for identifying the number of tomato fruits was 86.19% and the recall rate was 83.39%. The accuracy of SVM classifier based on feature matrix training for predicting the ripeness of tomato fruits was 94.27% in the training set and 96.09% in the test set.

Conclusion
The measurement method based on color point cloud images can accurately identify the number of tomato fruits at different ripening stages, and provide data supports for evaluating the yield of tomato fruits in greenhouse.
- Tomato fruit,
- Color point cloud,
- PointRCNN,
- Support vector machine,
- Ripening stage

FullText(HTML)

References (23)

References

[1]	罗华, 李敏, 胡大刚, 等. 不同有机肥对肥城桃果实产量及品质的影响[J]. 植物营养与肥料学报, 2012, 18(4): 955-964.
[2]	武阳, 王伟, 雷廷武, 等. 调亏灌溉对滴灌成龄香梨果树生长及果实产量的影响[J]. 农业工程学报, 2012, 28(11): 118-124. doi: 10.3969/j.issn.1002-6819.2012.11.020
[3]	GONG A, YU J, HE Y, et al. Citrus yield estimation based on images processed by an android mobile phone [J]Biosystems Engineering, 2013, 115(2): 162-170. GONG A, YU J, HE Y, et al. Citrus yield estimation based on images processed by an android mobile phone[J]. Biosystems Engineering, 2013, 115(2): 162-170.
[4]	刘芳, 刘玉坤, 林森, 等. 基于改进型YOLO的复杂环境下番茄果实快速识别方法[J]. 农业机械学报, 2020, 51(6): 229-237. doi: 10.6041/j.issn.1000-1298.2020.06.024
[5]	王晓楠, 伍萍辉, 冯青春, 等. 番茄采摘机器人系统设计与试验[J]. 农机化研究, 2016, 38(4): 94-98. doi: 10.3969/j.issn.1003-188X.2016.04.020
[6]	SAEDI S I, KHOSRAVI H. A deep neural network approach towards real-time on-branch fruit recognition for precision horticulture[J]. Expert Systems with Applications, 2020, 159: 113594. doi: 10.1016/J.ESWA.2020.113594
[7]	司永胜, 乔军, 刘刚, 等. 苹果采摘机器人果实识别与定位方法[J]. 农业机械学报, 2010, 41(9): 148-153.
[8]	傅隆生, 宋珍珍, ZHANG X, 等. 深度学习方法在农业信息中的研究进展与应用现状[J]. 中国农业大学学报, 2020, 25(2): 105-120. doi: 10.11841/j.issn.1007-4333.2020.02.12
[9]	PRZYBYLO J, JABLONSKI M. Using deep convolutional neural network for oak acorn viability recognition based on color images of their sections[J]. Computers and Electronics in Agriculture, 2019, 156: 490-499.
[10]	车金庆, 王帆, 吕继东, 等. 重叠苹果果实的分离识别方法[J]. 江苏农业学报, 2019, 35(2): 469-475. doi: 10.3969/j.issn.1000-4440.2019.02.030
[11]	崔永杰, 苏帅, 王霞霞, 等. 基于机器视觉的自然环境中猕猴桃识别与特征提取[J]. 农业机械学报, 2013, 44(5): 247-252. doi: 10.6041/j.issn.1000-1298.2013.05.043
[12]	ZHANG Y D, DONG Z C, CHEN X Q, et al. Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation[J]. Multimedia Tools and Applications, 2019, 78(3): 3613-3632. doi: 10.1007/s11042-017-5243-3
[13]	熊俊涛, 郑镇辉, 梁嘉恩, 等. 基于改进YOLO v3网络的夜间环境柑橘识别方法[J]. 农业机械学报, 2020, 51(4): 199-206. doi: 10.6041/j.issn.1000-1298.2020.04.023
[14]	PAYNE A B, WALSH K B, SUBEDI P P, et al. Estimation of mango crop yield using image analysis- Segmentation method[J]. Computers and Electronics in Agriculture, 2013, 91: 57-64. doi: 10.1016/j.compag.2012.11.009
[15]	贾伟宽, 赵德安, 刘晓洋, 等. 机器人采摘苹果果实的K-means和GA-RBF-LMS神经网络识别[J]. 农业工程学报, 2015, 31(18): 175-183. doi: 10.11975/j.issn.1002-6819.2015.18.025
[16]	金保华, 殷长魁, 张卫正, 等. 基于机器视觉的苹果园果实识别研究综述[J]. 轻工学报, 2019, 34(2): 71-81. doi: 10.3969/j.issn.2096-1553.2019.02.010
[17]	SI Y S, LIU G, FENG J. Location of apples in trees using stereoscopic vision[J]. Computers and Electronics in Agriculture, 2015, 112: 68-74. doi: 10.1016/j.compag.2015.01.010
[18]	熊龙烨, 王卓, 何宇, 等. 果树重建与果实识别方法在采摘场景中的应用[J]. 传感器与微系统, 2019, 38(8): 153-156.
[19]	MEHTA S S, BURKS T F. Multi-camera fruit localization in robotic harvesting[J]. IFAC-PapersOnLine, 2016, 49(16): 90-95. doi: 10.1016/j.ifacol.2016.10.017
[20]	李寒, 陶涵虓, 崔立昊, 等. 基于SOM-K-means算法的番茄果实识别与定位方法[J]. 农业机械学报, 2021, 52 (1): 23-29.
[21]	张星, 张双星. 基于point-to-plane ICP的点云与影像数据自动配准[J]. 计算机与数字工程, 2017, 45(12): 2510-2514. doi: 10.3969/j.issn.1672-9722.2017.12.039
[22]	SHI S, WANG X, LI H. PointRCNN: 3D object proposal generation and detection from point cloud[C]//IEEE. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City: IEEE, 2019.
[23]	SAUNDERS C, STITSON M O, WESTON J, et al. Support vector machine: Reference manual[J]. Department Computer Science, 2000, 1(4): 1-28. doi: 10.1007/978-0-387-39940-9_557

Cited By

Cited by

Periodical cited type(4)

1.	宋鹏，李理想，江厚龙，王茹，李慧，赵鹏宇，张均，秦平伟，任江波，陈庆明. 施用侧孢短芽孢杆菌对烤后烟叶钾含量及烟株生理特征的影响. 浙江农业学报. 2024(03): 494-502 .
2.	杜蓉惠，何涛，杜鸿燕，邓维萍，朱书生，杜飞. 枯草芽孢杆菌对‘红地球’葡萄白粉病防效及叶际细菌群落的影响. 中外葡萄与葡萄酒. 2024(03): 38-46 .
3.	李妍，胡斯乐，白晓雄，刘朝斌，张敏，王迎，余旋. 核桃根际耐旱促生菌的分离筛选及其促生作用研究. 西北林学院学报. 2024(03): 84-92 .
4.	吕嘉妍，毛健辉，霍春宇，黄永芳，罗连荷，梁家俊，陈祖静. 广东省本地油茶和引种油茶根际土壤微生物群落特征. 微生物学通报. 2023(11): 4938-4953 .

Other cited types(6)

Get Citation

PDF

XML

Article views (496) PDF downloads (724) Cited by(10)

Turn off MathJax

Article Contents

Abstract

References

Measurement of tomato fruits quantity at different ripening stages based on color point cloud images

Abstract

References

Cited by

Periodical cited type(4)

Other cited types(6)

Catalog

Export File

Citation

Format

Content