Apple fruit recognition at different maturity stages based on image fusion

LIU Mingyang; CUI Kai; GONG Jinliang; ZHANG Yanfei

doi:10.7671/j.issn.1001-411X.202212020

Journal of South China Agricultural University > 2024 > 45(2): 293-303. > DOI: 10.7671/j.issn.1001-411X.202212020

LIU Mingyang, CUI Kai, GONG Jinliang, et al. Apple fruit recognition at different maturity stages based on image fusion[J]. Journal of South China Agricultural University, 2024, 45(2): 293-303. DOI: 10.7671/j.issn.1001-411X.202212020

Citation:

PDF (8205 KB)

Apple fruit recognition at different maturity stages based on image fusion

1.
School of Agricultural Engineering and Food Science, Shandong University of Technology, Zibo 255000, China
2.
Nanjing Dragonfly Smart Agriculture Research Co., Ltd., Nanjing 210019, China
3.
School of Mechanical Engineering, Shandong University of Technology, Zibo 255000, China

More Information

Received Date: December 15, 2022
Available Online: December 15, 2023
Published Date: December 14, 2023

Graphical Abstract

Abstract

Abstract

Objective
To address the challenge of apple target recognition in different mature stages in a complex agricultural environment, an apple recognition algorithm based on image fusion was studied.
Method
A mean-shift filter with good edge-preserving performance was used to preprocess the image and filter out small parts of background noise. The R−G component and the I component feature images were extracted from RGB color space and YIQ color space, respectively. The pixel-level image fusion algorithm was used to fuse the information of two feature images to highlight the fruit target area. The Otsu adaptive threshold algorithm was used to obtain the best threshold to segment the target apple from the background. In order to identify the apple target, an improved Hough transform circle detection algorithm based on gradient fields was proposed. The morphological reconstruction algorithm was introduced to clean up the small area remaining in the background so as to improve the detection efficiency. At the same time, the algorithm to eliminate false circles was constructed using the segmented binary image of apple as the judgment standard, so as to avoid false targets.
Result
Fifty images of immature apples and fifty images of fully matured apples were recognized and compared with the minimum circumscribed circle method. The experiment results showed that the average recognition time of the algorithm in this paper was 0.367 s, and the recognition accuracy for completely exposed fruits was 100%, 92.46% for fruits with occluded areas ≤ 1/2, 81.87% for fruits with occluded areas > 1/2. The overall recognition accuracy was increased by 11.43 percentage points compared with that of the minimum circumscribed circle algorithm. The mean center relative error and the mean radius relative error of the algorithm in this paper were 0.216 and 0.048% respectively, and the mean center relative error and the mean radius relative error of the minimum circumscribed circle algorithm were 0.508 and 0.370% respectively.
Conclusion
The method proposed in this paper can quickly identify the apple target, locate the fruit with high accuracy and efficiency, and can be used for fruit picking by the apple picking robot.
- Apple,
- Image fusion,
- Hough transform,
- Ripening stage

FullText(HTML)

References (27)

References

[1]	ZHANG R F, WANG C, HU X P, et al. Weed location and recognition based on UAV imaging and deep learning[J]. International Journal of Precision Agricultural Aviation, 2020, 3(1): 23-29.
[2]	WANG L L, XIAO W W, QI Y, et al. Farmland human-shape obstacles identification based on Viola-Jones algorithm[J]. International Journal of Precision Agricultural Aviation, 2020, 3(3): 35-40.
[3]	CAI N, ZHOU X G, YANG Y B, et al. Use of UAV images to assess narrow brown leaf spot severity in rice[J]. International Journal of Precision Agricultural Aviation, 2019, 2(2): 38-42.
[4]	李娜, 陈宁. 自然场景下苹果采摘机器人视觉系统研究[J]. 计算机技术与发展, 2018, 28(12): 137-141.
[5]	SI Y, QIAO J, LIU G, et al. Recognition and location of fruits for apple harvesting robot[J]. Transactions of the Chinese Society for Agricultural Machinery, 2010, 41(9): 148-153.
[6]	ARIVAZHAGAN S, SHEBIAH R N, NIDHYANANDHAN S S, et al. Fruit recognition using color and texture features[J]. Journal of Emerging Trends in Computing & Information Sciences, 2010, 1(2): 90-94.
[7]	王丹丹, 宋怀波, 何东健. 苹果采摘机器人视觉系统研究进展[J]. 农业工程学报, 2017, 33(10): 59-69.
[8]	金保华, 殷长魁, 张卫正, 等. 基于机器视觉的苹果园果实识别研究综述[J]. 轻工学报, 2019, 34(2): 71-81.
[9]	SI Y, LIU G, FENG J. Location of apples in trees using stereoscopic vision[J]. Computers and Electronics in Agriculture, 2015, 112: 68-74. doi: 10.1016/j.compag.2015.01.010
[10]	GONGAL A, SILWAL A, AMATYA S, et al. Apple crop-load estimation with over-the-row machine vision system[J]. Computers and Electronics in Agriculture, 2016, 120: 26-35. doi: 10.1016/j.compag.2015.10.022
[11]	吕继东, 赵德安, 姬伟. 苹果采摘机器人目标果实快速跟踪识别方法[J]. 农业机械学报, 2014, 45(1): 65-72.
[12]	魏亚辉, 黄耿楠, 吴福培. 基于改进分水岭算法的苹果识别方法[J]. 包装工程, 2021, 42(8): 255-260.
[13]	钱建平, 杨信廷, 吴晓明, 等. 自然场景下基于混合颜色空间的成熟期苹果识别方法[J]. 农业工程学报, 2012, 28(17): 137-142.
[14]	JI W, ZHAO D A, CHENG F Y, et al. Automatic recognition vision system guided for apple harvesting robot[J]. Computers and Electrical Engineering, 2012, 38(5): 1186-1195. doi: 10.1016/j.compeleceng.2011.11.005
[15]	LV J D. Recognition of overlapping and occluded fruits in natural environment[J]. Journal of Computational and Theoretical Nanoscience, 2016, 127(3): 2475-2484.
[16]	XU L M, LV J D. Recognition method for apple fruit based on SUSAN and PCNN[J]. Multimedia Tools and Applications, 2018, 77(6): 7205-7219. doi: 10.1007/s11042-017-4629-6
[17]	代沁伶. 边缘保持图像滤波的应用研究[D]. 武汉: 武汉大学, 2018.
[18]	HUANG L W, HE D J. Fuji apple detection model analysis in natural tree canopy[J]. TELKOMNIKA Indonesian Journal of Electrical Engineering, 2012, 10(7): 1771-1778.
[19]	于慧杰, 李大华, 高强, 等. 自然环境中重叠与遮挡绿苹果图像的识别[J]. 激光杂志, 2020, 41(2): 20-24.
[20]	ZHAO Y S, GONG L, HUANG Y X, et al. Robust tomato recognition for robotic harvesting using feature images fusion[J]. Sensors, 2016, 16(2): 173. doi: 10.3390/s16020173
[21]	奔粤阳, 汤瑞, 戴平安, 等. 基于加权融合的水下视觉图像增强算法[J/OL]. 北京航空航天大学学报, (2022-09-14)[ 2022-12-01]. https://doi.org/10.13700/j.bh.1001-5965.2022.0540.
[22]	黄士凯, 祁力钧, 张建华, 等. 基于行宽的玉米行间杂草识别算法[J]. 中国农业大学学报, 2013, 18(1): 165-171.
[23]	LI S, KANG X, FANG L, et al. Pixel-level image fusion: A survey of the state of the art[J]. Information Fusion, 2017, 33: 100-112. doi: 10.1016/j.inffus.2016.05.004
[24]	孙俊, 宋佳, 武小红, 等. 基于改进Otsu算法的生菜叶片图像分割方法[J]. 江苏大学学报(自然科学版), 2018, 39(2): 179-184.
[25]	马聪, 陈学东, 周慧. 基于改进RHT及均值漂移聚类方法的双孢菇图像目标提取研究[J]. 中国农机化学报, 2022, 43(11): 195-202.
[26]	程鹏, 朱美琳, 耿华. 一种基于梯度Hough变换和SVM的圆检测算法[J]. 计算机与现代化, 2013(2): 22-26.
[27]	吴庆岗, 张卫国, 常化文, 等. 基于梯度Hough变换的遮挡苹果目标定位[J]. 浙江农业学报, 2017, 29(6): 1009-1016.

Cited By

Cited by

Periodical cited type(3)

1.	徐文青，丁爱霞，朱文卿. 基于数字图像处理技术的水果分级检测研究进展. 食品安全导刊. 2025(04): 183-189 .
2.	李彦龙，王加泽，杨乐平，张健滔. 基于深度相机的机器人目标定位. 工业控制计算机. 2025(03): 19-21 .
3.	王明友，王达，宋卫东，孙玉利，张振烨，赵鑫培. 菇类采摘机器人研究现状和展望. 农业机械学报. 2024(S1): 1-8 .

Other cited types(6)

Get Citation

PDF

XML

Article views (209) PDF downloads (49) Cited by(9)

Turn off MathJax

Article Contents

Abstract

References

Apple fruit recognition at different maturity stages based on image fusion

Abstract

References

Cited by

Periodical cited type(3)

Other cited types(6)

Catalog

Export File

Citation

Format

Content