Apple fruit recognition at different maturity stages based on image fusion
-
摘要:目的
针对复杂农业环境中不同成熟阶段苹果目标识别困难的问题，研究一种基于图像融合的苹果识别算法。
方法采用保边性能较好的均值漂移滤波对图像进行预处理，滤除少量背景噪声。分别从RGB颜色空间和YIQ颜色空间提取R−G分量和I分量特征图像，采用像素级图像融合算法融合2幅特征图像信息，突出显示果实目标区域。利用Otsu自适应阈值算法获得最佳阈值，将目标苹果从背景中分割出来。为识别苹果目标，提出一种基于改进梯度场的Hough变换圆检测算法，通过引入形态学重建算法清理背景中残留的小面积区域，提高检测效率；同时以分割的苹果二值图像为判断标准构造剔除虚假圆算法，避免检测出现虚假目标。
结果对采集到的50幅未完全成熟的苹果图像和50幅完全成熟的苹果图像进行识别，并与最小外接圆法进行对比，试验结果表明，本文算法平均识别时间为0.367 s，对完全裸露果实、被遮挡面积≤1/2果实和被遮挡面积>1/2果实的识别正确率分别为100%、92.46%，和81.87%，整体识别准确率比最小外接圆算法提高了11.43个百分点。本文算法圆心相对误差均值和半径相对误差均值分别为0.216和0.048%，最小外接圆算法圆心相对误差均值和半径相对误差均值分别为0.508和0.370%。
结论本文提出的方法能够快速识别苹果目标，具有较高精度和效率的果实定位，可以服务于苹果采摘机器人进行果实采摘。
Abstract:ObjectiveTo address the challenge of apple target recognition in different mature stages in a complex agricultural environment, an apple recognition algorithm based on image fusion was studied.
MethodA mean-shift filter with good edge-preserving performance was used to preprocess the image and filter out small parts of background noise. The R−G component and the I component feature images were extracted from RGB color space and YIQ color space, respectively. The pixel-level image fusion algorithm was used to fuse the information of two feature images to highlight the fruit target area. The Otsu adaptive threshold algorithm was used to obtain the best threshold to segment the target apple from the background. In order to identify the apple target, an improved Hough transform circle detection algorithm based on gradient fields was proposed. The morphological reconstruction algorithm was introduced to clean up the small area remaining in the background so as to improve the detection efficiency. At the same time, the algorithm to eliminate false circles was constructed using the segmented binary image of apple as the judgment standard, so as to avoid false targets.
ResultFifty images of immature apples and fifty images of fully matured apples were recognized and compared with the minimum circumscribed circle method. The experiment results showed that the average recognition time of the algorithm in this paper was 0.367 s, and the recognition accuracy for completely exposed fruits was 100%, 92.46% for fruits with occluded areas ≤ 1/2, 81.87% for fruits with occluded areas > 1/2. The overall recognition accuracy was increased by 11.43 percentage points compared with that of the minimum circumscribed circle algorithm. The mean center relative error and the mean radius relative error of the algorithm in this paper were 0.216 and 0.048% respectively, and the mean center relative error and the mean radius relative error of the minimum circumscribed circle algorithm were 0.508 and 0.370% respectively.
ConclusionThe method proposed in this paper can quickly identify the apple target, locate the fruit with high accuracy and efficiency, and can be used for fruit picking by the apple picking robot.
-
Keywords:
- Apple /
- Image fusion /
- Hough transform /
- Ripening stage
-
![]()
图 1 均值漂移滤波前后图像对比
Figure 1. Comparison of the images before and after mean shift filtering
![]()
图 2 均值漂移滤波前后能量图对比
a1、a2分别为滤波前、后的未完全成熟果实；b1、b2分别为滤波前、后的完全成熟果实
Figure 2. Comparison of energy diagrams before and after mean shift filtering
a1 and a2 are immature fruits before and after filtering respectively; b1 and b2 are fully matured fruits before and after filtering respectively
![]()
图 3 未完全成熟果实(a)和完全成熟果实(b) 归一化后的R−G分量特征图像
Figure 3. Normalized R−G component feature images of immature fruits (a) and fully matured fruits (b)
![]()
图 4 未完全成熟果实(a)和完全成熟果实(b) 归一化后的I分量特征图像
Figure 4. Normalized I component feature images of immature fruits (a) and fully matured fruits (b)
![]()
图 5 未完全成熟果实(a)和完全成熟果实(b) 融合后的图像
Figure 5. Fusion images of immature fruits (a) and fully matured fruits (b)
![]()
图 6 未完全成熟果实(a)和完全成熟果实(b) Otsu分割效果
Figure 6. Otsu segmentation effects of immature fruits (a) and fully matured fruits (b)
![]()
图 7 形态学重建滤波处理前后图像对比
Figure 7. Comparison of images before and after morphological reconstruction filtering
![]()
图 8 未完全成熟果实(a)和完全成熟果实(b) 累加矩阵3D视图
Figure 8. Cumulative matrix 3D views of immature fruits (a) and fully matured fruits (b)
![]()
图 9 未完全成熟果实(a)和完全成熟果实(b) 卷积滤波结果
Figure 9. Convolution filtering results of immature fruits (a) and fully matured fruits (b)
![]()
图 10 未完全成熟果实(a)和完全成熟果实(b) 圆心检测图
Figure 10. Center detection images of immature fruits (a) and fully matured fruits (b)
![]()
图 11 未完全成熟果实(a)和完全成熟果实(b) Hough变换算法识别结果
Figure 11. Hough transform algorithm recognition results of immature fruits (a) and fully matured fruits (b)
![]()
图 12 未完全成熟果实(a)和完全成熟果实(b) 虚假目标去除结果
Figure 12. False target removal results of immature fruits (a) and fully matured fruits (b)
![]()
图 14 最小外接圆算法识别结果
a1、a2和a3：未完全成熟果实；b1、b2和b3：完全成熟果实
Figure 14. Recognition result of the minimum circumscribed circle algorithm
a1, a2 and a3: Immature fruits; b1, b2 and b3: Fully matured fruits
![]()
图 15 本文算法识别结果
a1、a2和a3：未完全成熟果实；b1、b2和b3：完全成熟果实
Figure 15. Recognition results of the algorithm in this paper
a1, a2 and a3: Immature fruits; b1, b2 and b3: Fully matured fruits
表 1 2种算法识别时间及正确率对比
Table 1 Comparison of recognition time and accuracy of two algorithms
算法
Algorithm成熟度
Maturity识别时间/s
Recognition time识别正确率/%
Recognition accuracy本文方法
Algorithm in this paper未完全成熟 Immature 0.351 91.92 完全成熟 Fully matured 0.382 90.96 平均 Average 0.367 91.44 最小外接圆法
The minimum circumscribed
circle algorithm未完全成熟 Immature 0.632 80.61 完全成熟 Fully matured 0.675 79.58 平均 Average 0.654 80.09 
下载: 导出CSV
表 2 不同遮挡程度的果实识别正确率对比
Table 2 Comparison of recognition accuracy of fruits with different occlusion degrees
算法
Algorithm成熟度
Maturity完全裸露果实
Completely exposed fruits被遮挡面积≤1/2果实
Fruits with ≤ 1/2 covered area被遮挡面积>1/2果实
Fruits with > 1/2 covered area数量
Quantity识别数量
Recognition
quantity识别正确率/%
Recognition
accuracy数量
Quantity识别数量
Recognition
quantity识别正确率/%
Recognition
accuracy数量
Quantity识别数量
Recognition
quantity识别正确率/%
Recognition
accuracy本文方法
Algorithm in
this paper未完全成熟
Immature307 307 100 112 105 93.75 67 55 82.01 完全成熟
Fully matured283 283 100 147 134 91.16 93 76 81.72 最小外接圆法
The minimum
circumscribed
circle algorithm未完全成熟
Immature307 307 100 112 92 82.14 67 40 59.70 完全成熟
Fully matured283 283 100 147 117 79.59 93 55 59.14
下载: 导出CSV
表 3 2种算法圆心与半径相对误差对比
Table 3 Comparison of the relative errors of the center and radius between two algorithms
算法
Algorithm成熟度
Maturity平均圆心坐标误差/%
Mean center coordinate error平均半径误差/%
Mean radius error本文方法
Algorithm in this paper未完全成熟 Immature 0.226 0.051 完全成熟 Fully matured 0.205 0.044 平均值 Mean 0.216 0.048 最小外接圆法
The minimum circumscribed
circle algorithm未完全成熟 Immature 0.574 0.162 完全成熟 Fully matured 0.441 0.578 平均值 Mean 0.508 0.370
下载: 导出CSV
-
[1] ZHANG R F, WANG C, HU X P, et al. Weed location and recognition based on UAV imaging and deep learning[J]. International Journal of Precision Agricultural Aviation, 2020, 3(1): 23-29.
[2] WANG L L, XIAO W W, QI Y, et al. Farmland human-shape obstacles identification based on Viola-Jones algorithm[J]. International Journal of Precision Agricultural Aviation, 2020, 3(3): 35-40.
[3] CAI N, ZHOU X G, YANG Y B, et al. Use of UAV images to assess narrow brown leaf spot severity in rice[J]. International Journal of Precision Agricultural Aviation, 2019, 2(2): 38-42.
[4] 李娜, 陈宁. 自然场景下苹果采摘机器人视觉系统研究[J]. 计算机技术与发展, 2018, 28(12): 137-141. [5] SI Y, QIAO J, LIU G, et al. Recognition and location of fruits for apple harvesting robot[J]. Transactions of the Chinese Society for Agricultural Machinery, 2010, 41(9): 148-153.
[6] ARIVAZHAGAN S, SHEBIAH R N, NIDHYANANDHAN S S, et al. Fruit recognition using color and texture features[J]. Journal of Emerging Trends in Computing & Information Sciences, 2010, 1(2): 90-94.
[7] 王丹丹, 宋怀波, 何东健. 苹果采摘机器人视觉系统研究进展[J]. 农业工程学报, 2017, 33(10): 59-69. [8] 金保华, 殷长魁, 张卫正, 等. 基于机器视觉的苹果园果实识别研究综述[J]. 轻工学报, 2019, 34(2): 71-81. [9] SI Y, LIU G, FENG J. Location of apples in trees using stereoscopic vision[J]. Computers and Electronics in Agriculture, 2015, 112: 68-74. doi: 10.1016/j.compag.2015.01.010
[10] GONGAL A, SILWAL A, AMATYA S, et al. Apple crop-load estimation with over-the-row machine vision system[J]. Computers and Electronics in Agriculture, 2016, 120: 26-35. doi: 10.1016/j.compag.2015.10.022
[11] 吕继东, 赵德安, 姬伟. 苹果采摘机器人目标果实快速跟踪识别方法[J]. 农业机械学报, 2014, 45(1): 65-72. [12] 魏亚辉, 黄耿楠, 吴福培. 基于改进分水岭算法的苹果识别方法[J]. 包装工程, 2021, 42(8): 255-260. [13] 钱建平, 杨信廷, 吴晓明, 等. 自然场景下基于混合颜色空间的成熟期苹果识别方法[J]. 农业工程学报, 2012, 28(17): 137-142. [14] JI W, ZHAO D A, CHENG F Y, et al. Automatic recognition vision system guided for apple harvesting robot[J]. Computers and Electrical Engineering, 2012, 38(5): 1186-1195. doi: 10.1016/j.compeleceng.2011.11.005
[15] LV J D. Recognition of overlapping and occluded fruits in natural environment[J]. Journal of Computational and Theoretical Nanoscience, 2016, 127(3): 2475-2484.
[16] XU L M, LV J D. Recognition method for apple fruit based on SUSAN and PCNN[J]. Multimedia Tools and Applications, 2018, 77(6): 7205-7219. doi: 10.1007/s11042-017-4629-6
[17] 代沁伶. 边缘保持图像滤波的应用研究[D]. 武汉: 武汉大学, 2018. [18] HUANG L W, HE D J. Fuji apple detection model analysis in natural tree canopy[J]. TELKOMNIKA Indonesian Journal of Electrical Engineering, 2012, 10(7): 1771-1778.
[19] 于慧杰, 李大华, 高强, 等. 自然环境中重叠与遮挡绿苹果图像的识别[J]. 激光杂志, 2020, 41(2): 20-24. [20] ZHAO Y S, GONG L, HUANG Y X, et al. Robust tomato recognition for robotic harvesting using feature images fusion[J]. Sensors, 2016, 16(2): 173. doi: 10.3390/s16020173
[21] 奔粤阳, 汤瑞, 戴平安, 等. 基于加权融合的水下视觉图像增强算法[J/OL]. 北京航空航天大学学报, (2022-09-14)[ 2022-12-01]. https://doi.org/10.13700/j.bh.1001-5965.2022.0540. [22] 黄士凯, 祁力钧, 张建华, 等. 基于行宽的玉米行间杂草识别算法[J]. 中国农业大学学报, 2013, 18(1): 165-171. [23] LI S, KANG X, FANG L, et al. Pixel-level image fusion: A survey of the state of the art[J]. Information Fusion, 2017, 33: 100-112. doi: 10.1016/j.inffus.2016.05.004
[24] 孙俊, 宋佳, 武小红, 等. 基于改进Otsu算法的生菜叶片图像分割方法[J]. 江苏大学学报(自然科学版), 2018, 39(2): 179-184. [25] 马聪, 陈学东, 周慧. 基于改进RHT及均值漂移聚类方法的双孢菇图像目标提取研究[J]. 中国农机化学报, 2022, 43(11): 195-202. [26] 程鹏, 朱美琳, 耿华. 一种基于梯度Hough变换和SVM的圆检测算法[J]. 计算机与现代化, 2013(2): 22-26. [27] 吴庆岗, 张卫国, 常化文, 等. 基于梯度Hough变换的遮挡苹果目标定位[J]. 浙江农业学报, 2017, 29(6): 1009-1016. -
期刊类型引用(3)
1. 徐文青，丁爱霞，朱文卿. 基于数字图像处理技术的水果分级检测研究进展. 食品安全导刊. 2025(04): 183-189 . 
百度学术
2. 李彦龙，王加泽，杨乐平，张健滔. 基于深度相机的机器人目标定位. 工业控制计算机. 2025(03): 19-21 . 百度学术
3. 王明友，王达，宋卫东，孙玉利，张振烨，赵鑫培. 菇类采摘机器人研究现状和展望. 农业机械学报. 2024(S1): 1-8 . 百度学术
其他类型引用(6)
