Recognition of mango skin defect based on semantic segmentation

LIU Xiaogang; LI Rongmei; FAN Cheng; YANG Qiliang; ZHAO Lu

doi:10.7671/j.issn.1001-411X.202204014

Journal of South China Agricultural University > 2023 > 44(1): 134-141. > DOI: 10.7671/j.issn.1001-411X.202204014

LIU Xiaogang, LI Rongmei, FAN Cheng, et al. Recognition of mango skin defect based on semantic segmentation[J]. Journal of South China Agricultural University, 2023, 44(1): 134-141. DOI: 10.7671/j.issn.1001-411X.202204014

Citation:

PDF (3739 KB)

Recognition of mango skin defect based on semantic segmentation

1.
Faculty of Modern Agricultural Engineering, Kunming University of Science and Technology, Kunming 650000, China
2.
Zoomlion Intelligent Technology Limited Company, Changsha 410013, China
3.
College of Water Resources and Hydropower, Sichuan University, Chengdu 610065, China

More Information

Received Date: April 21, 2022
Available Online: May 17, 2023

Graphical Abstract

Abstract

Abstract

Objective
The semantic segmentation technology was used to automatically identify mango and its skin defects, to realize the quality evaluation and sorting of mango and provide a reference for the rapid and nondestructive testing of mango quality.

Method
Mango skin defect images in multi-scene of natural environment were collected for model training and testing. Atrous spatial pyramid pooling (ASPP) in DeepLabV3+ was replaced by joint pyramid upsampling (JPU) structure, and Xception model in DeepLabV3+ was replaced by Atrous-ResNet model. Class pixel accuracy (CPA), mean pixel accuracy (MPA) and mean intersection over union (MIoU) were used as the accuracy evaluation indexes of each model.

Result
JPU module was used to replace ASPP module, and Atrous convolution was applied to ResNet network which was conductive to increase the receptive field of the model. In general, the predicted boundary was smoother, and the identification of small defects was more accurate. The comparison with SegNet and LinkNet algorithms showed that Atrous-ResNet model had higher accuracy, with CPA slightly improved, MPA was up 3.79 percent point and MIoU was up 4.57 percent. Atrous-ResNet model had better identification effects.

Conclusion
The method based on semantic segmentation is feasible for mango skin defect recognition. Compared with SegNet and LinkNet algorithms, Atrous-ResNet model has higher recognition accuracy.
- Semantic segmentation,
- DeepLabV3+,
- Joint pyramid upsampling,
- Defect recognition,
- Mango skin

FullText(HTML)

References (28)

References

[1]	DEEPA M, PUSHPA B, UDAYKUMAR K. Physicochemical properties, nutritional and antinutritional composition of pulp and peel of three mango varieties[J]. International Journal of Educational Science and Research, 2017, 7(3): 81-94.
[2]	KANGD, WANGY, FANY, et al. Research and development of Camellia oleifera fruit sheller and sorting machine[J]. Earth and Environmental Science, 2018, 108: 042051. doi: 10.1088/1755-1315/108/4/042051.
[3]	SIHOMBING P, TOMMY F, SEMBIRING S, et al. The Citrus fruit sorting device automatically based on color method by using tcs320 color sensor and arduino uno microcontroller[J]. Journal of Physics: Conference Series, 2019, 1235: 012064. doi: 10.1088/1742-6596/1235/1/012064.
[4]	邓继忠, 任高生, 兰玉彬, 等. 基于可见光波段的无人机超低空遥感图像处理[J]. 华南农业大学学报, 2016, 37(6): 16-22. doi: 10.7671/j.issn.1001-411X.2016.06.003
[5]	戴泽翰, 郑正, 黄莉舒, 等. 基于深度卷积神经网络的柑橘黄龙病症状识别[J]. 华南农业大学学报, 2020, 41(4): 111-119. doi: 10.7671/j.issn.1001-411X.201909031
[6]	赵德安, 吴任迪, 刘晓洋, 等. 基于YOLO深度卷积神经网络的复杂背景下机器人采摘苹果定位[J]. 农业工程学报, 2019, 35(3): 164-173. doi: 10.11975/j.issn.1002-6819.2019.03.021
[7]	刘小刚, 范诚, 李加念, 等. 基于卷积神经网络的草莓识别方法[J]. 农业机械学报, 2020, 51(2): 237-244. doi: 10.6041/j.issn.1000-1298.2020.02.026
[8]	XING S L, LEE M. Classification accuracy improvement for small-size Citrus pests and diseases using bridge connections in deep neural networks[J]. Sensors, 2020, 20(17): 4992. doi: 10.3390/s20174992.
[9]	SELVARAJ M G, VERGARA A, RUIZ H, et al. AI-powered banana diseases and pest detection[J]. Plant Methods, 2019, 15(1): 92. doi: 10.1186/s13007-018-0385-5
[10]	PATEL K K, KAR A, KHAN M A. Common external defect detection of mangoes using color computer vision[J]. Journal of the Institution of Engineers: Series A, 2019, 100(4): 559-568. doi: 10.1007/s40030-019-00396-6
[11]	刘平, 朱衍俊, 张同勋, 等. 自然环境下贴叠葡萄串的识别与图像分割算法[J]. 农业工程学报, 2020, 36(6): 161-169. doi: 10.11975/j.issn.1002-6819.2020.06.019
[12]	李江波, 彭彦昆, 黄文倩, 等. 桃子表面缺陷分水岭分割方法研究[J]. 农业机械学报, 2014, 45(8): 288-293. doi: 10.6041/j.issn.1000-1298.2014.08.046
[13]	张德军, 周学成, 杨旭东. 基于图像处理和深度迁移学习的芒果果实病状识别[J]. 华南农业大学学报, 2021, 42(4): 113-124. doi: 10.7671/j.issn.1001-411X.202011002
[14]	袁培森, 黎薇, 任守纲, 等. 基于卷积神经网络的菊花花型和品种识别[J]. 农业工程学报, 2018, 34(5): 152-158. doi: 10.11975/j.issn.1002-6819.2018.05.020
[15]	尚增强, 杨东福, 马质璞. 基于深度卷积神经网络的大豆叶片多种病害分类识别[J]. 大豆科学, 2021, 40(5): 662-668.
[16]	程曦, 吴云志, 张友华, 等. 基于深度卷积神经网络的储粮害虫图像识别[J]. 中国农学通报, 2018, 34(1): 154-158. doi: 10.11924/j.issn.1000-6850.casb16110146
[17]	JHURIA M, KUMAR A, BORSE R. Image processing for smart farming: Detection of disease and fruit grading[C]//IEEE 20^th International Conference on Image Information Processing. Shimla, India: IEEE, 2013: 521-526.
[18]	BASAVARAJ T, BHAVANA S. Banana plant disease detection and grading using image processing[J]. International Journal of Engineering Science and Computing, 2016: 6512-6516.
[19]	SAHU D, POTDAR M. Defect identification and maturity detection of mango fruits using image analysis[J]. American Journal of Artificial Intelligence, 2017, 1(1): 5-14.
[20]	HUANG X Y, LÜ R Q, WANG S, et al. Integration of computer vision and colorimetric sensor array for nondestructive detection of mango quality[J]. Journal of Food Process Engineering, 2018, 41(8): 1-9.
[21]	SUDARJAT, KUSUMIYATI, HASANUDDIN, et al. Rapid and non-destructive detection of insect infestations on intact mango by means of near infrared spectroscopy[J]. IPO Conference Series: Earth and Environmental Science, 2019, 365(1): 012037. doi: 10.1088/1755-1315/365/1/012037.
[22]	KESTUR R, MEDURI A, NARASIPURA O. MangoNet: A deep semantic segmentation architecture for a method to detect and count mangoes in an open orchard[J]. Engineering Applications of Artificial Intelligence, 2018, 77: 59-69.
[23]	SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. Proceedings of the IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. doi: 10.1109/TPAMI.2016.2572683
[24]	CHOLLET F. Xception: Deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017: 1800-1807
[25]	WU H K, ZHANG J G, HUANG K Q, et al. FastFCN: Rethinking dilated convolution in the backbone for semantic segmentation[EB/OL]. ArXiv preprint arXiv, 2019: 1903.11816. (2019-03-28)[2022-03-20]. https://ariv.org/abs/1903.11816.
[26]	CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[M]//Computer Vision ECCV. Cham: Springer, 2018: 833-851
[27]	CHAURASIA A, CULURCIELLO E. LinkNet: Exploiting encoder representations for efficient semantic segmentation[J]. IEEE Visual Communications and Image Processing (VCIP), 2017: 1-4. doi: 10.1109/VCIP.2017.8305148.
[28]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation[J]. Proceedings of the IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. doi: 10.1109/TPAMI.2016.2644615

Cited By

Get Citation

PDF

XML

Article views (124) PDF downloads (283)

Turn off MathJax

Article Contents

Abstract

References

Recognition of mango skin defect based on semantic segmentation

Abstract

References

Catalog

Export File

Citation

Format

Content