A corn silk detection method based on MF-SSD convolutional neural network
-
摘要:目的
玉米穗丝是玉米的授粉器官，生长发育状况会影响玉米的产量。为了在玉米生长状态监测和产量预测工作中实时准确识别玉米穗丝，提出一种基于多特征融合SSD (MF-SSD)卷积神经网络的玉米穗丝检测模型。
方法基于特征图对玉米穗丝进行检测，在VGG16-SSD的基础上，用MobileNet替换特征提取器，加入多层特征融合结构，得到MF-SSD网络模型；通过网络优化调整，试验了MF-SSD-cut-3、MF-SSD和MF-SSD-add-3共3种网络结构，优选出检测性能最好的网络结构用于玉米穗丝检测。基于玉米穗丝图像数据集，应用0~180°随机旋转原始图像和水平翻转、平移原始图像2种数据增广技术提升模型训练效果。对是否使用二次训练策略和是否使用Focal loss解决样本不平衡问题进行了试验，并对比分析Loss的下降过程。
结果通过加入多层特征融合结构对SSD模型改进后能够提高网络的检测能力，提升识别速度。与VGG16-SSD相比，MF-SSD在交并比指标方面的平均精度提高7.2%，对玉米穗丝小目标检测的平均召回率提高19.6%，检测速度最高能提升18.7%。在存储空间和运行时间有较高要求的嵌入式环境下，MF-SSD-cut-3模型在满足检测效果的前提下，以较小的空间代价获得了相对较短的运行时间；在不考虑空间和时间因素的情况下，MF-SSD模型获得更好的检测效果。二次训练策略提高了网络的收敛速度和模型的稳定性；Focal loss有效解决了SSD算法中正负样本数量不平衡问题，使网络模型的训练更容易收敛。
结论MF-SSD模型对小目标的检测能力能满足农业生产中对玉米穗丝的实时检测需要，可以用于玉米生长状态的自动监控和产量的精准预测。
Abstract:ObjectiveCorn silk is the pollination organ of maize, its growth state will affect the yield of corn. In order to identify the corn silk in real-time and accurately in corn growth state monitoring and yield prediction, a corn silk detection model based on multi-feature fusion SSD (MF-SSD) convolutional neural network was proposed.
MethodCorn silk detection was based on feature images. The MF-SSD network model was modified from VGG16-SSD through replacing feature extractor by MobileNet and integrating multi-layer feature fusion structure. By optimizing and adjusting the network, three kinds of MF-SSD with different network structures (MF-SSD-cut-3, MF-SSD and MF-SSD-add-3) were tested, and the structure with the best detection performance was selected for corn silk detection. Based on the image data set of corn silk, two kinds of data augmentation techniques (randomly rotating original image from 0 to 180°, horizontally rolling over or translating original image) were applied to improve the training effect of the model. Whether using secondary training strategy and Focal loss to solve sample number imbalance was investigated, and the decrease process of Loss was compared and analyzed.
ResultThe improved SSD model added with multi-layer feature fusion structure could improve the detection ability and recognition speed of network. Compared with VGG16-SSD, the average accuracy of intersection over union increased by 7.2%, the average recall of small target detection of corn silk increased by 19.6%, and the detection speed increased by 18.7%. In the embedded environment having high demand for storage space and run time, MF-SSD-cut-3 obtained shorter run time with smaller storage space in the premise of satisfying detection effect. MF-SSD obtained better detection effect in the condition of taking no account of storage space and run time. The secondary training strategy improved the network convergence speed and model stability. Focal loss effectively solved number imbalance problem of positive and negative samples, and made the training of network model more convergent.
ConclusionThe detection effect of the proposed MF-SSD model for small targets can meet the needs of real-time detection of corn silk in agricultural production. It can be used for automatic monitoring of corn growth state and yield prediction.
-
Keywords:
- corn silk /
- object detection /
- convolutional neural network /
- feature fusion /
- MF-SSD
-
-
![]()
图 5 MF-SSD的网络结构
C3、C5、C11、C13表示卷积层，M3、M5、M11、M13表示融合结果，P3、P5、P11、P13表示特征图结果
Figure 5. MF-SSD network structure
C3, C5, C11 and C13 indicate convolution layers; M3, M5, M11 and M13 indicate fusion results; P3, P5, P11 and P13 indicate feature map results
![]()
图 6 横向连接过程
C11表示卷积层，M11、M13表示融合结果
Figure 6. Horizontal connection process
C11 indicates convolution layer; M11 and M13 indicate fusion results
![]()
图 7 MF-SSD-add-3、MF-SSD和MF-SSD-cut-3的特征提取结构
C2、C3、C4、C5、C7、C8、C10、C11、C13、C14、C16表示卷积层
Figure 7. Feature extraction structures of MF-SSD-add-3, MF-SSD and MF-SSD-cut-3
C2, C3, C4, C5, C7, C8, C10, C11, C13, C14 and C16 indicate convolution layers
![]()
图 8 4种不同类型的样本
黄色框：正确的穗丝目标标记框；红色框：目标检测的预测框，1：简单正例，2：困难正例，3：困难负例，4：简单负例
Figure 8. Samples of four different types
Yellow box: Correct marker box of silk; Red box: Predictive box of target detection, 1: Simple positive sample, 2: Difficult positive sample, 3: Difficult negative sample, 4: Simple negative sample
![]()
图 10 MF-SSD检测结果示例
绿色方框表示网络给出的目标区域，方框上方的百分数表示目标检测器检测到目标区域为玉米穗丝的概率
Figure 10. MF-SSD test result example
Green rectangle indicates the object area provided by network, the percentage above the rectangle indicates the probability of the object area is corn silk after target detector detection
表 1 不同网络模型试验结果对比
Table 1 Comparison of experimental results of different network models
网络模型
Network
model平均精度/%
Average precision平均精度/%
Average precision平均召回率/%
Average recall模型大小/MB
Model
size检测时间/s
Detection
timeIoU=
0.50~0.95IoU=
0.50IoU=
0.75小目标
Small
target中型目标
Medium
target大目标
Large
target小目标
Small
target中型目标
Medium
target大目标
Large
targetVGG16-SSD 64.1 97.3 74.5 39.0 63.1 79.4 48.5 70.7 84.3 160 0.320 MobileNet-SSD 62.9 96.5 72.8 37.9 61.9 78.7 46.7 68.7 82.8 84 0.261 MF-SSD-cut-3 70.9 95.8 85.2 55.3 70.4 80.2 65.5 76.1 83.3 80 0.260 MF-SSD 71.3 96.6 86.8 59.4 70.4 79.4 68.1 75.7 83.0 82 0.287 MF-SSD-add-3 71.9 96.7 87.6 57.0 71.8 79.8 67.3 76.8 83.1 86 0.275 
下载: 导出CSV
-
[1] 杨笛, 熊伟, 许吟隆, 等. 气候变化背景下中国玉米单产增速减缓的原因分析[J]. 农业工程学报, 2017, 33(增刊1): 231-238. [2] 刘哲, 曲艺伟, 赵祖亮, 等. 玉米优良品种推广重心转移及扩散的时空规律[J]. 农业工程学报, 2018, 34(1): 178-185. doi: 10.11975/j.issn.1002-6819.2018.01.24 [3] 王传宇, 郭新宇, 杜建军. 基于时间序列红外图像的玉米叶面积指数连续监测[J]. 农业工程学报, 2018, 34(6): 175-181. doi: 10.11975/j.issn.1002-6819.2018.06.022 [4] 王临铭, 高晓阳, 李红岭, 等. 基于神经网络的大麦病害识别研究[J]. 甘肃农业大学学报, 2015, 50(2): 173-176. doi: 10.3969/j.issn.1003-4315.2015.02.029 [5] 王璨, 武新慧, 李志伟. 基于卷积神经网络提取多尺度分层特征识别玉米杂草[J]. 农业工程学报, 2018, 34(5): 144-151. doi: 10.11975/j.issn.1002-6819.2018.05.019 [6] 周云成, 许童羽, 郑伟, 等. 基于面向通道分组卷积网络的番茄主要器官实时识方法[J]. 农业工程学报, 2018, 33(15): 153-162. [7] 刘永波, 雷波, 曹艳, 等. 基于深度卷积神经网络的玉米病害识别[J]. 中国农学通报, 2018, 34(36): 159-164. doi: 10.11924/j.issn.1000-6850.casb18030031 [8] KOIRALA A, WALSH K B, WANG Z, et al. Deep learning for real-time fruit detection and orchard fruit load estimation: Benchmarking of ‘MangoYOLO’[J]. Precis Agric, 2019, 20(6): 1107-1135. doi: 10.1007/s11119-019-09642-0
[9] TIAN Y, YANG G, WANG Z, et al. Apple detection during different growth stages in orchards using the improved YOLO-V3 model[J]. Comput Electron Agr, 2019, 157: 417-426. doi: 10.1016/j.compag.2019.01.012
[10] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE T Pattern Anal, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031
[11] LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector[C]//LEIBE B, MATAS J, SEBE N, et al. Lecture Notes in Computer Science. Cham: Springer International Publishing AG, 2016, 9905: 21-37.
[12] SILVER D, HUANG A, MADDISON C J, et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484-489. doi: 10.1038/nature16961
[13] YU Z, LI T, LUO G, et al. Convolutional networks with cross-layer neurons for image recognition[J]. Inform Sciences, 2018, 433: 241-254.
[14] ALAWAD M, LIN M. Stochastic-based deep convolutional networks with reconfigurable logic fabric[J]. IEEE Transactions on Multi-Scale Computing Systems, 2016, 2(4). doi: 10.1109/TMSCS.2016.2601326.
[15] LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition, New York: IEEE, 2017: 936-944.
[16] 马浚诚, 杜克明, 郑飞翔, 等. 基于卷积神经网络的温室黄瓜病害识别系统[J]. 农业工程学报, 2018, 34(12): 186-192. doi: 10.11975/j.issn.1002-6819.2018.12.022 [17] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE T Pattern Anal, 2020, 42(2): 318-327. doi: 10.1109/TPAMI.2018.2858826
[18] 彭红星, 黄博, 邵园园, 等. 自然环境下多类水果采摘目标识别的通用改进SSD模型[J]. 农业工程学报, 2018, 34(16): 155-162. doi: 10.11975/j.issn.1002-6819.2018.16.020 [19] LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: Common objects in context[C]//FLEET D, PAJDLA T, SCHIELE B, et al. Lecture Notes in Computer Science. Cham: Springer International Publishing AG, 2014, 8693: 740-755.
[20] REDMON J, FARHADI A. YOLO9000: Better, faster, stronger[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 6517-6525.
[21] REDMON J, FARHADI A. YOLOv3: An Incremental Improvement[C]//IEEE. IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2018.
[22] 李善军, 胡定一, 高淑敏, 等. 基于改进SSD的柑橘实时分类检测[J]. 农业工程学报, 2019, 35(24): 307-313. doi: 10.11975/j.issn.1002-6819.2019.24.036
计量
- 文章访问数: 1169
- HTML全文浏览量: 1
- PDF下载量: 1111
