基于深度强化学习的耕作层土壤水分、温度预测

    刘会丹, 万雪芬, 崔剑, 蔡婷婷, 杨义

    刘会丹, 万雪芬, 崔剑, 等. 基于深度强化学习的耕作层土壤水分、温度预测[J]. 华南农业大学学报, 2023, 44(1): 84-92. DOI: 10.7671/j.issn.1001-411X.202201032
    引用本文: 刘会丹, 万雪芬, 崔剑, 等. 基于深度强化学习的耕作层土壤水分、温度预测[J]. 华南农业大学学报, 2023, 44(1): 84-92. DOI: 10.7671/j.issn.1001-411X.202201032
    LIU Huidan, WAN Xuefen, CUI Jian, et al. Moisture and temperature prediction in tillage layer based on deep reinforcement learning[J]. Journal of South China Agricultural University, 2023, 44(1): 84-92. DOI: 10.7671/j.issn.1001-411X.202201032
    Citation: LIU Huidan, WAN Xuefen, CUI Jian, et al. Moisture and temperature prediction in tillage layer based on deep reinforcement learning[J]. Journal of South China Agricultural University, 2023, 44(1): 84-92. DOI: 10.7671/j.issn.1001-411X.202201032

    基于深度强化学习的耕作层土壤水分、温度预测

    基金项目: 国家重点研发计划(2018YFC0808306);廊坊市科学技术研究与发展计划(2021011035);秦皇岛市科学技术研究与发展计划(201805A016);河北省物联网监控工程技术研究中心项目(3142018055,3142016020)
    详细信息
      作者简介:

      刘会丹,硕士研究生,主要从事物联网技术研究,E-mail: 15038216873@163.com

      通讯作者:

      杨 义,副教授,博士,主要从事物联网技术及智慧农业研究,E-mail: yiyang@dhu.edu.cn

    • 中图分类号: S24

    Moisture and temperature prediction in tillage layer based on deep reinforcement learning

    • 摘要:
      目的 

      利用土壤近表面空气温湿度与土壤内部参数的关联关系对耕作层土壤水分、温度进行精准预测,为实现精细化农业种植管理提供服务。

      方法 

      针对土壤耕作层水分、温度预测在训练集获取与模型验证等方面的实际需求,设计了基于嵌入式系统及窄带物联网(Narrow band internet of things,NB-IoT)无线通信技术的物联网数据采集系统。在此基础上基于深度Q学习(Deep Q network,DQN)算法探索了一种模型组合策略,以长短期记忆(Long short-term memory,LSTM)、门限循环单元(Gated recurrent unit,GRU)与双向长短期记忆网络(Bi-directional long short-term memory,Bi-LSTM)为基础模型进行加权组合,获得了DQN-L-G-B组合预测模型。

      结果 

      数据采集系统实现了对等间隔时间序列环境数据的长时间稳定可靠采集,可以为基于深度学习的土壤水分、温度时间序列预测工作提供准确的训练集与验证集数据。相对于LSTM、Bi-LSTM、GRU、L-G-B等模型,DQN-L-G-B组合模型在2种土壤类型(壤土、砂土)耕作层上水分与温度预测中的均方根误差(Root mean square error,RMSE)、平均绝对误差(Mean absolute error,MAE)、平均百分比误差(Mean absolute percentage error,MAPE)都有一定程度的降低,R2提高了约0.1%。

      结论 

      通过该物联网数据采集系统与DNQ-L-G-B组合模型,可以有效地完成基于土壤近表面空气温、湿度对耕作层土壤中水分、温度的精准预测。

      Abstract:
      Objective 

      To accurately predict the water and temperature of the arable layer using the correlation between soil near surface air temperature and humidity and soil internal parameters, and serve for the realization of fine agricultural planting management.

      Method 

      Aiming at the actual needs of soil tillage layer moisture and temperature prediction in training set acquisition and model verification, an internet of things data acquisition system based on embedded system and narrow band internet of things (NB-IoT) wireless communication technology was designed. A model combination strategy was explored based on the deep Q network (DQN) deep reinforcement learning algorithm. Based on the weighted combination of long short-term memory (LSTM), gated recurrent unit (GRU) and Bi-directional long-short term memory (Bi-LSTM), the DQN-L-G-B combination prediction model was obtained.

      Result 

      The data acquisition system achieved long-term stable and reliable collection of time series environmental data with equal intervals, and provided accurate training set and verification set data for soil moisture, temperature time series prediction based on deep learning. Compared with models such as LSTM, Bi-LSTM, GRU and L-G-B, the DQN-L-G-B combined model not only lowered the root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) in the prediction of moisture and temperature on the tillage layer of the two soil types (loam and sand), but also increased R2 by about 0.1%.

      Conclusion 

      Through the internet of things data acquisition system and the DQN-L-G-B combined model, the accurate prediction of soil moisture and temperature in the cultivated layer based on soil near surface air temperature and humidity can be effectively completed.

    • 图  1   强化学习系统框图

      Figure  1.   Block diagram of reinforcement learning system

      图  2   组合预测模型结构图

      Figure  2.   Structure diagram of combination forecasting model

      图  3   现场终端节点主板(a)和试验中的现场终端节点(b)

      Figure  3.   Field terminal node main board (a) and field terminal node in the experiment (b)

      图  4   试验中用到的2种土壤类型

      Figure  4.   Two soil types used in the experiment

      图  5   不同土壤类型上的3种预测模型耕作层土壤温度的预测值与真实值

      Figure  5.   Predicted and true values of three prediction models for soil temperature of the cultivated layer in different soil types

      图  6   不同土壤类型3种预测模型耕作层土壤水分的预测值与真实值

      Figure  6.   Predicted and true values of three prediction models for soil moisture of the cultivated layer in different soil types

      表  1   2种土壤类型的土壤耕作层温度各模型的试验结果

      Table  1   Experimental results of various models of soil tillage layer temperature in two soil types

      模型 Model RMSE MAE MAPE R2
      壤土 Loam 砂土 Sand 壤土 Loam 砂土 Sand 壤土 Loam 砂土 Sand 壤土 Loam 砂土 Sand
      LSTM 0.752 0.953 0.509 0.643 0.0191 0.0239 0.916 0.872
      GRU 0.762 0.868 0.543 0.582 0.0202 0.0212 0.914 0.894
      Bi-LSTM 0.739 0.829 0.501 0.568 0.0188 0.0210 0.919 0.903
      L-G-B 0.721 0.838 0.484 0.560 0.0181 0.0207 0.923 0.901
      DQN-L-G-B 0.692 0.780 0.450 0.503 0.0167 0.0183 0.930 0.914
      下载: 导出CSV

      表  2   土壤耕作层水分预测各模型试验结果

      Table  2   Experimental results of various models for soil moisture prediction in cultivated layer

      模型 Model RMSE MAE MAPE R2
      壤土 Loam 砂土 Sand 壤土 Loam 砂土 Sand 壤土 Loam 砂土 Sand 壤土 Loam 砂土 Sand
      LSTM 0.472 0.485 0.134 0.226 0.0399 0.0391 0.995 0.994
      GRU 0.471 0.502 0.150 0.237 0.0469 0.0487 0.995 0.994
      Bi-LSTM 0.468 0.477 0.114 0.186 0.0249 0.0360 0.995 0.995
      L-G-B 0.459 0.469 0.106 0.169 0.0312 0.0353 0.996 0.995
      DQN-L-G-B 0.431 0.452 0.090 0.142 0.0217 0.0336 0.996 0.996
      下载: 导出CSV
    • [1]

      XING L, LI L H, GONG J K, et al. Daily soil temperatures predictions for various climates in United States using data-driven model[J]. Energy, 2018, 160: 430-440. doi: 10.1016/j.energy.2018.07.004

      [2]

      HAO H, YU F H, LI Q L. Soil temperature prediction using convolutional neural network based on ensemble empirical mode decomposition[J]. IEEE Access, 2020, 9: 4084-4096.

      [3] 杜娟. 关中平原土壤耕作层形成过程研究[D]. 西安: 陕西师范大学, 2014.
      [4] 付威, 雍晨旭, 马东豪, 等. 黄土丘陵沟壑区治沟造地土壤快速培肥效应[J]. 农业工程学报, 2019, 35(21): 252-261. doi: 10.11975/j.issn.1002-6819.2019.21.031
      [5]

      PRASAD R, DEO R C, LI Y, et al. Soil moisture forecasting by a hybrid machine learning technique: ELM integrated with ensemble empirical mode decomposition[J]. Geoderma, 2018, 330: 136-161. doi: 10.1016/j.geoderma.2018.05.035

      [6]

      FU Q, MA Z A, WANG E L, et al. Impact factors and dynamic simulation of tillage-layer temperature in frozen-thawed soil under different cover conditions[J]. International Journal of Agricultural and Biological Engineering, 2018, 11(2): 101-107. doi: 10.25165/j.ijabe.20181102.3068

      [7] 李宏鹏, 张婉婷, 李颖姣. 气温对浅层地温的影响研究综述[J]. 现代农业研究, 2020, 26(7): 57-58. doi: 10.3969/j.issn.1674-0653.2020.07.026
      [8] 付强, 马梓奡, 李天霄, 等. 北方高寒区不同覆盖条件下土壤温度差异性分析[J]. 农业机械学报, 2014, 45(12): 152-159. doi: 10.6041/j.issn.1000-1298.2014.12.023
      [9]

      HAN G L, WANG J L, PAN Y Y, et al. Temporal and spatial variation of soil moisture and its possible impact on regional air temperature in China[J]. Water, 2020, 12(6): 1807. doi: 10.3390/w12061807.

      [10] 薛晓萍, 王新, 张丽娟, 等. 基于支持向量机方法建立土壤湿度预测模型的探讨[J]. 土壤通报, 2007, 38(3): 427-433. doi: 10.3321/j.issn:0564-3945.2007.03.003
      [11]

      WU W, TANG X P, GUO N J, et al. Spatiotemporal modeling of monthly soil temperature using artificial neural networks[J]. Theoretical and Applied Climatology, 2013, 113(3/4): 481-494.

      [12]

      JUNG D H, KIM H S, JHIN C, et al. Time-serial analysis of deep neural network models for prediction of climatic conditions inside a greenhouse[J]. Computers and Electronics in Agriculture, 2020, 173: 105402. doi: 10.1016/j.compag.2020.105402.

      [13] 柴萌, 王振龙, 陈元芳, 等. 淮北南部区地温变化及其对气温变化的响应[J]. 土壤通报, 2020, 51(3): 568-573. doi: 10.19336/j.cnki.trtb.2020.03.09
      [14]

      LIU Y Q, ZHANG Q, SONG L H, et al. Attention-based recurrent neural networks for accurate short-term and long-term dissolved oxygen prediction[J]. Computers and Electronics in Agriculture, 2019, 165: 104964. doi: 10.1016/j.compag.2019.104964.

      [15]

      MA M, MAO Z. Deep-convolution-based LSTM network for remaining useful life prediction[J]. IEEE Transactions on Industrial Informatics, 2021, 17(3): 1658-1667. doi: 10.1109/TII.2020.2991796

      [16]

      KIM H Y, WON C H. Forecasting the volatility of stock price index: A hybrid model integrating LSTM with multiple GARCH-type models[J]. Expert Systems with Applications, 2018, 103: 25-37. doi: 10.1016/j.eswa.2018.03.002

      [17]

      CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//ACL. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing(EMNLP). Doha, Qatar: Association for Computational Linguistics, 2014: 1724-1734.

      [18]

      YU J X, ZHANG X, XU L L, et al. A hybrid CNN-GRU model for predicting soil moisture in maize root zone[J]. Agricultural Water Management, 2021, 245: 106649. doi: 10.1016/j.agwat.2020.106649.

      [19]

      HE Y L, CHEN L, GAO Y, et al. Novel double-layer bidirectional LSTM network with improved attention mechanism for predicting energy consumption[J]. ISA Transactions, 2022, 127: 350-360. doi: 10.1016/j.isatra.2021.08.030

      [20]

      YIN J, DENG Z, INES A V M, et al. Forecast of short-term daily reference evapotranspiration under limited meteorological variables using a hybrid bi-directional long short-term memory model (Bi-LSTM)[J]. Agricultural Water Management, 2020, 242: 106386. doi: 10.1016/j.agwat.2020.106386.

      [21] 周志华. 机器学习[M]. 北京: 清华大学出版社, 2016: 371-372.
      [22]

      SUTTON R S, BARTO A G. Reinforcement learning: An introduction[J]. IEEE Transactions on Neural Networks, 1998, 9(5): 1054. doi: 10.1109/TNN.1998.712192.

      [23]

      WU J D, HE H W, PENG J K, et al. Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus[J]. Applied Energy, 2018, 222: 799-811. doi: 10.1016/j.apenergy.2018.03.104

      [24]

      ZHU J, SONG Y, JIANG D, et al. A new deep-Q-learning-based transmission scheduling mechanism for the cognitive internet of things[J]. IEEE Internet of Things Journal, 2018, 5(4): 2375-2385.

      [25]

      MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533. doi: 10.1038/nature14236

      [26]

      CARTA S, FERREIRA A, PODDA A S, et al. Multi-DQN: An ensemble of deep Q-learning agents for stock market forecasting[J]. Expert Systems with Applications, 2021, 164: 113820. doi: 10.1016/j.eswa.2020.113820.

    图(6)  /  表(2)
    计量
    • 文章访问数: 
    • HTML全文浏览量:  0
    • PDF下载量: 
    • 被引次数: 0
    出版历程
    • 收稿日期:  2022-01-24
    • 网络出版日期:  2023-05-17
    • 刊出日期:  2023-01-09

    目录

      /

      返回文章
      返回