Construction of a Virtual Interactive System for Orchards Based on Digital Twin

doi:10.16182/j.issn1004731x.joss.23-0317

Abstract

Abstract:

Aiming at the low visibility, poor real-time, weak adaptability and single interaction mode in orchard planting management system, a six-dimensional model of orchard digital twin system for planting management process is proposed. The system model construction theory and technology system is discussed from four aspects, entity modeling of management elements, dynamic modeling of management process, simulation modeling of management system and optimization modeling of management strategy. Based on the six-dimensional model, supported by the theory and technology system, the virtual interactive system architecture of the orchard based on the digital twin is designed, and the key technologies of the system development process are standard. With the Unity3D simulation platform, the orchard virtual interactive system is built to realize the three-dimensional visualization monitoring of the orchard. The experimental results show that the system improves the monitoring ability of orchard plantation management system through data-driven, 3D display, human-computer interaction, reconstructed events and intelligent decision-making on the basis of data sharing on the platform of internet of things in agriculture, which provides a reference for the construction of smart orchard.

Key words: DT, orchard planting management, internet of things in agriculture, VR, DRL

CLC Number:

TP242

Wang Hongjun, Lin Junqiang, Zou Xiangjun, Zhang Po, Zhou Mingxuan, Zou Weirui, Tang Yunchao, Luo Lufeng. Construction of a Virtual Interactive System for Orchards Based on Digital Twin[J]. Journal of System Simulation, 2024, 36(6): 1493-1508.

Figures/Tables 16

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 1

Fig. 9

Fig. 10

Fig. 11

Fig. 12

Fig. 13

Fig. 14

Fig. 15

References 25

1	邹湘军, 孙健, 何汉武, 等. 虚拟现实技术的演变发展与展望[J]. 系统仿真学报, 2004, 16(9): 1905-1909.
	Zou Xiangjun, Sun Jian, He Hanwu, et al. The Development and Prospects of Virtual Reality[J]. Journal of System Simulation, 2004, 16(9): 1905-1909.
2	Tang Yunchao, Chen Mingyou, Wang Chenglin, et al. Recognition and Localization Methods for Vision-based Fruit Picking Robots: A Review[J]. Frontiers in Plant Science, 2020, 11: 510.
3	Tang Yunchao, Zhou Hao, Wang Hongjun, et al. Fruit Detection and Positioning Technology for a Camellia Oleifera C. Abel Orchard Based on Improved YOLOv4-tiny Model and Binocular Stereo Vision[J]. Expert Systems with Applications, 2023, 211: 118573.
4	顾生浩, 卢宪菊, 王勇健, 等. 数字孪生系统在农业生产中的应用探讨[J]. 中国农业科技导报, 2021, 23(10): 82-89.
	Gu Shenghao, Lu Xianju, Wang Yongjian, et al. Application of Agricultural Digital Twin System in Crop Production System[J]. Journal of Agricultural Science and Technology, 2021, 23(10): 82-89.
5	Tao Fei, Qi Qinglin. Make More Digital Twins[J]. Nature, 2019, 573(7775): 490-491.
6	Grieves M, Vickers J. Digital Twin: Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems[M]//Kahlen F J, Flumerfelt S, Alves A. Transdisciplinary Perspectives on Complex Systems: New Findings and Approaches. Cham: Springer International Publishing, 2017: 85-113.
7	陶飞, 刘蔚然, 刘检华, 等. 数字孪生及其应用探索[J]. 计算机集成制造系统, 2018, 24(1): 1-18.
	Tao Fei, Liu Weiran, Liu Jianhua, et al. Digital Twin and Its Potential Application Exploration[J]. Computer Integrated Manufacturing Systems, 2018, 24(1): 1-18.
8	陶飞, 马昕, 戚庆林, 等. 数字孪生连接交互理论与关键技术[J]. 计算机集成制造系统, 2023, 29(1): 1-10.
	Tao Fei, Ma Xin, Qi Qinglin, et al. Theory and Key Technologies of Digital Twin Connection and Interaction[J]. Computer Integrated Manufacturing Systems, 2023, 29(1): 1-10.
9	Wang Baicun, Zheng Pai, Yin Yue, et al. Toward Human-centric Smart Manufacturing: A Human-cyber-physical Systems (HCPS) Perspective[J]. Journal of Manufacturing Systems, 2022, 63: 471-490.
10	Lee J, Bagheri B, Kao H A. A Cyber-physical Systems Architecture for Industry 4.0-based Manufacturing Systems[J]. Manufacturing Letters, 2015, 3: 18-23.
11	Jiang J R. An Improved Cyber-physical Systems Architecture for Industry 4.0 Smart Factories[J]. Advances in Mechanical Engineering, 2018, 10(6): 1-15.
12	Alam K M, El Saddik A. C2PS: A Digital Twin Architecture Reference Model for the Cloud-based Cyber-physical Systems[J]. IEEE Access, 2017, 5: 2050-2062.
13	Wang Ziran, Gupta R, Han K, et al. Mobility Digital Twin: Concept, Architecture, Case Study, and Future Challenges[J]. IEEE Internet of Things Journal, 2022, 9(18): 17452-17467.
14	周成, 孙恺庭, 李江, 等. 基于数字孪生的车间三维可视化监控系统[J]. 计算机集成制造系统, 2022, 28(3): 758-768.
	Zhou Cheng, Sun Kaiting, Li Jiang, et al. Workshop 3D Visual Monitoring System Based on Digital Twin[J]. Computer Integrated Manufacturing Systems, 2022, 28(3): 758-768.
15	赵浩然, 刘检华, 熊辉, 等. 面向数字孪生车间的三维可视化实时监控方法[J]. 计算机集成制造系统, 2019, 25(6): 1432-1443.
	Zhao Haoran, Liu Jianhua, Xiong Hui, et al. 3D Visualization Real-time Monitoring Method for Digital Twin Workshop[J]. Computer Integrated Manufacturing Systems, 2019, 25(6): 1432-1443.
16	Hemming Silke, Feije de Zwart, Elings Anne, et al. Cherry Tomato Production in Intelligent Greenhouses-sensors and AI for Control of Climate, Irrigation, Crop Yield, and Quality[J]. Sensors, 2020, 20(22): 6430.
17	Daniel Anthony Howard, Ma Zheng, Jesper Mazanti Aaslyng, et al. Data Architecture for Digital Twin of Commercial Greenhouse Production[C]//2020 RIVF International Conference on Computing and Communication Technologies (RIVF). Piscataway, NJ, USA: IEEE, 2020: 1-7.
18	Melesse Tsega Y, Bollo Matteo, Valentina Di Pasquale, et al. Machine Learning-based Digital Twin for Monitoring Fruit Quality Evolution[J]. Procedia Computer Science, 2022, 200: 13-20.
19	Zhang Qinglei, Yang Zhiwei, Duan Jianguo, et al. Three-dimensional Visualization Interactive System for Digital Twin Workshop[J]. Journal of Southeast University(English Edition), 2021, 37(2): 137-152.
20	李浩, 刘根, 文笑雨, 等. 面向人机交互的数字孪生系统工业安全控制体系与关键技术[J]. 计算机集成制造系统, 2021, 27(2): 374-389.
	Li Hao, Liu Gen, Wen Xiaoyu, et al. Industrial Safety Control System and Key Technologies of Digital Twin System Oriented to Human-machine Interaction[J]. Computer Integrated Manufacturing Systems, 2021, 27(2): 374-389.
21	Lin Junqiang, Zhang Po, Li Chengen, et al. APF-DPPO: An Automatic Driving Policy Learning Method Based on the Artificial Potential Field Method to Optimize the Reward Function[J]. Machines, 2022, 10(7): 533.
22	罗陆锋, 邹湘军, 刘天湖, 等. 大数据量虚拟景观的三维模型优化与漫游[J]. 系统仿真学报, 2009, 21(6): 1654-1657.
	Luo Lufeng, Zou Xiangjun, Liu Tianhu, et al. 3D Model Optimization and Navigation of Large-scale Virtual Landscape[J]. Journal of System Simulation, 2009, 21(6): 1654-1657.
23	林俊强, 王红军, 邹湘军, 等. 基于DPPO的移动采摘机器人避障路径规划及仿真[J]. 系统仿真学报, 2023, 35(8): 1692-1704.
	Lin Junqiang, Wang Hongjun, Zou Xiangjun, et al. Obstacle Avoidance Path Planning and Simulation of Mobile Picking Robot Based on DPPO[J]. Journal of System Simulation, 2023, 35(8): 1692-1704.
24	董力中. 荔枝果园智能监测关键技术研究[D]. 广州: 华南农业大学, 2020.
	Dong Lizhong. Research on Key Technologies of Litchi Orchard Intelligent Monitoring[D]. Guangzhou: South China Agricultural University, 2020.
25	陶飞, 刘蔚然, 张萌, 等. 数字孪生五维模型及十大领域应用[J]. 计算机集成制造系统, 2019, 25(1): 1-18.
	Tao Fei, Liu Weiran, Zhang Meng, et al. Five-dimension Digital Twin Model and Its Ten Applications[J]. Computer Integrated Manufacturing Systems, 2019, 25(1): 1-18.

[1]	Wei Sheng, Wang Yan, Ji Zhicheng. Just-in-time Learning Energy Consumption Predictive Modeling Method in Multi-condition Production Process [J]. Journal of System Simulation, 2024, 36(6): 1378-1391.
[2]	Zhu Zilu, Liu Yongkui, Zhang Lin, Wang Lihui, Lin Tingyu. Simulation of Robotic Peg-in-hole Assembly Strategy Based on DRL [J]. Journal of System Simulation, 2024, 36(6): 1414-1424.
[3]	Wang Yuan, Xu Lin, Gong Xiaoze, Zhang Yongliang, Wang Yongli. Gradient-based Deep Reinforcement Learning Interpretation Methods [J]. Journal of System Simulation, 2024, 36(5): 1130-1140.
[4]	Zhao Yingying, Dong Pusen, Zhu Tianchen, Li Fan, Su Yun, Tai Zhenying, Sun Qingyun, Fan Hang. Efficiency Optimization Method for Data Sampling in Power Grid Topology Scheduling Simulation [J]. Journal of System Simulation, 2024, 36(2): 283-295.
[5]	Wang Xinpeng, Fu Huiqiao, Deng Guizhou, Tang Kaiqiang, Chen Chunlin, Liu Canghai. Research on Motion Planning of Hexapod Robot Based on DRL and Free Gait [J]. Journal of System Simulation, 2024, 36(2): 373-384.
[6]	Pan Hainan, Chen Bailiang, Huang Kaihong, Ren Junkai, Cheng Chuang, Lu Huimin, Zhang Hui. Flipper Control Method for Tracked Robot Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2024, 36(2): 405-414.
[7]	An Jing, Si Guangya, Zhang Lei. Strategy Optimization Method of Multi-dimension Projection Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2024, 36(1): 39-49.
[8]	Yang Li, Huijuan Zhang, Chenchen Ge, Kang Xie, Zhuang Li, Jinyuan Jia. Lightweight WebVR Real-Time Simulation of Large-Scale Fire Scenario in Metro [J]. Journal of System Simulation, 2023, 35(3): 646-657.
[9]	Wang Yukun, Wang Ze, Dong Liwei, Li Ni. Research on Multi-aircraft Air Combat Behavior Modeling Based on Hierarchical Intelligent Modeling Methods [J]. Journal of System Simulation, 2023, 35(10): 2249-2261.
[10]	Wei Quan, Chao Wang, Xuena Geng, Cheng Han. Research on VR Experience Comfort Based on Motion Perception [J]. Journal of System Simulation, 2023, 35(1): 169-177.
[11]	Liying Wang, Jialin Lin, Houqi Dong, Ming Zeng, Yuqing Wang. Optimal Dispatch of Integrated Energy System Considering Ladder-Type Carbon Trading [J]. Journal of System Simulation, 2022, 34(7): 1393-1404.
[12]	Cheng Lu, Xuesheng Jin. Design of Interactive Simulated Water Gun Fire Fighting Training System Based on Steam VR [J]. Journal of System Simulation, 2022, 34(6): 1312-1319.
[13]	Yufei Guo, Kang Zhao, Yongqing Hai. Triangular Mesh Boolean Operation Method for Finite Element Analysis [J]. Journal of System Simulation, 2022, 34(5): 1003-1014.
[14]	Xinhang He, Erkang Li, Hongchao Zhang. Research on Optimization of Network Resource Utilization in Static Segment of FlexRay Bus [J]. Journal of System Simulation, 2022, 34(3): 564-572.
[15]	Jiaji Lou, Jing Ma, Xiaowei Li, Yue Zhao, Youbin Song. Research on Modeling and Simulation Technology of Microwave Radar High Precision Tracking Loop [J]. Journal of System Simulation, 2022, 34(12): 2619-2628.