用户自定义语义的地形表面纹理生成网络

doi:10.16182/j.issn1004731x.joss.23-FZ0802E

摘要/Abstract

摘要：

基于用户语义的地形定制在军事仿真应用的虚拟地形建模中具有实用价值。研究中提供一种基于用户输入语义合成真实地形的TSTG-Net网络（terrain surface texture generation networks）。该网络被设计为基于Pix2pix结构，并基于条件生成对抗网络CGAN设计，通过编码和解析用户语义来学习定制地形的拓扑结构，并将其作为约束CGAN的语义特征。在生成器-鉴别器结构中，用户自定义的语义作为输入，并使用语义真实地形作为网络优化约束的真值。为了获得具有细节的地形信息，研究了一种基于小波变换的地形生成策略，充分利用小波变换分析方法来解决地形合成问题。TSTG Net被设计为双重编码，第一个编码-解码过程采用普通的4层编码和4层解码结构，第二个编码过程基于离散小波变换来提取更精确的纹理表面特征，不同于现有技术，双重编码结构可以生成尽可能逼真的地形纹理图，通过约束条件可以生成更精细的地形纹理。它提取了多个通道的冗余特征，可以获得复杂度较低的轻量模型。通过在公开真实的地形数据以及合成地形数据进行实验，均证明所研究算法能够实现用户地形的定制，并能以较好的时间性能生成逼真的结果。

关键词: 地形定制, CGAN网络, Pix2pix结构, 小波变换

Abstract:

Customizing terrain based on user semantics has practical value in the virtual terrain modeling of military simulation applications. This study provides a terrain surface texture generation network (TSTG-Net) that can synthesize realistic terrain based on user input semantics. TSTG-Net is designed as a Pix2pix structure and is based on CGAN. It learns the topology of customized terrain by encoding and parsing user semantics and regards the semantics feature as the constraint of CGAN. In the generator-discriminator structure, user-customized semantics are used as the input, and the real terrain with semantics is employed as the ground truth in network optimization. In order to obtain more detailed terrain information, the terrain generation strategy based on wavelet transform is studied, which takes full use of the wavelet transformation analysis method to solve the problem of terrain synthesis. TSTG-Net is designed as double encoding. The first encoding-decoding process takes ordinary 4-layer encoding and 4-layer decoding structures, and the second encoding process is designed based on DWT to extract more precise features of the texture surface. Different from the state of the art, our double encoding structure can generate the terrain texture map as realistically as possible, and then finer terrain textures can be generated through the constraints conditions. In addition, it extracts the redundant features of multiple channels, and a lightweight model with lower complexity can be obtained. Experiments carried out on the public real terrain data and the synthesized terrain data both verify that the proposed algorithm can achieve user terrain customization and generate realistic results in better time performance.

Key words: terrain customization, CGAN, Pix2pix structure, wavelet transform

中图分类号:

TP391

高岩,李吉梦,许建中等 . 用户自定义语义的地形表面纹理生成网络[J]. 系统仿真学报, 2023, 35(10): 2077-2086.

Gao Yan,Li Jimeng,Xu Jianzhong,et al . Terrain Surface Texture Generation Networks for User Semantics Customization[J]. Journal of System Simulation, 2023, 35(10): 2077-2086.

图/表 9

参考文献 27

1	Luo Kunming, Wang Xiaoyu. The Application of Virtual Reality Technology in Environmental Art Design[C]//The 2020 International Conference on Machine Learning and Big Data Analytics for IoT Security and Privacy. Cham: Springer International Publishing, 2021: 744-751.
2	Zhang Tao. Research on Environmental Landscape Design Based on Virtual Reality Technology and Deep Learning[J]. Microprocessors and Microsystems, 2021, 81: 103796.
3	Li Wanwan. Terrain Synthesis for Treadmill Exergaming in Virtual Reality[C]//2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Piscataway, NJ, USA: IEEE, 2023: 263-269.
4	Easson L, Tavakkoli A, Greenberg J. An Automatic Digital Terrain Generation Technique for Terrestrial Sensing and Virtual Reality Applications[C]//Advances in Visual Computing. Cham: Springer International Publishing, 2019: 619-630.
5	Jain A, Sharma A, Rajan K S. Adaptive & Multi-resolution Procedural Infinite Terrain Generation With Diffusion Models and Perlin Noise[C]//Proceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing. New York, NY, USA: Association for Computing Machinery, 2022: 55.
6	Scott J. Realism in Data-based Terrain Synthesis[D]. Wellington: Victoria University of Wellington, 2020.
7	Hojo A, Takagi K, Avtar R, et al. Synthesis of L-band SAR and Forest Heights Derived From TanDEM-X DEM and 3 Digital Terrain Models for Biomass Mapping[J]. Remote Sensing, 2020, 12(3): 349.
8	Szabó Zsuzsanna, Csaba Albert Tóth, Holb I, et al. Aerial Laser Scanning Data as a Source of Terrain Modeling in a Fluvial Environment: Biasing Factors of Terrain Height Accuracy[J]. Sensors, 2020, 20(7): 2063.
9	Nguyen T, Bui M, Ninh H, et al. Efficient Heuristic Algorithm to Speed up GraphCut in GPU for Image Stitching[C]//2022 IEEE 13th International Symposium on Parallel Architectures, Algorithms and Programming (PAAP). Piscataway, NJ, USA: IEEE, 2022: 1-6.
10	Liu Lu, Song Xiao, Zhou Zhetao. Aircraft Engine Remaining Useful Life Estimation Via a Double Attention-based Data-driven Architecture[J]. Reliability Engineering & System Safety, 2022, 221: 108330.
11	Sirish Kumar Pagoti, Srilatha Indira Dutt Vemuri, Laveti Ganesh. GPS Receiver Position Augmentation Using Correntropy Kalman Filter in Low Latitude Terrain[J]. The International Arab Journal of Information Technology, 2022, 19(1): 72-80.
12	Wang Ganlin, Chen Jing. A Vector Data Model for Efficiently Rendering Large Vector Maps on Global 3D Terrain Surfaces[J]. ISPRS International Journal of Geo-Information, 2022, 11(4): 234.
13	Li Bowen, Lu Hao, Wang Han, et al. Terrain-net: A Highly-efficient, Parameter-free, and Easy-to-use Deep Neural Network for Ground Filtering of UAV LiDAR Data in Forested Environments[J]. Remote Sensing, 2022, 14(22): 5798.
14	Lê Hoàng-Ân, Guiotte F, Pham M T, et al. Learning Digital Terrain Models From Point Clouds: ALS2DTM Dataset and Rasterization-based GAN[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 4980-4989.
15	Wang Chen, Cui Yong, Song Xiao, et al. Model, Design, and Testing of an Electret-based Portable Transmitter for Low-frequency Applications[J]. IEEE Transactions on Antennas and Propagation, 2021, 69(9): 5305-5314.
16	Toulatzis Vasilis, Fudos Ioannis. Deep Terrain Expansion: Terrain Texture Synthesis With Deep Learning[M]. CGVC, Eurographics Association, 2019: 95-96.
17	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks[J]. Communications of the ACM, 2020, 63(11): 139-144.
18	Isola P, Zhu Junyan, Zhou Tinghui, et al. Image-to-image Translation With Conditional Adversarial Networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2017: 5967-5976.
19	Ronneberger O, Fischer P, Brox T. U-net: Convolutional Networks for Biomedical Image Segmentation[C]//Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer International Publishing, 2015: 234-241.
20	Feng Xubin, Zhang Wuxia, Su Xiuqin, et al. Optical Remote Sensing Image Denoising and Super-resolution Reconstructing Using Optimized Generative Network in Wavelet Transform Domain[J]. Remote Sensing, 2021, 13(9): 1858.
21	Li Haisheng, Fan Ping, Peng Huiling, et al. Multilevel 2-D Quantum Wavelet Transforms[J]. IEEE Transactions on Cybernetics, 2022, 52(8): 8467-8480.
22	Li Jin, Li Wanyun, Xu Zichen, et al. Wavelet Transform-assisted Adaptive Generative Modeling for Colorization[J/OL]. IEEE Transactions on Multimedia. (2022-05-26) [2022-08-18]. .
23	Ch M M I, Riaz M M, Iltaf N, et al. Shift-invariant Discrete Wavelet Transform-based Sparse Fusion of Medical Images[J]. Signal Image and Video Processing, 2023, 17(4): 881-889.
24	Pappas T. Earth Terrain, Height, and Segmentation Map Images[EB/OL]. [2022-07-28]. .
25	Schoenfeld E, Sushko V, Zhang Dan, et al. You Only Need Adversarial Supervision for Semantic Image Synthesis[C]//International Conference on Learning Representations (ICLR) 2021. New York, USA: ICLR, 2021: 2686.
26	Park T, Liu Mingyu, Wang Tingchun, et al. Semantic Image Synthesis With Spatially-adaptive Normalization[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2019: 2332-2341.
27	Shi Bo, Ma Ziji, Liu Jianwei, et al. Shadow Extraction Method Based on Multi-information Fusion and Discrete Wavelet Transform[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-15.

Methods	SSIM	PSNR	FID
Backbone	0.756	20.271	160.891
Backbone + 1DWT	0.760	20.812	128.124
Backbone + 2DWT	0.762	20.876	109.362
Backbone + 4DWT	0.788	21.342	67.811
Backbone + 4DWT + 1Con	0.790	21.580	25.825
Ours(Backbone + 4DWT + 2Con)	0.824	22.953	21.024

[1]	苏本跃, 张利, 何清旋, 盛敏. 基于小波特征匹配的短时人体行为识别[J]. 系统仿真学报, 2023, 35(1): 158-168.
[2]	陈涛, 王艳, 纪志成. 基于经验小波变换的光伏功率组合预测模型[J]. 系统仿真学报, 2021, 33(11): 2627-2635.
[3]	黄升, 张涛, 黄崇君, 李玉梅, 邓虎, 张霞. 井下数据获取及粘滑特征分析[J]. 系统仿真学报, 2019, 31(11): 2517-2526.
[4]	王依人, 邓国庆, 夏营威, 张龙, 刘勇, 张文. 基于方向可调滤波器的血管图像增强算法[J]. 系统仿真学报, 2018, 30(6): 2095-2101.
[5]	闫国强, 周宁宁, 张少白. 基于MMTD与小波硬阈值的脑电信号去噪方法[J]. 系统仿真学报, 2018, 30(4): 1490-1495.
[6]	张鸿喜, 张洁. 小波变换在舷外有源诱饵仿真模型验证中的应用[J]. 系统仿真学报, 2018, 30(1): 318-324.
[7]	赵迪, 徐志胜. 基于多分辨率奇异熵的混凝土缺陷检测[J]. 系统仿真学报, 2017, 29(4): 761-766.
[8]	周国雄, 周先雁, 王解军, 黄特. 基于小波分析—蚁群BP网络的木构件缺陷无损检测[J]. 系统仿真学报, 2015, 27(11): 2804-2810.
[9]	李豪, 李锐华, 胡波, 潘玲, 郭其一. 基于导波的大电机定子绝缘损伤定位仿真研究[J]. 系统仿真学报, 2015, 27(11): 2816-2821.