Terrain Surface Texture Generation Networks for User Semantics Customization

doi:10.16182/j.issn1004731x.joss.23-FZ0802E

Abstract

Abstract:

Customizing terrain based on user semantics has practical value in the virtual terrain modeling of military simulation applications. This study provides a terrain surface texture generation network (TSTG-Net) that can synthesize realistic terrain based on user input semantics. TSTG-Net is designed as a Pix2pix structure and is based on CGAN. It learns the topology of customized terrain by encoding and parsing user semantics and regards the semantics feature as the constraint of CGAN. In the generator-discriminator structure, user-customized semantics are used as the input, and the real terrain with semantics is employed as the ground truth in network optimization. In order to obtain more detailed terrain information, the terrain generation strategy based on wavelet transform is studied, which takes full use of the wavelet transformation analysis method to solve the problem of terrain synthesis. TSTG-Net is designed as double encoding. The first encoding-decoding process takes ordinary 4-layer encoding and 4-layer decoding structures, and the second encoding process is designed based on DWT to extract more precise features of the texture surface. Different from the state of the art, our double encoding structure can generate the terrain texture map as realistically as possible, and then finer terrain textures can be generated through the constraints conditions. In addition, it extracts the redundant features of multiple channels, and a lightweight model with lower complexity can be obtained. Experiments carried out on the public real terrain data and the synthesized terrain data both verify that the proposed algorithm can achieve user terrain customization and generate realistic results in better time performance.

Key words: terrain customization, CGAN, Pix2pix structure, wavelet transform

CLC Number:

TP391

Gao Yan, Li Jimeng, Xu Jianzhong, Quan Hongyan. Terrain Surface Texture Generation Networks for User Semantics Customization[J]. Journal of System Simulation, 2023, 35(10): 2077-2086.

Figures/Tables 9

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Table 1

Table 2

Fig. 6

Table 3

References 27

1	Luo Kunming, Wang Xiaoyu. The Application of Virtual Reality Technology in Environmental Art Design[C]//The 2020 International Conference on Machine Learning and Big Data Analytics for IoT Security and Privacy. Cham: Springer International Publishing, 2021: 744-751.
2	Zhang Tao. Research on Environmental Landscape Design Based on Virtual Reality Technology and Deep Learning[J]. Microprocessors and Microsystems, 2021, 81: 103796.
3	Li Wanwan. Terrain Synthesis for Treadmill Exergaming in Virtual Reality[C]//2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Piscataway, NJ, USA: IEEE, 2023: 263-269.
4	Easson L, Tavakkoli A, Greenberg J. An Automatic Digital Terrain Generation Technique for Terrestrial Sensing and Virtual Reality Applications[C]//Advances in Visual Computing. Cham: Springer International Publishing, 2019: 619-630.
5	Jain A, Sharma A, Rajan K S. Adaptive & Multi-resolution Procedural Infinite Terrain Generation With Diffusion Models and Perlin Noise[C]//Proceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image Processing. New York, NY, USA: Association for Computing Machinery, 2022: 55.
6	Scott J. Realism in Data-based Terrain Synthesis[D]. Wellington: Victoria University of Wellington, 2020.
7	Hojo A, Takagi K, Avtar R, et al. Synthesis of L-band SAR and Forest Heights Derived From TanDEM-X DEM and 3 Digital Terrain Models for Biomass Mapping[J]. Remote Sensing, 2020, 12(3): 349.
8	Szabó Zsuzsanna, Csaba Albert Tóth, Holb I, et al. Aerial Laser Scanning Data as a Source of Terrain Modeling in a Fluvial Environment: Biasing Factors of Terrain Height Accuracy[J]. Sensors, 2020, 20(7): 2063.
9	Nguyen T, Bui M, Ninh H, et al. Efficient Heuristic Algorithm to Speed up GraphCut in GPU for Image Stitching[C]//2022 IEEE 13th International Symposium on Parallel Architectures, Algorithms and Programming (PAAP). Piscataway, NJ, USA: IEEE, 2022: 1-6.
10	Liu Lu, Song Xiao, Zhou Zhetao. Aircraft Engine Remaining Useful Life Estimation Via a Double Attention-based Data-driven Architecture[J]. Reliability Engineering & System Safety, 2022, 221: 108330.
11	Sirish Kumar Pagoti, Srilatha Indira Dutt Vemuri, Laveti Ganesh. GPS Receiver Position Augmentation Using Correntropy Kalman Filter in Low Latitude Terrain[J]. The International Arab Journal of Information Technology, 2022, 19(1): 72-80.
12	Wang Ganlin, Chen Jing. A Vector Data Model for Efficiently Rendering Large Vector Maps on Global 3D Terrain Surfaces[J]. ISPRS International Journal of Geo-Information, 2022, 11(4): 234.
13	Li Bowen, Lu Hao, Wang Han, et al. Terrain-net: A Highly-efficient, Parameter-free, and Easy-to-use Deep Neural Network for Ground Filtering of UAV LiDAR Data in Forested Environments[J]. Remote Sensing, 2022, 14(22): 5798.
14	Lê Hoàng-Ân, Guiotte F, Pham M T, et al. Learning Digital Terrain Models From Point Clouds: ALS2DTM Dataset and Rasterization-based GAN[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 4980-4989.
15	Wang Chen, Cui Yong, Song Xiao, et al. Model, Design, and Testing of an Electret-based Portable Transmitter for Low-frequency Applications[J]. IEEE Transactions on Antennas and Propagation, 2021, 69(9): 5305-5314.
16	Toulatzis Vasilis, Fudos Ioannis. Deep Terrain Expansion: Terrain Texture Synthesis With Deep Learning[M]. CGVC, Eurographics Association, 2019: 95-96.
17	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks[J]. Communications of the ACM, 2020, 63(11): 139-144.
18	Isola P, Zhu Junyan, Zhou Tinghui, et al. Image-to-image Translation With Conditional Adversarial Networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2017: 5967-5976.
19	Ronneberger O, Fischer P, Brox T. U-net: Convolutional Networks for Biomedical Image Segmentation[C]//Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer International Publishing, 2015: 234-241.
20	Feng Xubin, Zhang Wuxia, Su Xiuqin, et al. Optical Remote Sensing Image Denoising and Super-resolution Reconstructing Using Optimized Generative Network in Wavelet Transform Domain[J]. Remote Sensing, 2021, 13(9): 1858.
21	Li Haisheng, Fan Ping, Peng Huiling, et al. Multilevel 2-D Quantum Wavelet Transforms[J]. IEEE Transactions on Cybernetics, 2022, 52(8): 8467-8480.
22	Li Jin, Li Wanyun, Xu Zichen, et al. Wavelet Transform-assisted Adaptive Generative Modeling for Colorization[J/OL]. IEEE Transactions on Multimedia. (2022-05-26) [2022-08-18]. .
23	Ch M M I, Riaz M M, Iltaf N, et al. Shift-invariant Discrete Wavelet Transform-based Sparse Fusion of Medical Images[J]. Signal Image and Video Processing, 2023, 17(4): 881-889.
24	Pappas T. Earth Terrain, Height, and Segmentation Map Images[EB/OL]. [2022-07-28]. .
25	Schoenfeld E, Sushko V, Zhang Dan, et al. You Only Need Adversarial Supervision for Semantic Image Synthesis[C]//International Conference on Learning Representations (ICLR) 2021. New York, USA: ICLR, 2021: 2686.
26	Park T, Liu Mingyu, Wang Tingchun, et al. Semantic Image Synthesis With Spatially-adaptive Normalization[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2019: 2332-2341.
27	Shi Bo, Ma Ziji, Liu Jianwei, et al. Shadow Extraction Method Based on Multi-information Fusion and Discrete Wavelet Transform[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-15.

Methods	SSIM	PSNR	FID
Backbone	0.756	20.271	160.891
Backbone + 1DWT	0.760	20.812	128.124
Backbone + 2DWT	0.762	20.876	109.362
Backbone + 4DWT	0.788	21.342	67.811
Backbone + 4DWT + 1Con	0.790	21.580	25.825
Ours(Backbone + 4DWT + 2Con)	0.824	22.953	21.024

[1]	Benyue Su, Li Zhang, Qingxuan He, Min Sheng. Short-time Human Activity Recognition Based on Wavelet Features Matching [J]. Journal of System Simulation, 2023, 35(1): 158-168.
[2]	Chen Tao, Wang Yan, Ji Zhicheng. Combination Forecasting Model of Photovoltaic Power Based on Empirical Wavelet Transform [J]. Journal of System Simulation, 2021, 33(11): 2627-2635.
[3]	Huang Sheng, Zhang Tao, Huang Chongjun, Li Yumei, Deng Hu, Zhang Xia. Downhole Drilling Processing Data Acquisition and Stick Slip Characteristic Analysis [J]. Journal of System Simulation, 2019, 31(11): 2517-2526.
[4]	Wang Yiren, Deng Guoqing, Xia Yingwei, Zhang Long, Liu Yong, Zhang Wen. Vascular Image Enhancement Using Steerable Filters [J]. Journal of System Simulation, 2018, 30(6): 2095-2101.
[5]	Yan Guoqiang, Zhou Ningning, Zhang Shaobai. De-noising Method of EEG Signal Based on MMTD and Wavelet Hard-threshold [J]. Journal of System Simulation, 2018, 30(4): 1490-1495.
[6]	Zhang Hongxi, Zhang Jie. Application of Wavelet Transform in Simulation Model Validation of Outboard Active Decoy [J]. Journal of System Simulation, 2018, 30(1): 318-324.
[7]	Zhou Guoxiong, Zhou Xianyan, Wang Jiejun, Huang Te. Wood Structure Nondestructive Detection Based on Wavelet Analysis Ant-colony BP Network [J]. Journal of System Simulation, 2015, 27(11): 2804-2810.
[8]	Li Hao, Li Ruihua, Hu Bo, Pan Ling, Guo Qiyi. Simulation Study on Damage Localization of Large Generator Stator Insulation Based on Guided Waves [J]. Journal of System Simulation, 2015, 27(11): 2816-2821.