Spatio-temporal Swin Transformer-based Flow-solid Coupling Interaction Sequence Image Prediction Network

doi:10.16182/j.issn1004731x.joss.25-0824

Abstract

Abstract:

To address limitations in modeling long-term dependencies and multi-scale features in fluid-structure interaction scenarios, a spatiotemporal deep learning model (SwinLSTM) integrating ConvLSTM and Swin Transformer is proposed. The model employs a gated spatiotemporal attention mechanism that dynamically embeds Swin Transformer's window-based multi-head self-attention into ConvLSTM's output gate, enabling adaptive temporal-spatial feature coupling, and designs a multi-level ConvLSTM framework to hierarchically capture complex spatiotemporal correlations. Experiments on a self-built fluid-interaction dataset show that our method achieves the highest PSNR and leading SSIM scores, with superior performance in preserving vortex details and boundary consistency. This work provides an efficient solution for fluid dynamics prediction in interactive scenarios.

Key words: deep learning, computational fluid dynamics, image prediction, ConvLSTM, Swin Transformer

CLC Number:

TP391.41

Zou Changjun, Ge Zhiyu, Zhong Chenxi. Spatio-temporal Swin Transformer-based Flow-solid Coupling Interaction Sequence Image Prediction Network[J]. Journal of System Simulation, 2026, 38(1): 112-124.

Figures/Tables 8

Fig. 1

Table 1

Table 2

Fig. 2

Fig. 3

Table 3

Table 4

Table 5

References 30

[1]	Wang Haixin, Cao Yadi, Huang Zijie, et al. Recent Advances on Machine Learning for Computational Fluid Dynamics: A Survey[EB/OL]. (2024-08-22) [2025-07-26]. .
[2]	Lino M, Fotiadis S, Bharath A A, et al. Current and Emerging Deep-learning Methods for the Simulation of Fluid Dynamics[J]. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2023, 479(2275): 20230058.
[3]	Cuomo S, Di Cola V S, Giampaolo F, et al. Scientific Machine Learning Through Physics-informed Neural Networks: Where We Are and What's Next[J]. Journal of Scientific Computing, 2022, 92(3): 88.
[4]	Lucas B D, Kanade T. An Iterative Image Registration Technique with an Application to Stereo Vision[C]//Proceedings of the 7th International Joint Conference on Artificial Intelligence. San Francisco: Morgan Kaufmann Publishers Inc., 1981: 674-679.
[5]	Box G E P, Jenkins G M. Time Series Analysis: Forecasting and Control[M]. Oakland: Holden-day, 1970.
[6]	Kalman R E. A New Approach to Linear Filtering and Prediction Problems[J]. Journal of Basic Engineering, 1960, 82(1): 35-45.
[7]	Cortes C, Vapnik V. Support-vector Networks[J]. Machine Learning, 1995, 20(3): 273-297.
[8]	Rumelhart D E, Hinton G E, Williams R J. Learning Representations by Back-propagating Errors[J]. Nature, 1986, 323(6088): 533-536.
[9]	Lecun Y, Bottou L, Bengio Y, et al. Gradient-based Learning Applied to Document Recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[10]	Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation[C]//Medical Image Computing and Computer-assisted Intervention – MICCAI 2015. Cham: Springer International Publishing, 2015: 234-241.
[11]	Elman J L. Finding Structure in Time[J]. Cognitive Science, 1990, 14(2): 179-211.
[12]	Hochreiter S, Schmidhuber J. Long Short-term Memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[13]	Cho K, Van Merrienboer B, Bahdanau D, et al. On the Properties of Neural Machine Translation: Encoder-decoder Approaches[EB/OL]. (2014-10-07) [2025-07-26]. .
[14]	Shi Xingjian, Chen Zhourong, Wang Hao, et al. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2015: 802-810.
[15]	Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2014: 2672-2680.
[16]	Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 6000-6010.
[17]	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale[EB/OL]. (2021-06-03) [2025-07-26]. .
[18]	Liu Ze, Lin Yutong, Cao Yue, et al. Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows[EB/OL]. (2021-08-17) [2025-07-26]. .
[19]	Geng Shaoyang, Zhai Shuo, Li Chengyong. Swin Transformer Based Transfer Learning Model for Predicting Porous Media Permeability from 2D Images[J]. Computers and Geotechnics, 2024, 168: 106177.
[20]	Ruan Weilin, Zhong Siru, Wen Haomin, et al. Vision-enhanced Time Series Forecasting via Latent Diffusion Models[EB/OL]. (2025-02-16) [2025-07-26]. .
[21]	Guan Shanyan, Deng Huayu, Wang Yunbo, et al. NeuroFluid: Fluid Dynamics Grounding with Particle-driven Neural Radiance Fields[C]//Proceedings of the 39th International Conference on Machine Learning. Chia Laguna Resort: PMLR, 2022: 7919-7929.
[22]	Zhu M, Bazaga A, Liò P. FLUID-LLM: Learning Computational Fluid Dynamics with Spatiotemporal-aware Large Language Models[EB/OL]. (2024-06-06) [2025-07-26]. .
[23]	Zhang Bailing. Koopman Framework with Self-supervised Spectral Alignment for Multi-domain Time-series Modeling and Prediction[J]. Neurocomputing, 2025, 652: 131109.
[24]	Raissi M, Perdikaris P, Karniadakis G E. Physics-informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations[J]. Journal of Computational Physics, 2019, 378: 686-707.
[25]	肖祥云, 杨旭波. 基于物理及数据驱动的流体动画研究[J]. 软件学报, 2020, 31(10): 3251-3265.
	Xiao Xiangyun, Yang Xubo. Physically-based and Data-driven Fluid Simulation Research[J]. Journal of Software, 2020, 31(10): 3251-3265.
[26]	Li Ao, Zhang Wanshun, Zhang Xiao, et al. A Deep U-Net-ConvLSTM Framework with Hydrodynamic Model for Basin-scale Hydrodynamic Prediction[J]. Water, 2024, 16(5): 625.
[27]	Foster N, Fedkiw R. Practical Animation of Liquids[C]//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. New York: Association for Computing Machinery, 2001: 23-30.
[28]	Stam J. Stable Fluids[C]//Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. USA: ACM Press/Addison-wesley Publishing Co., 1999: 121-128.
[29]	Fedkiw R, Stam J, Jensen H W. Visual Simulation of Smoke[C]//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. New York: Association for Computing Machinery, 2001: 15-22.
[30]	Müller M, Charypar D, Gross M. Particle-based Fluid Simulation for Interactive Applications[C]//Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Goslar: Eurographics Association, 2003: 154-159.

scens	abbreviation	sample size	size	format	resolution
Simple stationary solid	SSS	3 000	351 M	PNG	256×256
Complex stationary solids	CSS	2 000	1.1 G	PNG	512×512
Fine motor skills	FMS	2 500	12.7 G	PNG	1 024×1 024
Simple motion solids	SMS	3 000	346 M	PNG	256×256
Complex motion solids	CMS	2 000	1.1 G	PNG	512×512

Method	SSS		CSS		FMS		SMS		CMS		Param	FLOPs
Method	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	Param	FLOPs
ResNet	30.423 79	0.925 73	30.459 19	0.944 72	26.399 58	0.830 61	27.917 62	0.891 05	27.189 96	0.889 57	113 M	0.63 G
UNet	41.516 35	0.976 63	42.839 81	0.990 65	37.907 98	0.974 28	40.172 34	0.979 38	35.082 20	0.961 85	118 M	1.78 G
ViT	24.371 35	0.851 09	39.475 74	0.989 30	32.812 23	0.954 65	22.617 29	0.839 15	29.341 60	0.939 72	124 M	0.61 G
Swin	42.294 05	0.981 35	43.186 84	0.990 00	36.652 52	0.963 81	39.618 30	0.976 09	34.041 74	0.942 67	147 M	0.14 G
LSTM	40.966 93	0.976 29	37.071 56	0.981 36	35.872 86	0.959 57	41.137 11	0.982 39	33.064 14	0.954 74	118 M	0.61 G
Ours	43.474 31	0.984 29	43.353 44	0.991 35	38.200 63	0.977 36	41.938 44	0.981 18	35.415 44	0.957 06	149 M	0.63 G

Setting	SMS		Param	Flops
Setting	PSNR/dB	SSIM	Param	Flops
ViT-style MSA	23.447 81	0.842 64	153 M	0.17 G
Conv2D	38.729 75	0.944 66	126 M	0.63 G
W-MSA	41.938 44	0.981 18	149 M	0.63 G

Setting	SMS		Param	Flops
Setting	PSNR/dB	SSIM	Param	Flops
单层ConvLSTM	34.630 26	0.971 30	137 M	0.34 G
多级ConvLSTM	41.938 44	0.981 18	149 M	0.63 G

Setting	SMS		Param	Flops
Setting	PSNR/dB	SSIM	Param	Flops
L1 loss	34.219 41	0.967 39	149 M	0.63 G
MSE loss	41.938 44	0.981 18	149 M	0.63 G