A Real-time Ultrasound Simulation Platform Using Ray Tracing and Its Integration with Virtual Reality

doi:10.16182/j.issn1004731x.joss.21-0180

Abstract

Abstract:

In order to further improve the efficacy of ultrasound training and reduce the cost. An ultrasound training system that is integrated with a VR environment is developed. The main contribution of this study is to incorporate the Ray-tracing based ultrasound image simulation approach into a virtual reality environment, taking advantage of immersive VR experience for medical ultrasound training. The simulated ultrasound images obtained by the proposed method are then compared to images that are simulated using a generative adversarial network (GAN) technique and Field II ultrasound simulator. The data show that the ultrasound simulator can produce high-quality simulated ultrasound images in real-time at 43 frames/s by tracing over 256 rays through numerical phantom models with more than 200 000 triangles, at the same time, it is verified that ray tracing combined with VR environment for ultrasonic simulation is feasible.

Key words: virtual reality, ultrasound simulation, medical image simulation, ray-tracing, sonography training

CLC Number:

TP391.9

Bo Peng, Qiang Wang, Ruibing Qing, Lixue Yin, Jingfeng Jiang. A Real-time Ultrasound Simulation Platform Using Ray Tracing and Its Integration with Virtual Reality[J]. Journal of System Simulation, 2022, 34(11): 2425-2436.

Figures/Tables 13

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Table 2

Table 3

Fig. 5

Fig. 6

Fig. 7

Table 4

Fig. 8

Fig. 9

References 27

[1]	Hayward M, Chan T, Healey A. Dedicated Time for Deliberate Practice: One Emergency Medicine Program's Approach to point-of-care Ultrasound (PoCUS) Training[J]. CJEM: Canadian Journal of Emergency Medicine (S1481-8035), 2015, 17(5): 558-561.
[2]	Heer I M, Middendorf K, Müller-Egloff S, et al. Ultrasound Training: The Virtual Patient[J]. Ultrasound in Obstetrics & Gynecology: The Official Journal of the International Society of Ultrasound in Obstetrics and Gynecology (S0960-7692), 2004, 24(4): 440-444.
[3]	Nicastro I, Barletta V, Conte L, et al. Professional Education, Training and Role of the Cardiac Sonographer in Different Countries[J]. Journal of Cardiovascular Echography (S2211-4122), 2013, 23(1): 18-23.
[4]	Tafra L. The Learning Curve and Sentinel Node Biopsy[J]. American Journal of Surgery (S0002-9610), 2001, 182(4): 347-350.
[5]	Gardner C J, Brown S, Hagen-Ansert S, et al. Guidelines for Cardiac Sonographer Education: Report of the American Society of Echocardiography Sonographer Education and Training Committee[J]. Journal of the American Society of Echocardiography (S0894-7317), 1992, 14(6): 77-84.
[6]	Blum T, Rieger A, Navab N, et al. A Review of Computer-Based Simulators for Ultrasound Training[J]. Simulation in Healthcare (S1559-2332), 2013, 8(2): 98-108.
[7]	Mattausch O, Goksel O. Monte-Carlo Ray-Tracing for Realistic Interactive Ultrasound Simulation[C]//Eurographics Workshop on Visual Computing for Biology and Medicine. Bergen, Norway: Eurographics Association, 2016: 173-181.
[8]	Dror Aiger, Daniel Cohen-Or. Real-Time Ultrasound Imaging Simulation[J]. Real-Time Imaging (S1077-2014), 1998, 4(4): 263-274.
[9]	Troccaz J, Henry D, Laieb N, et al. Simulators for Medical Training: Application to Vascular Ultrasound Imaging[J]. Journal of Visualization and Computer Animation (S1049-8907), 2000, 11(1): 51-65.
[10]	D’ Aulignac D, Laugier C, Troccaz J, et al. Towards a Realistic Echographic Simulator[J]. Medical Image Analysis (S1361-8415), 2006, 10(1): 71-81.
[11]	Goksel O, Salcudean S E. B-Mode Ultrasound Image Simulation in Deformable 3-D Medium[J]. IEEE Transactions on Medical Imaging (S0278-0062), 2009, 28(11): 1657-1669.
[12]	Jensen J A, Svendsen N B. Calculation of Pressure Fields from Arbitrarily Shaped, Apodized, and Excited Ultrasound Transducers[J]. IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control (S0885-3010), 1992, 39(2): 262-267.
[13]	Treeby B E, Cox B T. k-Wave: MATLAB Toolbox for the Simulation and Reconstruction of Photoacoustic Wave Fields[J]. Journal of Biomedical Optics (S1560-2281), 2010, 15(2): 021314.
[14]	Alles E J, Zhu Y, van Dongen K W A, et al. Rapid Transient Pressure Field Computations in the Nearfield of Circular Transducers Using Frequency-Domain Time-Space Decomposition[J]. Ultrasonic imaging (S0161-7346), 2012, 34(4): 237-260.
[15]	Zemp R J, Abbey C K, Insana M F. Linear System Models for Ultrasonic Imaging: Application to Signal Statistics[J]. IEEE Transactions on Ultrasonics Ferroelectrics & Frequency Control (S0885-3010), 2003, 50(6): 642-654.
[16]	Burger B, Bettinghausen S, Radle M, et al. Real-Time GPU-Based Ultrasound Simulation Using Deformable Mesh Models[J]. IEEE Transactions on Medical Imaging (S0278-0062), 2013, 32(3): 609-618.
[17]	Hu Y, Gibson E, Lee L L, et al. Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks[C]//Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment. Québec City, QC, Canada: Springer, Cham, 2017: 105-115.
[18]	Tom Francis, Sheet Debdoot. Simulating Patho- Realistic Ultrasound Images Using Deep Generative Networks with Adversarial Learning[C]//2018 IEEE 15^th International Symposium on Biomedical Imaging (ISBI 2018). Washington DC: IEEE, 2018: 1174-1177.
[19]	Peng B, Huang X, Wang S, et al. A Real-Time Medical Ultrasound Simulator Based on a Generative Adversarial Network Model[C]//2019 IEEE International Conference on Image Processing (ICIP). Taipei, China: IEEE, 2019: 4629-4633.
[20]	Hall R A, Greenberg D P. A Testbed for Realistic Image Synthesis[J]. Computer Graphics & Applications IEEE (S0272-1716), 1983, 3(8): 10-20.
[21]	Jan Novák, Dachsbacher C. Rasterized Bounding Volume Hierarchies[J]. Computer Graphics Forum (S0167-7055), 2012, 31(2): 403-412.
[22]	Benny Buerger, Ciamak Abkai, Juergen Hesser. Simulation of Dynamic Ultrasound Based on CT Models for Medical Education[J]. Studies in Health Technology and Informatics (S0926-9630), 2008(132): 56-61.
[23]	Parker K J. Shapes and Distributions of Soft Tissue Scatterers(Article)[J]. Physics in Medicine and Biology (S0031-9155), 2019, 64: 17.
[24]	Richard G Barr, Kazutaka Nakashima, Dominique Amy, et al. WFUMB Guidelines and Recommendations for Clinical Use of Ultrasound Elastography: Part 2; Breast[J]. Ultrasound in Medicine & Biology (S1879-291X), 2015, 41(5): 1148-1160.
[25]	Coolbaugh C L, Bush E C, Caskey C F, et al. FloWaveUS: Validated, Open-Source, and Flexible Software for Ultrasound Blood Flow Analysis[J]. Journal of Applied Physiology (S8750-7587), 2016, 121(4): 849-857.
[26]	Wang Y, Helminen E, Jiang J. Building a Virtual Simulation Platform for Quasistatic Breast Ultrasound Elastography Using Open Source Software: A Preliminary Investigation[J]. Medical Physics (S0094-2405), 2015, 42(9): 5453-5466.
[27]	Wang Y, Peng B, Jiang J. Building an Open-Source Simulation Platform of Acoustic Radiation Force-Based Breast Elastography[J]. Physics in Medicine & Biology (S0031-9155), 2017, 62(5): 1949-1968.

组织	Z/(g/(cm²·s))	α/(dB/MHz)	σ₀	μ₀	μ₁
背景	1.99×10⁵	0	0	0	0
肌肉	1.63×10⁵	2	0.0	0.53	0.51
血液	1.61×10⁵	0.18	0.01	0.01	0

参数名称	参数含义
nr	光线数量
nt	三角片数量
h	组织厚度
Z	组织声阻抗
α	组织衰减系数
c	声速
σ₀	散斑sigma值
μ₀	散斑均值
μ₁	散斑密度

模型区域	Z/ (g/(cm²·s))	α/ (dB/MHz)	σ₀	μ₀	μ₁
背景	1.99×10⁵	0	0	0	0
高回声目标	1.63×10⁵	2.00	0.0	0.53	0.51
低回声目标	1.58×10⁵	1.09	0.20	0.20	0.10

场景	三角片	光线数量
场景	三角片	128	256	512	1 024	2 048
1	742	55.5	40.5	23.4	12.3	6.1
	164 726	66.5	40.0	22.7	13.6	7.0
	236 252	63	43.0	23.0	12.3	6.8
2	15 882	82	47.7	27.7	15.5	8.0
	60 750	76	52.3	28.5	14.5	7.4
	94 820	78	50.0	27.4	13.5	7.7