基于光线追踪的实时超声模拟与虚拟现实的集成

doi:10.16182/j.issn1004731x.joss.21-0180

系统仿真学报 ›› 2022, Vol. 34 ›› Issue (11): 2425-2436.doi: 10.16182/j.issn1004731x.joss.21-0180

• 仿真支撑平台/系统技术 • 上一篇下一篇

基于光线追踪的实时超声模拟与虚拟现实的集成

彭博¹^,²^,³(), 汪强¹, 青芮冰¹, 尹立雪²^,³, 姜劲枫⁴

^1.西南石油大学，计算机科学学院，四川　成都　610500
^2.四川省医学科学院·四川省人民医院　心血管超声及心功能科，四川　成都　610072
^3.四川省医学科学院·四川省人民医院　超声心脏电生理学与生物力学四川省重点实验室，四川　成都　610072
^4.美国密歇根理工大学，生物医学工程系，美国密歇根州　霍顿　49931

收稿日期:2021-03-08 修回日期:2021-04-28 出版日期:2022-11-18 发布日期:2022-11-25
作者简介:彭博(1980-)，男，博士，教授，研究方向为医学超声成像、医学图像与信号分析。E-mail：bopeng@swpu.edu.cn
基金资助:
四川省科技厅应用基础研究(2021YJ0248);四川省成都市科技局国际合作项目(2019-GH02-00040-HZ)

A Real-time Ultrasound Simulation Platform Using Ray Tracing and Its Integration with Virtual Reality

Bo Peng¹^,²^,³(), Qiang Wang¹, Ruibing Qing¹, Lixue Yin²^,³, Jingfeng Jiang⁴

^1.School of Computer Science, Southwest Petroleum University, Chengdu 610500, China
^2.Dept. of Echo-cardiology and Non-invasive Cardiology, Sichuan Academy of Medical Science, Sichuan Provincial People's Hospital, Chengdu 610072, China
^3.Sichuan Academy of Medical Sciences·Sichuan Provincial People's Hospital, Sichuan Provincial Key Laboratory of Echocardiography and Biomechanics, Chengdu 610072, China
^4.Dept. of Biomedical Engineering, Michigan Tech University, Houghton, MI 49931, USA

Received:2021-03-08 Revised:2021-04-28 Online:2022-11-18 Published:2022-11-25

摘要/Abstract

摘要：

为进一步提高超声医师培训效率以及降低培训成本，开发了一种与虚拟现实(VR)环境集成在一起的超声训练系统。将基于光线追踪的超声图像模拟方法整合到虚拟现实环境中，利用沉浸式VR体验进行医学超声培训，并将该方法获得的模拟超声图像与使用基于生成对抗网络(GAN)技术和Field II超声模拟器模拟的超声图像进行比较。实验数据表明，本文的超声模拟器在光线数量为256，模型的三角片数量超过200 000的条件下，能够以43帧/s的速度实时生成高质量的模拟超声图像，同时验证了光线追踪结合VR环境进行超声模拟是可行的。

关键词: 虚拟现实, 超声模拟, 医学图像模拟, 光线追踪, 超声训练检查

Abstract:

In order to further improve the efficacy of ultrasound training and reduce the cost. An ultrasound training system that is integrated with a VR environment is developed. The main contribution of this study is to incorporate the Ray-tracing based ultrasound image simulation approach into a virtual reality environment, taking advantage of immersive VR experience for medical ultrasound training. The simulated ultrasound images obtained by the proposed method are then compared to images that are simulated using a generative adversarial network (GAN) technique and Field II ultrasound simulator. The data show that the ultrasound simulator can produce high-quality simulated ultrasound images in real-time at 43 frames/s by tracing over 256 rays through numerical phantom models with more than 200 000 triangles, at the same time, it is verified that ray tracing combined with VR environment for ultrasonic simulation is feasible.

Key words: virtual reality, ultrasound simulation, medical image simulation, ray-tracing, sonography training

中图分类号:

TP391.9

彭博, 汪强, 青芮冰, 尹立雪, 姜劲枫. 基于光线追踪的实时超声模拟与虚拟现实的集成[J]. 系统仿真学报, 2022, 34(11): 2425-2436.

Bo Peng, Qiang Wang, Ruibing Qing, Lixue Yin, Jingfeng Jiang. A Real-time Ultrasound Simulation Platform Using Ray Tracing and Its Integration with Virtual Reality[J]. Journal of System Simulation, 2022, 34(11): 2425-2436.

图/表 13

图1

图2

图3

图4

表1

表2

表3

图5

图6

图7

表4

图8

图9

参考文献 27

[1]	Hayward M, Chan T, Healey A. Dedicated Time for Deliberate Practice: One Emergency Medicine Program's Approach to point-of-care Ultrasound (PoCUS) Training[J]. CJEM: Canadian Journal of Emergency Medicine (S1481-8035), 2015, 17(5): 558-561.
[2]	Heer I M, Middendorf K, Müller-Egloff S, et al. Ultrasound Training: The Virtual Patient[J]. Ultrasound in Obstetrics & Gynecology: The Official Journal of the International Society of Ultrasound in Obstetrics and Gynecology (S0960-7692), 2004, 24(4): 440-444.
[3]	Nicastro I, Barletta V, Conte L, et al. Professional Education, Training and Role of the Cardiac Sonographer in Different Countries[J]. Journal of Cardiovascular Echography (S2211-4122), 2013, 23(1): 18-23.
[4]	Tafra L. The Learning Curve and Sentinel Node Biopsy[J]. American Journal of Surgery (S0002-9610), 2001, 182(4): 347-350.
[5]	Gardner C J, Brown S, Hagen-Ansert S, et al. Guidelines for Cardiac Sonographer Education: Report of the American Society of Echocardiography Sonographer Education and Training Committee[J]. Journal of the American Society of Echocardiography (S0894-7317), 1992, 14(6): 77-84.
[6]	Blum T, Rieger A, Navab N, et al. A Review of Computer-Based Simulators for Ultrasound Training[J]. Simulation in Healthcare (S1559-2332), 2013, 8(2): 98-108.
[7]	Mattausch O, Goksel O. Monte-Carlo Ray-Tracing for Realistic Interactive Ultrasound Simulation[C]//Eurographics Workshop on Visual Computing for Biology and Medicine. Bergen, Norway: Eurographics Association, 2016: 173-181.
[8]	Dror Aiger, Daniel Cohen-Or. Real-Time Ultrasound Imaging Simulation[J]. Real-Time Imaging (S1077-2014), 1998, 4(4): 263-274.
[9]	Troccaz J, Henry D, Laieb N, et al. Simulators for Medical Training: Application to Vascular Ultrasound Imaging[J]. Journal of Visualization and Computer Animation (S1049-8907), 2000, 11(1): 51-65.
[10]	D’ Aulignac D, Laugier C, Troccaz J, et al. Towards a Realistic Echographic Simulator[J]. Medical Image Analysis (S1361-8415), 2006, 10(1): 71-81.
[11]	Goksel O, Salcudean S E. B-Mode Ultrasound Image Simulation in Deformable 3-D Medium[J]. IEEE Transactions on Medical Imaging (S0278-0062), 2009, 28(11): 1657-1669.
[12]	Jensen J A, Svendsen N B. Calculation of Pressure Fields from Arbitrarily Shaped, Apodized, and Excited Ultrasound Transducers[J]. IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control (S0885-3010), 1992, 39(2): 262-267.
[13]	Treeby B E, Cox B T. k-Wave: MATLAB Toolbox for the Simulation and Reconstruction of Photoacoustic Wave Fields[J]. Journal of Biomedical Optics (S1560-2281), 2010, 15(2): 021314.
[14]	Alles E J, Zhu Y, van Dongen K W A, et al. Rapid Transient Pressure Field Computations in the Nearfield of Circular Transducers Using Frequency-Domain Time-Space Decomposition[J]. Ultrasonic imaging (S0161-7346), 2012, 34(4): 237-260.
[15]	Zemp R J, Abbey C K, Insana M F. Linear System Models for Ultrasonic Imaging: Application to Signal Statistics[J]. IEEE Transactions on Ultrasonics Ferroelectrics & Frequency Control (S0885-3010), 2003, 50(6): 642-654.
[16]	Burger B, Bettinghausen S, Radle M, et al. Real-Time GPU-Based Ultrasound Simulation Using Deformable Mesh Models[J]. IEEE Transactions on Medical Imaging (S0278-0062), 2013, 32(3): 609-618.
[17]	Hu Y, Gibson E, Lee L L, et al. Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks[C]//Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment. Québec City, QC, Canada: Springer, Cham, 2017: 105-115.
[18]	Tom Francis, Sheet Debdoot. Simulating Patho- Realistic Ultrasound Images Using Deep Generative Networks with Adversarial Learning[C]//2018 IEEE 15^th International Symposium on Biomedical Imaging (ISBI 2018). Washington DC: IEEE, 2018: 1174-1177.
[19]	Peng B, Huang X, Wang S, et al. A Real-Time Medical Ultrasound Simulator Based on a Generative Adversarial Network Model[C]//2019 IEEE International Conference on Image Processing (ICIP). Taipei, China: IEEE, 2019: 4629-4633.
[20]	Hall R A, Greenberg D P. A Testbed for Realistic Image Synthesis[J]. Computer Graphics & Applications IEEE (S0272-1716), 1983, 3(8): 10-20.
[21]	Jan Novák, Dachsbacher C. Rasterized Bounding Volume Hierarchies[J]. Computer Graphics Forum (S0167-7055), 2012, 31(2): 403-412.
[22]	Benny Buerger, Ciamak Abkai, Juergen Hesser. Simulation of Dynamic Ultrasound Based on CT Models for Medical Education[J]. Studies in Health Technology and Informatics (S0926-9630), 2008(132): 56-61.
[23]	Parker K J. Shapes and Distributions of Soft Tissue Scatterers(Article)[J]. Physics in Medicine and Biology (S0031-9155), 2019, 64: 17.
[24]	Richard G Barr, Kazutaka Nakashima, Dominique Amy, et al. WFUMB Guidelines and Recommendations for Clinical Use of Ultrasound Elastography: Part 2; Breast[J]. Ultrasound in Medicine & Biology (S1879-291X), 2015, 41(5): 1148-1160.
[25]	Coolbaugh C L, Bush E C, Caskey C F, et al. FloWaveUS: Validated, Open-Source, and Flexible Software for Ultrasound Blood Flow Analysis[J]. Journal of Applied Physiology (S8750-7587), 2016, 121(4): 849-857.
[26]	Wang Y, Helminen E, Jiang J. Building a Virtual Simulation Platform for Quasistatic Breast Ultrasound Elastography Using Open Source Software: A Preliminary Investigation[J]. Medical Physics (S0094-2405), 2015, 42(9): 5453-5466.
[27]	Wang Y, Peng B, Jiang J. Building an Open-Source Simulation Platform of Acoustic Radiation Force-Based Breast Elastography[J]. Physics in Medicine & Biology (S0031-9155), 2017, 62(5): 1949-1968.

组织	Z/(g/(cm²·s))	α/(dB/MHz)	σ₀	μ₀	μ₁
背景	1.99×10⁵	0	0	0	0
肌肉	1.63×10⁵	2	0.0	0.53	0.51
血液	1.61×10⁵	0.18	0.01	0.01	0

参数名称	参数含义
nr	光线数量
nt	三角片数量
h	组织厚度
Z	组织声阻抗
α	组织衰减系数
c	声速
σ₀	散斑sigma值
μ₀	散斑均值
μ₁	散斑密度

模型区域	Z/ (g/(cm²·s))	α/ (dB/MHz)	σ₀	μ₀	μ₁
背景	1.99×10⁵	0	0	0	0
高回声目标	1.63×10⁵	2.00	0.0	0.53	0.51
低回声目标	1.58×10⁵	1.09	0.20	0.20	0.10

场景	三角片	光线数量
场景	三角片	128	256	512	1 024	2 048
1	742	55.5	40.5	23.4	12.3	6.1
	164 726	66.5	40.0	22.7	13.6	7.0
	236 252	63	43.0	23.0	12.3	6.8
2	15 882	82	47.7	27.7	15.5	8.0
	60 750	76	52.3	28.5	14.5	7.4
	94 820	78	50.0	27.4	13.5	7.7

基于光线追踪的实时超声模拟与虚拟现实的集成

A Real-time Ultrasound Simulation Platform Using Ray Tracing and Its Integration with Virtual Reality

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 27

相关文章 15

编辑推荐

Metrics

本文评价

[1]	周晓光, 朱鹏, 张源原, 路欢, 周媛. 六自由度跳伞模拟训练系统设计与实现[J]. 系统仿真学报, 2022, 34(6): 1320-1329.
[2]	许世健, 赵丹, 苏铖宇, 王赋攀, 张晓蓉, 王昉, 吴亚东. 面向流场可视化的沉浸式虚拟现实交互系统研究[J]. 系统仿真学报, 2022, 34(5): 1160-1172.
[3]	邵绪强, 张浩伟, 冯小华. 面向电力变压器虚拟装配的多感官融合交互方法[J]. 系统仿真学报, 2022, 34(10): 2244-2254.
[4]	孙佳文, 任鸿翔, 肖方兵, 蒋效彬. 基于虚拟现实的整船仿真训练平台[J]. 系统仿真学报, 2021, 33(9): 2243-2251.
[5]	魏士松, 周正东, 章栩苓, 毛玲, 贾峻山, 刘传乐. 基于桌面虚拟现实技术的航天器虚拟维修训练系统[J]. 系统仿真学报, 2021, 33(6): 1358-1363.
[6]	贾春洋, 邹湘军, 李锦慧, 曾泽钦, 王杰. 虚拟环境下的复杂变速箱交互式装配仿真平台[J]. 系统仿真学报, 2020, 32(9): 1736-1743.
[7]	郭园, 童倩倩, 郑宇铠, 王子琦, 张玉茹, 王党校. 融合多元触觉和沉浸式视觉的可移植VR软件框架[J]. 系统仿真学报, 2020, 32(7): 1385-1392.
[8]	文国军, 夏雨, 王玉丹, 胡自飞, 吴丹. 基于Unity3D的水平定向钻机虚拟实训系统设计[J]. 系统仿真学报, 2020, 32(5): 801-807.
[9]	刘畅, 贾金原, 陆一凡, 张乾, 赵磊. 光图树：Web3D动态场景全局光照的协同实时渲染[J]. 系统仿真学报, 2019, 31(8): 1591-1604.
[10]	谭力恒, 马强, 蒋秉川, 李锋. 面向头盔式虚拟现实的OsgEarth立体视觉生成方法[J]. 系统仿真学报, 2019, 31(7): 1290-1299.
[11]	祁彬斌, 朱学芳, 范丽鹏. 基于触觉交互的三维弹性物体形变仿真算法[J]. 系统仿真学报, 2019, 31(11): 2296-2305.
[12]	闫丰亭, 史志才, 苏前敏, 董婧, 陈珏, 石蕴玉, 汤显. 物联网与虚拟现实融合的智能火灾营救机制[J]. 系统仿真学报, 2019, 31(11): 2306-2322.
[13]	焦欢欢, 潘志庚, 徐舒畅, 袁庆曙. 基于VR的言语失用症康复训练系统[J]. 系统仿真学报, 2019, 31(11): 2344-2349.
[14]	张波, 孙俊, 商蕾, 徐弘基, 杨诚. 基于HMD的船舶虚拟消防训练系统设计及实现[J]. 系统仿真学报, 2019, 31(1): 43-52.
[15]	刘婷婷, 刘箴, 钱平安, 宣荣荣, 王瑾, 柴艳杰. 虚拟现实在特殊人群康复中的应用研究[J]. 系统仿真学报, 2018, 30(9): 3229-3237.