Research Progress on Weapon and Equipment Effectiveness Evaluation

doi:10.16182/j.issn1004731x.joss.20-0442

Abstract

Abstract: Effectiveness is an important indicator to evaluate the weapon and equipment, and can support the decision-making of the weapon and equipment planning, scheme examination, design, development and operational application, et al. With the application of digital thread and digital twin in the life cycle of weapon and equipment, effectiveness evaluation faces a series of challenges, including the complex operational scenario, complicated data scene, expensive simulation, pervasive and agile evaluation, et al. The related concepts of weapon and equipment effectiveness and the development process of effectiveness evaluation are summarized. The problems of effectiveness evaluation are discussed. The solutions for the effectiveness evaluation of weapon and equipment are provided. The digital era research direction in the future is forecasted and summarized.

Key words: weapon &, equipment;, effectiveness evaluation, multi-resource data, modeling and simulation

CLC Number:

TP391.9

Zhou Yuchen, Lin Shengling, Ma Ping, Li Wei, Yang Ming. Research Progress on Weapon and Equipment Effectiveness Evaluation[J]. Journal of System Simulation, 2020, 32(8): 1413-1424.

References 75

[1]	Hootman J C, Whitcomb C.A military effectiveness analysis and decision making framework for naval ship design and acquisition[J]. Naval Engineers Journal (S0028-1425), 2005, 117(3): 43-61.
[2]	焦松. 武器装备效能仿真评估关键问题研究[D]. 哈尔滨: 哈尔滨工业大学, 2014.Jiao Song.Research on key problems of the simulation evaluation for weapon effectiveness[D]. Harbin: Harbin Institute of Technology, 2014.
[3]	WSEIAC. Final report of task group I: Requirement methodology[R]. 1965.
[4]	Dordick H S.An introduction to system effectiveness [R]. Rand. 1965.
[5]	Hayward P.The measurement of combat effectiveness[J]. Operations Research (S0030-364X), 1968, 16(2): 314-323.
[6]	Tillman F A, Lie C H, Hwang C L.Simulation model for mission effectiveness for military systems[J]. IEEE Transactions on Reliability (S0018-9529), 1978, 27(3): 191-194.
[7]	Rau J G.Measures of effectiveness handbook[R]. 1974.
[8]	Bouthonnier V, Levis A H.System effectiveness analysis of C3 systems[J]. IEEE Transactions on System, Man, and Cybernetics (S2168-2232), 1984, 14(1): 48-54.
[9]	Cpthier P H, Levis A H.Timeliness and measures of in command and control[J]. IEEE Transactions on System, Man, and Cybernetics (S2168-2232), 1986, 16(6): 844-853.
[10]	Sivazlian B D.Aircraft sortie effectiveness model[J]. Naval Research Logistics (S0894-069X), 1989, 36(2): 127-137.
[11]	Jung H D.Computer study of air defense gun effectiveness[D]. Monterey: Naval Postgraduate School. 1987.
[12]	Gallagher M A, Kelly E J.New methodology for military force structure analysis[J]. Operations Research (S0030-364X), 1991, 39(6): 877-885.
[13]	Gluckman A G, Celmins A.Cost effectiveness analysis using fuzzy set theory[R]. U.S. Army Research Laboratory. 1993.
[14]	Kohlberg I, Greer W.Weapons systems effectiveness and minimum cost for ballistic missile defense alternatives[C]. AIAA missile system conference. Reston: AIAA, 1996.
[15]	Bowden F, Gabrisch C, Davies M.C3I systems analysis using the distributed interactive C3I effectiveness (DICE) simulation[C]. IEEE International Conference on Systems, Man, and Cybernetics. Los Alamitos: IEEE, 1997: 4326-4331.
[16]	Pawlowski T J.Command, control, communications, intelligence, electronic warfare measures of effectiveness (C3IEW MOE) [R]. Military Operations Research Society. 1992.
[17]	Horn D, Colvert P, Scrip C.THAAD integrated system effectiveness simulation (TISES) theater high altitude area defense[C]. AIAA Modeling and Simulation Technologies Conference. Reston: AIAA, 1997.
[18]	Davis P K.Exploratory analysis enabled by multiresolultion multiperspective modeling[C]. Winter Simulation Conference. Los Alamitos: IEEE, 2000: 293-301.
[19]	Pinder J D.Evaluating future force options for the U.S. army[D]. Santa Monica: RAND Graduate School, 2000.
[20]	Hwang J S.Analysis of effectiveness of CEC (cooperative engagement capability) using Schutzer’s C2 theory[D]. Monterey: Naval Postgraduate School, 2003.
[21]	Lund F.Historic weapon system cost effectiveness comparison[C]. AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition. Reston: AIAA, 2009.
[22]	Poropudas J, Virtanen K.Analyzing air combat simulation results with dynamic Bayesian networks[C]. Winter Simulation Conference. Los Alamitos: IEEE, 2007: 1370-1377.
[23]	Gipson I G.Effectiveness of the U.S. missile defense capabilities as a deterrent to the North Korean missile[D]. Monterey: Naval Postgraduate School, 2007.
[24]	Davis P K, Shaver R D, Beck J.Portfolio-analysis methods for assessing capability options[R]. RAND. 2008.
[25]	Perry W, Button R W, Braken J, et al.Measures of effectiveness for the information-age navy: the effects of network-centric operations on combat outcomes[R]. Rand. 2002.
[26]	Darilek R, Perry W, Bracken J, et al.Measures of effectiveness for the information-age army[R]. Rand. 2001.
[27]	Combs R.Assessing strategic effectiveness in the war on terrorism[R]. U.S. Army War College. 2006.
[28]	Sutherland S P.Measuring effectiveness in conflict environments[D]. Monterey: Naval postgraduate school. 2009.
[29]	Stumborg M F.The impact of gun dimensions on the operational effectiveness of a naval surface fleet equipped with long range electromagnetic guns[J]. IEEE Transactions on Magnetics (S0018-9464), 2001, 37(1): 498-501.
[30]	Post J, Bennett M, Hall R.The cost and effectiveness of alternative space radar constellations[C]. Space. Reston: AIAA, 2006.
[31]	Fager C F.Weaponeering the future direct energy weapons effectiveness now and tomorrow[R]. Air University. 2007.
[32]	Ender T, Leurck R F, Weaver B, et al.Systems-of-systems analysis of ballistic missile defense architecture effectiveness through surrogate modeling and simulation[J]. IEEE Systems Journal (S2222-6362), 2010, 4(2): 156-166.
[33]	Mezzacappa E.Effectiveness testing of non-lethal weapons[J]. Journal of Defense Modeling and Simulation: Applications, Methodology, Technology (S1937-9234), 2014, 11(2): 91-101.
[34]	Au T A, Hoek P J, Lo S.Combat analysis of joint force options using agent-based simulation[C]. 2018 Military Communications and Information Systems Conference. Los Alamitos: IEEE, 2018.
[35]	Connors C D, Miller J O, Lunday B J.Using agent-based modeling and a designed experiment to simulate and analyze a new air-to-air missile[J]. Journal of Defense Modeling and Simulation: Applications, Methodology, Technology (S1548-5129), 2016, 13(3): 321-330.
[36]	Hendrickson P, Sarkani S, Mazzuchi T A.Benchmark requirement decision making tool to optimize effectiveness of weapon system modifications[J]. International Journal of Defense Acquisition Management (S1940-3445), 2013, 5(1): 1-19.
[37]	Grzesik N.Fuzzy sets in aircraft system efficiency evaluation[J]. Aircraft Engineering and Aerospace Technology (S1748-8842), 2016, 88(6): 707-716.
[38]	Jung W I, Lowe L, Rabelo L, et al.A methodology on guiding effectiveness-focused training of the weapon operator using big data and VC simulations[J]. SAE International Journal of Aerospace (S1687-5966), 2017, 10(2): 57-64.
[39]	Dillenburger S P, Jordan J D, Cochran J K.Pareto-optimality for lethality and collateral risk in the airstrike multiobjective problem[J]. Journal of the Operational Research Society (S0160-5682), 2019, 70(7): 1051-1064.
[40]	Hocaoğlu M F.Weapon target assignment optimization for land based multi-air defense systems: A goal programming approach[J]. Computers & Industrial Engineering (S0360-8352), 2019, 128: 681-689.
[41]	Mroczek A J.Determining the cost-effectiveness of nano satellites[D]. Monterey: Naval Postgraduate School, 2014.
[42]	Sabatini R.Innovative flight test instrumentation and techniques for airborne laser systems performance analysis and mission effectiveness evaluation[C]. IEEE international workshop on metrology for aerospace. Los Alamitos: IEEE, 2014.
[43]	Lee S D.Routing UAVS to Co-optimize mission effectiveness and network performance with dynamic programming[D]. Wright-Patterson AFB: Airforce Institute of Technology, 2011.
[44]	Menthe L, Hura M, Rhodes C.The effectiveness of remotely piloted aircraft in a permissive hunter-killer scenario[R]. Rand. 2014.
[45]	Kaymal T.Unmanned aircraft systems for maritime operations[C]. International Conference on Unmanned Aircraft Systems(ICUAS). Los Alamitos: IEEE, 2016: 763-768.
[46]	Ozcan B Y.Effectiveness of unmanned aerial vehicles in helping secure a border characterized by rough terrain and active terrorists[D]. Monterey: Naval Postgraduate School, 2013.
[47]	Domerçant J C, Mavris D.Understanding and evaluating command & control effectiveness by measuring battlespace awareness[C]. The 18th International Command & Control Research & Technology Symposium (ICCRTS). Alexandria: International C2 Institute, 2014.
[48]	Czarnecki J E.C2 effectiveness and operational success an exploration[C]. The 16th International Command and Control Research and Technology Symposium. Quebec: International C2 Institute, 2012.
[49]	Soh S S.Determining intelligence, surveillance and reconnaissance (ISR) system effectiveness, and integration as part of force protection and system survivability[D]. Monterey: Naval Postgraduate School, 2013.
[50]	Romich A, Lan G H, Smith J C.A robust sensor covering and communication problem[J]. Naval Research Logistics (S0894-069X), 2015, 62(7): 583-594.
[51]	Maybury M T.Global Horizons Final Report-United States Air force Global Science and Technology Vision[R]. U.S. Air Force. 2013.
[52]	张冰, 李欣, 万欣欣. 从数字孪生到数字工程建模仿真迈入新时代[J]. 系统仿真学报, 2019, 31(3): 369-376.Zhang Bing, Li Xin, Wan Xinxin.From digital twin to digital engineering modeling and simulation entering a new era. journal of system simulation[J]. Journal of System Simulation, 2019, 31(3): 369-376.
[53]	Watson M D, Kelley G W.Application of system operational effectiveness methodology to space launch vehicle development and operations[C]. Reston: AIAA, 2012.
[54]	裴云. 中程空地导弹武器系统作战效能评估[J]. 系统工程与电子技术, 2004, 26(10): 1435-1438.Pei Yun.Assessments of the middle-distance air-to-surface missile weapon systems’ combat efficiency[J]. Systems Engineering and Electronics, 2004, 26(10): 1435-1438.
[55]	孙海虹, 张平定. 预警机作战效能评估分析[J]. 弹箭与制导学报, 2004, 24(3): 399-401.Sun Haihong, Zhang Pingding.Analysis on the operational effective evaluation of early warning aircraft system[J]. Journal of Projectiles, Rockets, Missiles and Guidance, 2004, 24(3): 399-401.
[56]	黄俊, 武哲. 作战飞机的空-地攻击效能评估[J]. 航空学报, 1999, 20(1): 69-71.Huang Jun, Wu Zhe.Assessments of air-to-surface target attacking effectiveness for a combat aircraft[J]. Acta Aeronautica Et Astronautica Sinica, 1999, 20(1): 69-71.
[57]	关成启, 杨涤, 关世义. 导弹武器系统效能评估方法研究[J]. 系统工程与电子技术, 2000, 22(7): 32-36.Guan Chengqi, Yang Di, Guan Shiyi.Study of effectiveness evaluation method for guided weapon systems[J]. Systems Engineering and Electronics, 2000, 22(7): 32-36.
[58]	杨峰, 张学斌, 王维平, 等. 导弹作战效能仿真评估系统体系结构[J]. 国防科技大学学报, 1999, 21(1): 17-20.Yang Feng, Zhang Xuebin, Wang Weiping, et al.The architecture of the simulation-based evaluation system for missile effectiveness[J]. Journal of National University of Defense Technology, 1999, 21(1): 17-20.
[59]	许相莉, 胡晓峰. 一种基于复杂网络理论的网络空间作战效能评估指标体系框架[J]. 军事运筹与系统工程, 2014, 28(1): 33-41.
[60]	杨明, 焦松, 李伟. 武器作战效能参数化仿真评估技术研究[J]. 北京理工大学学报, 2013, 33(12): 1269-1273.Yang Ming, Jiao Shuang, Li Wei.Parameterized simulation evaluation technology for operational effectiveness of weapon[J]. Transactions of Beijing Institute of Technology, 2013, 33(12): 1269-1273.
[61]	周健, 龚春林, 粟华, 等. 飞行器体系优化设计问题[J]. 航空学报, 2018, 39(11): 102-114.Zhou Jian, Gong Chunlin, Su Hua, et al.Optimal design problem of system of systems of flight vehicle[J]. Acta Aeronautica Et Astronautica Sinica, 2018, 39(11): 102-114.
[62]	王庆龙, 王智学, 何红悦, 等. 基于模糊-云模型的C4ISR系统效能需求建模与分析方法[J]. 系统工程与电子技术, 2016, 38(9): 2065-2071.Wang Qinglong, Wang Zhixue, He Hongyue, et al.Modeling and analysis method to C4ISR system for efficiency requirements based on fuzzy cloud model[J]. Systems Engineering and Electronics, 2016, 38(9): 2065-2071.
[63]	Zhao D L, Dou Y J, Zhao Q S, et al.The method to evaluate the command and control effectiveness of operational system under uncertain threat situation[C]. Chinese Control and Decision Conference. Los Alamitos: IEEE, 2016: 5701-5706.
[64]	Zhang R W, Song B F, Pei Y, et al.Agent-based analysis of multi-uav area monitoring mission effectiveness[C]. AIAA AVIATION Forum-AIAA Modeling and Simulation Technologies Conference. Reston: AIAA, 2017.
[65]	Dong Z N, Zhang M Y, Liu Y M.Control Method of Manned/Unmanned Aerial Vehicle Cooperative Formation Based on Mission Effectiveness[C]. Chinese Guidance, Navigation and Control Conference. Los Alamitos: IEEE, 2016: 881-888.
[66]	钱晓超, 唐伟, 陈伟, 等. 面向关键能力的陆军全域作战体系贡献率评估[J]. 系统仿真学报, 2018, 30(12): 4786-4793.Qian Xiaochao, Tang Wei, Chen Wei, et al.Evaluation of contribution of army combined arms in all-domains weapon system-of-systems based on critical capability[J]. Journal of System Simulation, 2018, 30(12): 4786-4793.
[67]	杨克巍, 杨志伟, 谭跃进, 等. 面向体系贡献率的装备体系评估方法研究综述[J]. 系统工程与电子技术, 2019, 41(2): 311-321.Yang Kewei, Yang Zhiwei, Tan Yuejin, et al.Review of the evaluation methods of equipment system of systems facing the contribution rate[J]. Systems Engineering and Electronics, 2019, 41(2): 311-321.
[68]	罗承昆, 陈云翔, 王莉莉, 等. 基于作战环和改进信息熵的体系效能评估方法[J]. 系统工程与电子技术, 2019, 41(1): 73-80.Luo Chengkun, Chen Yunxiang, Wang Lili, et al.Effectiveness evaluation method of system-of-systems based on operation loop and improved information entropy[J]. Systems Engineering and Electronics, 2019, 41(1): 73-80.
[69]	林圣琳. 面向复杂仿真的评估与优化方法研究[D]. 哈尔滨: 哈尔滨工业大学, 2020.Lin Shenglin.Complex simulation-oriented assessment and optimization methods[D]. Harbin: Harbin Institute of Technology, 2020.
[70]	钱晓超. 考虑不确定性影响的仿真模型验证及校准方法研究[D]. 哈尔滨: 哈尔滨工业大学, 2016.Qian Xiaochao.Research on simulation model validation and calibration methods under uncertainty[D]. Harbin: Harbin Institute of Technology, 2016.
[71]	林圣琳, 李伟, 钱晓超, 等.基于复杂昂贵仿真的体系效能多目标优化[J/OL]. 控制与决策, 2019. 10.13195/j.kzyjc.2019.0844.Lin Shenglin, Li Wei, Qian Xiaochao, et al.Complex and expensive simulation based multi-objective optimization to system-of-system effectiveness[J/OL]. Control and Decision, 2019. 10.13195/j.kzyjc.2019.0844.
[72]	刘志钊. 武器装备效能仿真评估与优化方法研究[D]. 哈尔滨: 哈尔滨工业大学, 2016.Liu Zhizhao.Research on the simulation evaluation and optimization methods of weapon effectiveness[D] Harbin: Harbin Institute of Technology, 2016.
[73]	周玉臣. 复杂仿真模型验证方法研究[D]. 哈尔滨: 哈尔滨工业大学, 2019.Zhou Yuchen.Research on validation method for complex simulation model[D]. Harbin: Harbin Institute of Technology, 2019.
[74]	Fang K, Zhou Y C, Ma P.An adaptive sequential experiment design method for model validation[J]. Chinese Journal of Aeronautics (S1000-9361), 2020, 33(6): 1661-1672.
[75]	Ma P, Zhou Y C, Shang X B, et al.Firing accuracy evaluation of electromagnetic railgun based on multicriteria optimal Latin hypercube design[J]. IEEE Transactions on Plasma Science (S0093-3813), 2017, 45(7): 1503-1511.