Review of System of Systems Combat Effectiveness Evaluation and Optimization Methods

doi:10.16182/j.issn1004731x.joss.21-0225

Abstract

Abstract:

The characteristics of system-of-systems combat effectiveness evaluation and optimization are analyzed. In light of the "holism", this paper proposes an idea of dividing system of systems combat effectiveness evaluation and optimization into three stages of comprehensive evaluation, analysis, and optimization. As for the practical problems that need to be solved in the three stages, typical methods suitable for each stage are summarized.The advantages and disadvantages of different methods are then compared. In view of the practical difficulties in implementing system of systems combat effectiveness evaluation and optimization guided by the "holism", this paper puts forward the next research directions from the perspectives of the design of a comprehensive application framework, the construction of an evaluation index system, and the innovation in the experimental deduction mode to provide support for the effective implementation of system of systems combat effectiveness evaluation and optimization methods.

Key words: system of systems, holism, combat effectiveness evaluation, combat effectiveness analysis, combat effectiveness optimization

CLC Number:

TP391

Ziwei Zhang, Qisheng Guo, Zhiming Dong, Ang Gao, Yifei Wang. Review of System of Systems Combat Effectiveness Evaluation and Optimization Methods[J]. Journal of System Simulation, 2022, 34(2): 303-313.

Figures/Tables 1

References 49

1	Kumar P, Merzouki R, Bouamama B O. Multilevel Modeling of System of Systems[J]. IEEE Trans. on Systems, Man and Cybernetics: Systems (S2168-2216), 2018, 48(8): 1309-1320.
2	Silva E, Batista T, Oquendo F. A Mission-oriented Approach for Designing System-of-systems[C]// Proc. of the 10th System of Systems Engineering Conference, 2015: 345-351.
3	Dou Y J, Zhou Z X, Zhao D L, et al. Weapons System Portfolio Selection Based on the Contribution Rate Evaluation of System of Systems[J]. Journal of Systems Engineering and Electronics(S1004-4132), 2019, 30(5): 905-919.
4	孙明. 数字化部队作战效能的系统评估与分析研究[D]. 北京: 装甲兵工程学院, 2010.
	Sun Ming. Research on the System Evaluation and Analysis of Combat Effectiveness for the Digital Army [D]. Beijing: Academy of Armored Forces Engineering, 2010.
5	Edgeworth F Y. Statistics of Examinations[J]. Journal of the Royal Statistical Society (S1467-985X), 1923, 86(1): 59-60.
6	卜广志. 基于AOE模型的装备对作战体系的贡献率评估方法[J]. 火力与指挥控制, 2020, 45(12): 18-22.
	Bu Guangzhi. Evaluation Method of Equipment Contribution Rate to Combat System Based on AOE Model [J]. Fire Control & Command Control, 2020, 45 (12): 18-22
7	缪建明, 沈丙振,贾广威. 一种基于任务完成度的体系贡献率评估方法[C]//第八届中国指挥控制大会论文集: 兵器工业出版社, 2020: 589-593.
	Miao Jianming, Shen Bingzhen, Jia Guangwei, et al. An Evaluation Method of System Contribution Rate Based on Task Completion Degree [C]// Proceedings of the 8th China Command and Control Conference: Ordnance Industry Press, 2020: 589-593.
8	陈奇, 姜宁, 吕明山,等. 基于效能的海军演习效果评估方法及关键技术[J]. 系统工程与电子技术, 2013, 35(6): 1226-1230.
	Chen Qi, Jiang Ning, Lu Mingshan, et al. Effectiveness Evaluation Method and Key Technologies of Naval Exercises Based on Effectiveness [J]. Systems Engineering and Electronics, 2013, 35 (6): 1226-1230.
9	Qi Z F, Wang G S. Effectiveness Evaluation of Electronic Warfare Command and Control System Based on Grey AHP Method[J]. Journal of Chemical and Pharmaceutical Research (S0975-7384), 2014, 6(7): 535-542.
10	Chang L L, Zhou Z J, Chen Y W, et al. Belief Rule Base Structure and Parameter Joint Optimization under Disjunctive Assumption for Nonlinear Complex System Modeling[J]. IEEE Trans. on Systems, Man & Cybernetics: Systems (S2168-2216), 2018, 48(9): 1542-1554.
11	罗承昆, 陈云翔, 张洋铭, 等. 基于混合参数证据网络的装备体系贡献率评估[J]. 兵工学报, 2018, 39(12): 2488-2496.
	Luo Chengkun, Chen Yunxiang, Zhang Yangming, et al. Evaluation of Equipment System Contribution Rate Based on Hybrid Parameter Evidence Network [J]. Acta Armamentarii, 2018, 39 (12): 2488-2496.
12	游雅倩, 姜江, 孙建彬, 等. 基于证据网络的装备体系贡献率评估方法研究[J]. 系统工程与电子技术, 2019, 41(8): 1780-1788.
	You Yaqian, Jiang Jiang, Sun Jianbin, et al. Research on Evaluation Method of Equipment System Contribution Rate Based on Evidence Network [J]. Systems Engineering and Electronics, 2019, 41 (8): 1780-1788.
13	Stemmer U. Locally Private K-means Clustering[C]// Proc. of the 2020 ACM-SIAM Symposium on Discrete Algorithms. Philadelphia: SIAM, 2020: 548-559.
14	Perez O J, Almanza O N, Adams L J, et al. Improving the Efficiency of the K-medoids Clustering Algorithm by Getting Initial Medoids[C]// World Conference on Information Systems and Technologies. Spinger, 2017: 125-132.
15	Huang Z X. Extensions to the K-means Algorithm for Clustering Large Data Sets with Categorical Values[J]. Data Mining and Knowledge Discovery (S1384-5810), 1998, 2(3): 283-304.
16	Madan S, Dana K J. Modified Balanced Iterative Reducing and Clustering Using Hierarchies (m-BIRCH) for Visual Clustering[J]. Pattern Analysis and Applications (S1433-7541), 2016, 19(4): 1023-1040.
17	Guha S, Rastogi R, Shim K, et al. CURE: An Efficient Clustering Algorithm for Large Databases[C]// Proc. of the 1998 ACM SIGMOD International Conference on Management of Data. New York: ACM, 1998: 73-84.
18	Guha S, Rastogi R, Shim K. ROCK: A Robust Clustering Algorithm for Categorical Attributes[J]. Information Systems (S0306-4379), 2000, 25(5): 345-366.
19	Ester M, Kriegel H P, Sander J, et al. A Density-based Algorithm for Discovering Clusters in Large Spatial Databases with Noise[C]// Proc. of the 2nd International Conference on Knowledge Discovery and Data Mining. Menlo Park: AAAI Press, 1996: 226-231.
20	Ankerst M, Breunig M M, Kriegel H P, et al. OPTICS: Ordering Points to Identify the Clustering Structure[C] // Proc. of the 1999 ACM SIGMOD International Conference on Management of Data. USA: ACM, 1999: 49-60.
21	Hinneburg A, Keim D A. An Efficient Approach to Clustering in Large Multimedia Databases with Noise[C]// Agrawal R. Proc. of the 4th International Conference on Knowledge Discovery and Data Mining. Menlo Park: AAAI Press, 1998: 58-65.
22	Wang W, Yang J, Muntz R. STING: A Statistical Information Grid Approach to Spatial Data Mining[C]// Jarke M. Proc. of the 23rd International Conference on Very Large Data Bases. San Francisco: Morgan Kaufmann, 1997: 186-195.
23	Agrawal R, Gehrke J, Gunopulos D, et al. Automatic Subspace Clustering of High Dimensional Data[J]. Data Mining and Knowledge Discovery (S1384-5810), 2005, 11(1): 5-33.
24	Yildirim A A, Watson D. A Comparative Study of the Parallel Wavelet-based Clustering Algorithm on Three-dimensional Dataset[J]. Journal of Supercomputing (S0920-8542), 2015, 71(7): 2365-2380.
25	Van Hulle Marc M. Self-organizing Maps[M]// Handbook of Natural Computing. Springer Berlin Heidelberg, 2012: 585-622.
26	王鹏, 李革, 张翔, 等. 一种基于模糊聚类的作战模拟训练效果评估方法[C]//第18届中国系统仿真技术及其应用学术年会论文集(18th CCSSTA 2017): 中国科学技术大学出版社, 2017: 212-216.
	Wang Peng, Li Ge, Zhang Xiang, et al. An Evaluation Method of Combat Simulation Training Effect Based on Fuzzy Clustering [C]// Proceedings of the 18th China Annual Conference on System Simulation Technology and Its Application (18th CCSSTA, 2017): China University of science and Technology Press, 2017: 212-216.
27	石崇林. 基于数据挖掘的兵棋推演数据分析方法研究[D]. 长沙: 国防科学技术大学, 2012.
	Shi Chonglin. Research on Data Analysis Method of Wargame Deduction Based on Data Mining [D]. Changsha: National University of Defense Technology, 2012.
28	Agrawal R, Srikant R. Fast Algorithm for Mining Association Rules in Large Database[C]// Proc. of the International Conference on Very Large Data Bases, Santiago, Chile, 1994: 487-499.
29	曹冠平, 王跃利, 张立韬. 关联规则挖掘在作战实验数据分析中的应用[J]. 指挥控制与仿真, 2019, 41(2): 70-74.
	Cao Guanping, Wang Yueli, Zhang Litao. Application of Association Rule Mining in Combat Experiment Data Analysis [J]. Command Control and Simulation, 2019, 41 (2): 70-74.
30	林圣琳. 面向复杂仿真的评估与优化方法研究[D]. 哈尔滨: 哈尔滨工业大学, 2020.
	Lin Shenglin. Research on Evaluation and Optimization Methods for Complex Simulation [D]. Harbin: Harbin Institute of Technology, 2020.
31	Han J W, Pei J, Yin Y W. Mining Frequent Patterns without Candidate Generation[J]. ACM SIGMOD Record (S0163-5808), 2000, 29(2): 1-12.
32	李政, 祝利, 韦伟. 关联规则挖掘在电子对抗目标分析中的应用[J]. 舰船电子对抗, 2015, 38(5): 1-4, 14.
	Li Zheng, Zhu Li, Wei Wei. Application of Association Rule Mining in Target Analysis of Electronic Countermeasure[J]. Ship Electronic Countermeasure, 2015, 38 (5): 1-4, 14.
33	Fusion Pearl J., Propagation, and Structuring in Belief Networks[J]. Artificial Intelligence (S0004-3702), 1986, 29(3): 241-288.
34	Liu H, Zhou S, Lam W, et al. A New Hybrid Method for Learning Bayesian Networks: Separation and Reunion[J]. Knowledge-Based Systems (S0950-7051), 2017, 121: 197-1855.
35	Kotsiantis S, Kanellopoulos D. Discretization Techniques: A Recent Survey[J]. GESTS International Transactions on Computer Science and Engineering (S1738-6438), 2006, 32(1): 47-58.
36	Maathuis M, Drton M, Lauritzen S, et al. Handbook of Graphical Models[M]. Boca Raton: CRC Press, 2018: 3-33.
37	舒健生, 姚群, 武健, 等. 基于贝叶斯网络的常规导弹反舰作战效能评估[J]. 火力与指挥控制, 2019, 44(1): 114-118, 124.
	Shu Jiansheng, Yao Qun, Wu Jian, et al. Anti Ship Operational Effectiveness Evaluation of Conventional Missile Based on Bayesian Network[J]. Fire Control & Command Control, 2019, 44 (1): 114-118, 124.
38	周兴旺, 从福仲, 庞世春. 基于BN-and-BP神经网络融合的陆空联合作战效能评估[J]. 火力与指挥控制, 2018, 43(4): 3-8.
	Zhou Xingwang, Cong Fuzhong, Pang Shichun. Effectiveness Evaluation of Land Air Joint Operation Based on BN and BP Neural Network Fusion[J]. Fire Control & Command Control, 2018, 43 (4): 3-8.
39	Hopfield J J. Neural Networks and Physical Systems with Emergent Collective Computational Abilities[J]. Proceedings of the National Academy of Sciences of the United States of America (S0027-8424), 1982, 79(8): 2554-2558.
40	Rumelhart D E, Mcclelland J L. Readings in Congnitive Science[M]. San Francisco: Margan Kaufmann, 1988: 399-421.
41	Kohonen T. Self-organization and Associative Memory[M]. Berlin Heidelberg: Springer-Verlag, 1989: 119-155.
42	Welling M, Rosen-zvi M, Hinton G. Exponential Family Harmoniums with An Application to Information Retrieval[M]. Cambridge: MA: MIT Press, 2005.
43	Hinton G E, Salakhutdinov R R. Reducing the Dimensionality of Data with Neural Networks[J]. Science (S0036-8075), 2006, 313(5786): 504-507.
44	刘海洋, 唐宇波, 胡晓峰, 等. 基于兵棋推演的联合作战方案评估框架研究[J]. 系统仿真学报, 2018, 30(11): 4115-4131.
	Liu Haiyang, Tang Yubo, Hu Xiaofeng, et al. Research on Evaluation Framework of Joint Operation Scheme Based on Wargame Deduction [J]. Journal of System Simulation, 2018, 30 (11): 4115-4131.
45	李妮, 李玉红, 龚光红, 等. 基于深度学习的体系作战效能智能评估及优化[J]. 系统仿真学报, 2020, 32(8): 1425-1435.
	Li Ni, Li Yuhong, Gong Guanghong, et al. Intelligent Evaluation and Optimization of System Combat Effectiveness Based on Deep Learning [J]. Journal of System Simulation, 2020, 32 (8): 1425-1435.
46	Cortes C, Vapnik V. Support-vector Networks[J]. Machine Learning (S0885-6125), 1995, 20(3): 273-297.
47	代耀宗, 沈建京, 郭晓峰, 等. 基于LS-SVM算法的实兵对抗演习作战效能评估[J]. 火力与指挥控制, 2019, 44(4): 17-21.
	Dai Yaozong, Shen Jianjing, Guo Xiaofeng, et al. Combat Effectiveness Evaluation of Actual Combat Exercise Based on LS-SVM Algorithm[J]. Fire Control & Command Control, 2019, 44 (4): 17-21.
48	郑潇. 体系效能仿真分析优化方法与工具研究[D]. 哈尔滨: 哈尔滨工业大学, 2016.
	Zheng Xiao. Research on Optimization Methods and Tools of System Effectiveness Simulation Analysis [D]. Harbin: Harbin Institute of Technology, 2016.
49	Ibrahim M, Louie M, Modarres C. Global Explanations of Neural Networks: Mapping the Landscape of Predictions[C]//Proc. of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 2019: 279-287.

聚类方法	计算复杂度	大规模数据集	高维数据集
K-means	O(NKd)(时间)，O(N+K)(空间)	是	否
BIRCH	O(N²log N)(时间)	否	否
CURE	O(N²log N)(时间)，O(N)(空间)	是	是
DBSCAN	O(Nlog N)(时间)	是	否
OPTICS	O(Elog N)(时间)	是	否
DENCLUE	O(Nlog N)(时间)	是	是
Wave-Cluster	O(N)(时间)	是	否

[1]	Lin Zhang, Kunyu Wang, Yuanjun Laili, Lei Ren. Modeling & Simulation based System of Systems Engineering [J]. Journal of System Simulation, 2022, 34(2): 179-190.
[2]	Li Ni, Li Yuhong, Gong Guanghong, Huang Xiaodong. Intelligent Effectiveness Evaluation and Optimization on Weapon System of Systems Based on Deep Learning [J]. Journal of System Simulation, 2020, 32(8): 1425-1435.
[3]	Zhu Tao, Liang Weitai, Huang Songhua, Zhang Jieyong. Research on Modeling and Analyzing Method of Task-oriented Network Information System of Systems [J]. Journal of System Simulation, 2020, 32(4): 727-737.
[4]	Yin Xiaojing, Hu Xiaofeng, Rong Ming, Liu Haiyang, Yang Yongli. Review of Evaluation Methods of Contribution Rate to System of Systems [J]. Journal of System Simulation, 2019, 31(6): 1027-1038.
[5]	Yang Yongli, Hu Xiaofeng, Rong Ming, Yin Xiaojing, Wang Wenxiang. Characteristic Index Digging of Combat SoS Capability Based on Machine Learning [J]. Journal of System Simulation, 2019, 31(6): 1048-1054.
[6]	Rong Ming, Hu Xiaofeng, Yang Jingyu. Resilience Analysis and Evaluation of Combat System of Systems Architecture Based on Dynamic Super Network [J]. Journal of System Simulation, 2019, 31(6): 1055-1061.
[7]	Si Guangya, Wang Fei, Liu Yang. System Analysis Method Based on Simulation Big Data [J]. Journal of System Simulation, 2019, 31(3): 511-519.
[8]	Hu Jianpeng, Huang Linpeng. Modeling and Simulation of Executable Architecture Based on P-DEVS [J]. Journal of System Simulation, 2016, 28(2): 283-291.
[9]	Ma Li, Zhang Mingzhi. Research Progress on War Complex System of Systems Modeling Based on Complex Network [J]. Journal of System Simulation, 2015, 27(2): 217-225.
[10]	Cao Jianjun, Ma Haizhou, Jiang Delong. Research on Modeling of Weapons System of Systems Evaluation [J]. Journal of System Simulation, 2015, 27(1): 37-42.