Effectiveness Evaluation Method of Space-based Information Systems Based on SEM

doi:10.16182/j.issn1004731x.joss.21-0361

Abstract

Abstract:

Aiming at the nonlinear emergence of effectiveness of the coupling and interaction of space-based information system (SIS), an operational effectiveness evaluation (OEE) model based on structural equation modeling (SEM) is proposed. Through the analysis of capacity requirements and system composition, the internal connections of sub-systems are obtained, and the effectiveness evaluation index system is constructed. By considering the synergistic interaction within SIS, the linear and nonlinear SEMs are constructed respectively to evaluate the effectiveness, and the analytical model of OEE is established on the basis of the corresponding parameter equations. With the background of integrated joint operations under the support of space-based information in the maritime direction, the effectiveness data is obtained through the joint simulation of MATLAB and STK. LISREL is used in the parameter estimation, and the evaluation results under linear and nonlinear models are obtained, which verify the validity of SEM model. Compared with the linear SEM, the nonlinear model is more accurate to the actual combat effectiveness.

Key words: structural equation modeling, operational effectiveness, space-based information system, evaluation, non-linear

CLC Number:

TP391

Chi Han, Wei Xiong, Wenwen Liu, Xiaolan Yu, Ping Jian. Effectiveness Evaluation Method of Space-based Information Systems Based on SEM[J]. Journal of System Simulation, 2022, 34(8): 1799-1810.

Figures/Tables 18

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Explicit and latent variables division of SEM

变量	潜变量	显变量
外生变量	信息获取能力 $ξ 1$	$目标发现概率$ $x 1$
		$目标跟踪概率$ $x 2$
		$目标识别概率$ $x 3$
	指挥控制能力 $ξ 2$	决策响应时间 $x 4$
	指挥控制能力 $ξ 2$	传输时延 $x 5$
	生存保障能力 $ξ 3$	系统生存能力 $x 6$
	生存保障能力 $ξ 3$	系统修复能力 $x 7$
内生变量	火力打击能力 $η$	战斗结果 $y 1$
		战斗损伤率 $y 2$
		任务完成时间 $y 3$

Table 1

Fig. 5

Fig. 6

Table 2

Evaluation standard of fitting index

统计指标	评价标准
$χ 2$	越小越好
RMSEA	$< 0.1, 越小越好$
NFI	$> 0.9, 越接近 1 越好$
CFI	$> 0.9, 越接近 1 越好$
GFI	$> 0.9, 越接近 1 越好$

Table 2

Table 3

Table 4

Fig. 7

Table 5

Operational effectiveness sample data

指标	方案
指标	1	2	3	4	…	24
$x 1$	0.594	0.836	0.721	0.415		0.512
$x 2$	0.380	0.982	0.953	0.970		0.896
$x 3$	0.629	0.119	0.541	0.328		0.635
$x 4$	0.770	0.940	0.705	0.484		0.329
$x 5$	0.391	0.102	0.037	0.517		0.265
$x 6$	0.317	0.157	0.246	0.434		0.116
$x 7$	0.663	0.459	0.374	0.346		0.494
$y 1$	0.581	0.633	0.467	0.484		0.353
$y 2$	0.703	0.680	0.686	0.716		0.445
$y 3$	0.651	0.621	0.720	0.710		0.824

Table 5

Fig. 8

Table 6

Estimated values of linear SEM

参数	数值	参数	数值	参数	数值
$λ 11 x$	0.76	$λ 3 y$	0.68	$δ 6$	0.07
$λ 21 x$	0.71	$γ 1$	0.85	$δ 7$	0.18
$λ 31 x$	0.75	$γ 2$	0.88	$ε 1$	0.24
$λ 42 x$	0.74	$γ 3$	0.86	$ε 2$	0.16
$λ 52 x$	0.88	$δ 1$	0.14	$ε 3$	0.20
$λ 63 x$	0.85	$δ 2$	0.16	$ζ$	0.10
$λ 73 x$	0.73	$δ 3$	0.20	$ϕ 12$	0.80
$λ 1 y$	0.80	$δ 4$	0.18	$ϕ 13$	0.72
$λ 2 y$	0.82	$δ 5$	0.15	$ϕ 23$	0.85

Table 6

Table 7

Partial fit index of linear SEM

$χ 2$	RMSEA	NFI	CFI	GFI
42.42	0.03	0.98	1.00	0.99

Table 7

Fig. 9

Table 8

Partial fit index of nonlinear SEM

$χ 2$	RMSEA	NFI	CFI	GFI
54.16	0.06	0.93	0.92	0.97

Table 8

Table 9

Partial fit index of nonlinear SEM

指标	模式	方案
指标	模式	1	7	12	17	23
$ξ 1$	线性	0.92	0.65	0.83	0.48	0.71
$ξ 1$	非线性	0.92	0.65	0.83	0.48	0.71
$ξ 2$	线性	0.79	0.86	0.88	0.07	0.82
$ξ 2$	非线性	0.79	0.86	0.88	0.07	0.82
$ξ 3$	线性	0.44	1	0.76	0.62	0.57
$ξ 3$	非线性	0.44	1	0.76	0.62	0.57
$ξ 12$	线性	-	-	-	-	-
$ξ 12$	非线性	0.86	0.49	0.71	0.29	0.85
$ξ 22$	线性	-	-	-	-	-
$ξ 22$	非线性	-0.66	0.83	0.76	0.03	0.66
$ξ 32$	线性	-	-	-	-	-
$ξ 32$	非线性	0.55	0.74	0.58	0.38	0.66
$ξ 1 ξ 2$	线性	-	-	-	-	-
$ξ 1 ξ 2$	非线性	0.68	-0.60	0.71	0.05	-0.68
$ξ 1 ξ 3$	线性	-	-	-	-	-
$ξ 1 ξ 3$	非线性	-0.44	0.67	0.84	0.33	0.69
$ξ 2 ξ 3$	线性	-	-	-	-	-
$ξ 2 ξ 3$	非线性	0.22	0.81	0.93	0.04	0.42
$η$	线性	0.45	0.83	0.85	0.14	0.58
$η$	非线性	0.69	0.72	0.90	0.40	0.78

Table 9

References 21

1	李德仁, 沈欣. 我国天基信息实时智能服务系统发展战略研究[J]. 中国工程科学, 2020, 22(2): 138-143.
	Li Deren, Shen Xin. Research on the Development Strategy of Real-Time and Intelligent Space-Based Information Service System in China[J]. Strategic Study of CAE, 2020, 22(2): 138-143.
2	齐小刚, 刘学星, 张博孜, 等. 基于结构方程模型的武器装备体系涌现性度量模型[J]. 兵工学报, 2020, 41(2): 406-416.
	Qi Xiaogang, Liu Xuexing, Zhang Bozi, et al. Emergence Measurement Model of Armament Systems Based on SEM[J]. Acta Armamentarii, 2020, 41(2): 406-416.
3	程恺, 张宏军, 柳亚婷, 等.作战效能及其评估方法研究综述[J]. 系统科学学报, 2014, 22(1): 88-92.
	Cheng Kai, Zhang Hongjun, Liu Yating, et al. Review on Operational Effectiveness and Evaluation Method[J]. Journal of System Science, 2014, 22(1): 88-92.
4	秦茂森, 赵丹玲, 杨克巍. 基于作战网络的反潜活动效能评估[J]. 系统工程与电子技术, 2018, 40(7): 1513-1520.
	Qin Maosen, Zhao Danling, Yang Kewei. Effectiveness Evaluation of Anti-Submarine Activities Based on Combat Network[J]. System Engineering and Electronics, 2018, 40(7): 1513-1520.
5	罗小明, 朱延雷, 何榕. 基于复杂适应系统的装备作战试验体系贡献度评估[J]. 装甲兵工程学院学报, 2015, 29(2): 1-6.
	Luo Xiaoming, Zhu Yanlei, He Rong. Evaluation of Contribution for Equipment Operational Test System Based on Complex Adaptive System[J]. Journal of Academy of Armored Force Engineering, 2015, 29(2): 1-6.
6	罗承昆, 陈云翔, 王莉莉, 等. 基于作战环和改进信息熵的体系效能评估方法[J]. 系统工程与电子技术, 2019, 41(1): 73-80.
	Luo Chengkun, Chen Yunxiang, Wang Lili, et al. Effectiveness Evaluation Method of System-of-Systems Based on Operational Loop and Improved Information Entropy[J]. System Engineering and Electronics, 2019, 41(1): 73-81.
7	李妮, 李玉红, 龚光红, 等. 基于深度学习的体系作战效能智能评估及优化[J]. 系统仿真学报, 2020, 32(8): 1425-1435.
	Li Ni, Li Yuhong, Gong Guanghong, et al. Intelligent Effectiveness Evaluation and Optimization on Weapon System of Systems Based on Deep Learning[J]. Journal of System Simulation, 2020, 32(8): 1425-1435.
8	魏海龙, 李清, 黄诗晟, 等. 模型驱动的武器装备系统效能评估方法[J]. 清华大学学报(自然科学版), 2019, 59(11): 925-933.
	Wei Hailong, Li Qing, Huang Shisheng, et al. Model-Driven MOE Method for Weapon Systems[J]. Journal of Tsinghua University (Science and Technology), 2019, 59(11): 925-933.
9	陆营波, 钱晓超, 陈伟, 等. 数据驱动的装备效能评估模型构建方法研究[J]. 系统仿真学报, 2018, 30(12): 4587-4594, 4601.
	Lu Yingbo, Qian Xiaochao, Chen Wei, et al. Research on Construction Method of Data-Driven Equipment Effectiveness Evaluation Model[J]. Journal of System Simulation, 2018, 30(12): 4587-4594, 4601.
10	梁家林, 熊伟. 基于作战环的武器装备体系能力评估方法[J]. 系统工程与电子技术, 2019, 41(8): 1810-1819.
	Liang Jialin, Xiong Wei. Capabilities Assessment of the Weaponry System Based on Combat Ring [J]. System Engineering and Electronics, 2019, 41(8): 1810-1819.
11	毛腾蛟, 张东戈, 梁雪峰, 等. 作战系统运用效能评估的线性系统响应分析法[J]. 系统工程与电子技术, 2019, 41(12): 2772-2779.
	Mao Tengjiao, Zhang Dongge, Liang Xuefeng, et al. Linear System Response Analysis Method for Operational Effectiveness Evaluation of Combat System[J]. System Engineering and Electronics, 2019, 41(12): 2722-2779.
12	Jöreskog Karl G. A General Method for Estimating a Linear Structural Equation System[J]. Psychometrika(S0033-3123), 1970(2): 1-41.
13	张少卿. 基于非线性SEM的航空反潜装备作战能力评估方法[D]. 长沙: 国防科技大学, 2011.
	Zhang Shaoqing. Combat Capability Evaluation Method of Aviation Antisubmarine Equipment Based on Nonlinear SEM[D]. Changsha: National University of Defense Technology, 2011.
14	王静. 基于可视化仿真的智能雷场效能评估技术研究[D]. 西安: 西安电子科技大学, 2019.
	Wang Jing. Research on Effectiveness Evaluation Technology of Intelligent Minefield Based on Visual Simulation[D]. Xi'an: Xidian University, 2019.
15	卞玉梅. 结构方程模型研究及其应用[D]. 大连: 大连海事大学, 2017.
	Bian Yumei. Research and Application of Structural Equation Model[D]. Dalian: Dalian Maritime University, 2017.
16	侯杰泰, 温忠麟, 成子娟. 结构方程模型及其应用[M]. 北京: 教育科学出版社, 2004: 14-17.
	Hou Jietai, Wen Zhonglin, Cheng Zijuan. Structural Equation Model and Its Application[M]. Beijing: Educational Science Publishing House, 2004: 14-17.
17	杨娟, 朱延雷, 何榕. 面向装备体系作战效能评估的SEM建模与评估方法[J]. 指挥控制与仿真, 2016, 38(2): 33-36.
	Yang Juan, Zhu Yanlei, He Rong. SEM Model and Evaluation Method for Operational Effectiveness Assessment of Equipment System[J]. Command Control & Simulation, 2016, 38(2): 33-36.
18	李宏辕. 面向反导作战的航天信息系统建模与效能评估[D]. 哈尔滨: 哈尔滨工业大学, 2017.
	Li Hongyuan. Modeling and Evaluation of Space Information Systems for Antimissile Operations[D]. Harbin: Harbin Institute of Technology, 2017.
19	韩驰, 熊伟. 航天侦察装备体系指标关联信息挖掘研究[J].系统仿真学报, 2021, 33(10): 2372-2380. doi: 10.16182/j.issn1004731x.joss.20-0553 .
	Han Chi, Xiong Wei. Research on Assocoation Information Mining of Space Reconnaissance Equipment System Index [J]. Journal of System Simulation, 2021, 33(10): 2372-2380. doi:10.16182/j.issn1004731x.joss.20-0553 .
20	殷小静, 胡晓峰, 荣明, 等. 体系贡献率评估方法研究综述与展望[J]. 系统仿真学报, 2019, 31(6): 1027-1038.
	Yin Xiaojing, Hu Xiaofeng, Rong Ming, et al. Review of Evaluation Methods of Contribution Rate to System of Systems[J]. Journal of System Simulation, 2019, 31(6): 1027-1038.
21	温忠麟, 侯杰泰, 马什赫伯特. 结构方程模型检验:拟合指数与卡方准则[J]. 心理学报, 2004, 36(2): 186-194.
	Wen Zhonglin, Hou Jietai, Marsh Herbert W. Structural Equation Model Testing: Cutoff Criteria for Goodness of Fit Indices and Chi-Square Tset[J]. Acta Psychologica Sinica, 2004, 36(2): 186-194.

方案要素	要素组成方式	编号
卫星平台类型及数量	2颗光学成像，4颗SAR，3颗电子侦察，3颗通信中继	A
	3颗光学成像，2颗SAR，4颗电子侦察，2颗通信中继	B
	4颗光学成像，3颗SAR，2颗电子侦察，1颗通信中继	C
传感器参数	光学成像(25.0/25.0/0.1)，SAR(35.0/25.0/0.1)，电子侦察(3 000)	D
	光学成像(30.0/25.0/0.1)，SAR(30.0/25.0/0.1)，电子侦察(3 000)	E
	光学成像(5.0/30.0/0.1)，SAR(30.0/25.0/0.1)，电子侦察(2 000)	F
	光学成像(25.0/25.0/0.1)，SAR(20.0/25.0/0.1)，电子侦察(2 000)	G
攻击武器类型及数量	2枚I型反舰导弹	H
攻击武器类型及数量	2枚II型反舰导弹	I

[1]	Sixian Zhang, Yi Yang, Meng Zhang, Pengbo Mi. An Efficient Tracker via Multi-feature Adaptive Correlation Filter [J]. Journal of System Simulation, 2022, 34(8): 1864-1873.
[2]	Muqing Wang, Lei Zhang, Xiumin Fan, Xiaomeng Luo, Wenmin Zhu. Simulation Method of Virtual Human Pose Optimization Based On VR Peripherals [J]. Journal of System Simulation, 2022, 34(6): 1296-1303.
[3]	Feng Wu, Xiuluo Liu, Jia Wang, Yang Liu, Sujiang Li, Yan Zhong. Research on Space Launch Visualization Simulation Analysis Technology and Application [J]. Journal of System Simulation, 2022, 34(4): 856-869.
[4]	Ziwei Zhang, Liang Li, Zhiming Dong, Yifei Wang, Li Duan. Research on the Construction Method of Simulation Evaluation Index of Operation Effectiveness Operation Concept Traction [J]. Journal of System Simulation, 2022, 34(3): 603-613.
[5]	Gaofeng Zhang, Liang Wu. Effectiveness Evaluation of Surface Ship Air Defense and Antimissile Combat in Complex Electromagnetic Environment [J]. Journal of System Simulation, 2022, 34(3): 640-650.
[6]	Ziwei Zhang, Qisheng Guo, Zhiming Dong, Ang Gao, Yifei Wang. Review of System of Systems Combat Effectiveness Evaluation and Optimization Methods [J]. Journal of System Simulation, 2022, 34(2): 303-313.
[7]	Luo Zhangzhen, Jiang Haifan, Fu Jianlin, Ding Guofu. Combination Weighting-based Comprehensive Evaluation for Discrete Workshop Production Plan [J]. Journal of System Simulation, 2021, 33(8): 1856-1865.
[8]	Huang Xiaodong, Xie Kongshu, Li Ni, Yan Xuefeng, Zhao Yali. Research and Application of Simulation Support Platform for System-of-Systems Combat [J]. Journal of System Simulation, 2021, 33(8): 1914-1926.
[9]	Luo Xueping, Cao Yunhe, Hu Qi, Chen Shutao, Yan Shijie, Cai Xi. SAR Imaging Seeker Interference Modeling & Evaluation and Simulation System Design [J]. Journal of System Simulation, 2021, 33(8): 1927-1937.
[10]	Liu Feng, Shao Chunfu, Jia Hongfei, Dong Chunjiao. Simulation Evaluation for Service Facilities Distribution and Flow Lines Optimization on Urban Rail Transfer Station [J]. Journal of System Simulation, 2021, 33(7): 1670-1681.
[11]	Wang Yecheng, Ma Jing, Jin Congjun, Jie Li, Shi Bin. Evaluation Method on Multi-stage Bayes Information Fusion for Scatters HWIL Simulation [J]. Journal of System Simulation, 2021, 33(7): 1682-1688.
[12]	Li Xinli, Zou Changming, Yang Guotian, Liu He. Research of Super-resolution Processing of Invoice Image Based on Generative Adversarial Network [J]. Journal of System Simulation, 2021, 33(6): 1307-1314.
[13]	Guo Qingjun, Hao Qianwen, Wang Yijie, Wang Jing. Subway System Resilience Evaluation in Based on ANP-Extension Cloud Model [J]. Journal of System Simulation, 2021, 33(4): 943-950.
[14]	Guo Ai, Chen Chao, Shi Junjie, Liu Zhengjie, Chen Weirong, Liang Jiayi, Liu Nan. Comparative Study of Energy Management Strategy Based on Value Loss of Tram Hybrid Power System [J]. Journal of System Simulation, 2021, 33(3): 572-580.
[15]	Wang Yinsong, Wang Kai. Performance Evaluation Method for Load Control System Considering “Two Detailed Rules” [J]. Journal of System Simulation, 2021, 33(3): 710-720.