Evolutionary Simulation of Online Public Opinion Based on the BERT-LDA Model under COVID-19

doi:10.16182/j.issn1004731x.joss.20-0690

Abstract

Abstract: The construction of a large-scale online public opinion evolution simulation model has guidance value for differentiated emergency management and public opinion guidance in the worst-hit areas in Wuhan and the other areas in China during the outbreak of the COVID-19. In order to realize the fine-grained simulation of the public sentiment evolution of the topic, the LDA topic model is deeply integrated with BERT word vector to optimize the topic vector and power the text topic clustering. At the same time, on the basis of improving BERT pre-training task, the deep pre-training task is superimposed to improve the accuracy of the model in emotion classification. The results show that the NPMI value of the improved BERT-LDA model is 0.357 higher than that of the original LDA model during the topic vector training. In terms of the emotional classification task of epidemic events, the AUC value exceeds 99.6%, which proves that the improved BERT-LDA model can be effectively applied to large-scale internet public opinion evolution simulation.

Key words: corona virus disease 2019 (COVID-19), BERT-LDA model, evolution simulation of public opinion, difference comparison

CLC Number:

TP391.9

Zhuang Muni, Li Yong, Tan Xu, Mao Taitian, Lan Kaicheng, Xing Lining. Evolutionary Simulation of Online Public Opinion Based on the BERT-LDA Model under COVID-19[J]. Journal of System Simulation, 2021, 33(1): 24-36.

References

[1] 吴世文. 重大突发公共卫生事件中的伪信息传播、治理困境及其突破路径——以新冠肺炎疫情为例[J].电子政务, 2020(9): 40-50.
Wu Shiwen.Novel Coronavirus Pneumonia, the Spread of False Information, and the Way to Solve the Problem: Taking the New Crown Pneumonia Epidemic as an Example[J]. E-Government, 2020(9): 40-50.
[2] 齐佳音, 方滨兴. 重大突发事件中网络舆情引导及治理研究—以新型冠状病毒肺炎疫情为例[J]. 上海对外经贸大学学报, 2020, 27(3): 5-13.
Qi Jiayin, Fang Binxing.Network Public Opinion Response and Governance Innovation in Serious Emergencies: Take the COVID-19 Epidemic as an Example[J]. Journal of Shanghai University of International Business and Economics, 2020, 27(3): 5-13.
[3] 曹武军, 陈秦秀, 薛朝改. 重大疫情网络舆情防控策略研究[J]. 情报杂志, 2020, 39(10): 107-114.
Cao Wujun, Chen Qinxiu, Xue Chaogai.Research on Online Public Opinion Prevention and Control Strategies for Major Epidemic Diseases[J]. Journal of Intelligence, 2020, 39(10): 107-114.
[4] 王文, 王树锋, 李洪华. 基于文本语义和表情倾向的微博情感分析方法[J]. 南京理工大学学报, 2014, 38(6): 733-738,749.
Wang Wen, Wang Shufeng, Li Honghua.Microblogging Sentiment Analysis Method Based on Text Semantics and Expression Tendentiousness[J]. Journal of Nanjing University of Science and Technology, 2014, 38(6): 733-738,749.
[5] Bose R, Dey R K, Roy S, et al.Analyzing Political Sentiment Using Twitter Data[C]. Information and Communication Technology for Intelligent Systems. Singapore: Springer, 2019: 427-436.
[6] Pang B, Lee L, Vaithyanathan S. Thumbs up? Sentiment Classification Using Machine Learning Techniques [J/OL]. EMNLP, 2002, 10: 79-86[2020-05-21]. https://arxiv.org/abs/ cs/0205070.
[7] Zhang H. The Optimality of Naive Bayes [J/OL]. American Association for Artificial Intelligence, 2004. [2020-05-21]. https://www.aaai.org.
[8] Purnamasari N M G D, Fauzi M A, Indriati L S D. Cyberbullying Identification in Twitter Using Support Vector Machine and Information Gain Based Feature Selection[J]. Indonesian Journal of Electrical Engineering and Computer Science (S2502-4752), 2020, 18(3): 1494-1500.
[9] Sherstinsky A.Fundamentals of Recurrent Neural Network (RNN) and Long Short-term Memory (LSTM) Network[J]. Physica D: Nonlinear Phenomena (S0167-2789), 2020, 404: 132306.
[10] Vaswani A, Shazeer N, Parmar N, et al.Attention is All You Need[C]. Advances in Neural Information Processing Systems, 2017: 5998-6008.
[11] Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[C/OL]. NAACL-HLT (1). 2019 [2020-05-21]. https://arxiv.org/abs/1810.04805.
[12] 杨晨, 宋晓宁, 宋威. SentiBERT: 结合情感信息的预训练语言模型[J]. 计算机科学与探索, 2020, 14(9): 1563-1570.
Yang Chen, Song Xiaoning, Song Wei.SentiBERT: A Pretraining Language Model Combining Sentiment Information[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(9): 1563-1570.
[13] Sun C, Qiu X, Xu Y, et al. How to Fine-Tune BERT for Text Classification?[J/OL]. Computation and Language, 2019, 11856: 194-206[2020-05-21]. https://doi.org/10. 1007/978-3-030-32381-3_16.
[14] Blei D M, Ng A Y, Jordan M I.Latent Dirichlet Allocation[J]. Journal of Machine Learning Research (S1532-4435), 2003, 3: 993-1022.
[15] He K, Zhang X, Ren S, et al.Deep Residual Learning for Image Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE Xplore, 2016: 770-778.
[16] Kingma D P, Ba J. Adam: A Method for Stochastic Optimization[J]. Learning, 2014 [2020-05-21]. https://arxiv. org/abs/1412.6980.
[17] Wang G, Wong K W, Lu J. AUC-Based Extreme Learning Machines for Supervised and Semi-Supervised Imbalanced Classification[J]. IEEE Transactions on Systems, Man,Cybernetics: Systems (S2168-2216), 2020: 1-12[2020-05-21]. https://ieeexplore.ieee.org/ abstract/document/9063675.
[18] Lalmas M C J V. Information Retrieval: Uncertainty and Logics: Advanced Models for the Representation and Retrieval of Information[M]. Boston: Kluwer Academic Publishers, 1998.
[19] Zhang Z, Sabuncu M. Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels[C/OL]. Advances in Neural Information Processing Systems.2018: 8778-8788 [2020-05-21]. https://papers.nips.cc/paper/2018/hash/f2925f97bc13ad2852a7a551802feea0-Abstract.html.
[20] Kumar N, Deepak G, Santhanavijayan A.A Novel Semantic Approach for Intelligent Response Generation using Emotion Detection Incorporating NPMI Measure[J]. Procedia Computer Science (S1877-0509), 2020, 167: 571-579.
[21] 郭业才, 张浩然. 基于改进LDA和自编码器的调制识别算法[J/OL].系统仿真学报: 1-6 [2020-05-21]. http://kns.cnki.net/kcms/detail/11.3092.V.20200102.1527. 008.html.
Guo Yecai, Zhang Haoran. Modulation Recognition Algorithm Based on Improved LDA and Autoencoders [J/OL]. Journal of System Simulation: 1-6 [2020-05-21]. http://kns.cnki.net/kcms/detail/11.3092.V.20200102.1527. 008.html.
[22] Chen X, Xu L, Liu Z, et al.Joint Learning of Character and Word Embeddings[C]. International Conference on Artificial Intelligence. Argentina: AAAI Press, 2015: 1236-1242.
[23] 郭景萍. 社会公共安全视野下的情感安全调控[J]. 湖南师范大学社会科学学报, 2009, 38(2): 87-90.
Guo Jingping.Emotional Security Control on the Perspective of Social Public Safety[J]. Journal of Social Science of Hunan Normal University, 2009, 38(2): 87-90.