基于随机森林和卷积特征学习的人机交互语音情感识别

doi:10.16182/j.issn1004731x.joss.20-FZ0494E

系统仿真学报 ›› 2020, Vol. 32 ›› Issue (12): 2388-2400.doi: 10.16182/j.issn1004731x.joss.20-FZ0494E

基于随机森林和卷积特征学习的人机交互语音情感识别

王静¹, 刘洪岩², 刘芳芳¹, 王青青^3*

1.长春理工大学光电信息学院,吉林长春 130000;
2.中国移动通信集团吉林有限公司网络管理中心,吉林长春 130012;
3.吉林动画学院,吉林长春 130012

收稿日期:2020-03-28 修回日期:2020-03-28 出版日期:2020-12-18 发布日期:2020-12-16

Human-computer Interaction Speech Emotion Recognition Based on Random Forest and Convolution Feature Learning

Wang Jing¹, Liu Hongyan², Liu Fangfang¹, Wang Qingqing^3*

1. College Of Optical And Electronical Information Changchun University Of Science And Technology,Changchun 130000,China;
2. Network management center of China Mobile Communication Group Jilin Co.,Ltd,Changchun 130012,China;
3. Jilin Animation Institute,Changchun 130012,China

Received:2020-03-28 Revised:2020-03-28 Online:2020-12-18 Published:2020-12-16

摘要/Abstract

摘要： 针对情绪机器人的自动语音情感识别在不同类型人群之间的语音特征差异,提出了一种用于语音情感识别的随机森林,结合卷积特征学习对情绪化社交机器人系统进行了初步的仿真实验,结果表明情绪机器人能够实时跟踪兴奋、愤怒、哀伤、高兴、惊讶、恐惧、中性7种基本情绪。通过采用非个性化的语音情感特征,补充了原始的个性化语音情感特征,实现了对通用性情感和特殊性情感的提取,对于情感机器人来说,利用这些指标在模拟实验和应用实验中都具有一定的应用前景。

关键词: 情绪机器人, 自动语音情感识别, 随机森林, 卷积特征学习

Abstract: Focus on the different speech features of different types of people in the automatic speech emotion recognition of emotional robots,a random forest for speech emotion recognition is proposed,and a preliminary simulation experiment of emotional social robot system based on convolution feature learning is carried out.The results show that the emotional robot can track in real time,the seven basic emotions of excitement,anger,sadness,happiness,surprise,fear and neutrality.By using non personalized speech emotion features,the original personalized speech emotion features are supplemented,and the general emotion and special emotion are extracted.For emotional robot,using these indicators has a certain application prospect in the simulation experiment and application experiment.

Key words: emotional robot, automatic speech emotion recognition, random forest, convolution feature learning

中图分类号:

TP391.9

王静, 刘洪岩, 刘芳芳, 王青青. 基于随机森林和卷积特征学习的人机交互语音情感识别[J]. 系统仿真学报, 2020, 32(12): 2388-2400.

Wang Jing, Liu Hongyan, Liu Fangfang, Wang Qingqing. Human-computer Interaction Speech Emotion Recognition Based on Random Forest and Convolution Feature Learning[J]. Journal of System Simulation, 2020, 32(12): 2388-2400.

参考文献 35

[1]	Flavia Marino.Outcomes of a Robot-Assisted Social-Emotional Understanding Intervention for Young Children with Autism Spectrum Disorders[J].Journal of Autism and Developmental Disorders (S0162-3257),2020,50(7):1973-1987.
[2]	Pejman Sajjadi,Laura Hoffmann.A Personality-Based Emotional Model for Embodied Conversational Agents:Effects on Perceived Social Presence and Game Experience of Users[J].Entertainment Computing (S2875-9521),2019,32:100313.
[3]	Robotics Androids.Differences in Spontaneous Interactions of Autistic Children in an Interaction with an Adult and Humanoid Robot[J].Frontiers in robotics and AI (S1945-8371),2020,7:28.
[4]	Man Hao,Wei-Hua Cao.Visual-Audio Emotion Recognition Based on Multi-Task and Ensemble Learning with Multiple Features[J].Neurocomputing (S0925-2312),2020,391:42-51.
[5]	Rao Y,Wu L W,Wang Y M,et al.Research Progress of Affective Computing Technology Based on Semantic Analysis[J].Journal of Software (S1000-9825),2018,29(8):2397-2426.
[6]	Yang M J,Zhang J B.Speech Emotion Recognition Based on Deep Neural Network[J].Science and Technology and Engineering (S1671-1815),2019,19(8):127-131.
[7]	Zhang L,LüJ,Qiang Y,et al.Speech Emotion Recognition Based on Deep Belief Network[J].Journal of Taiyuan University of technology (S1007-9432),2019,50(1):101-107.
[8]	Akçay M B,Oğuz K.Speech Emotion Recognition:Emotional Models,Databases,Features,Preprocessing Methods,Supporting Modalities,and Classifiers[J].Speech Communication (S0167-6393),2020,116:56-76.
[9]	Tang D,Qin B,Liu T.Deep Learning for Sentiment Analysis:Successful Approaches and Future Challenges[J].Wiley Interdisciplinary Reviews:Data Mining and Knowledge Discovery (S1942-4787),2015,5(6):292-303.
[10]	Lu G M,Yuan L,Yang W J,et al.Speech Emotion Recognition Based on Long-Term Memory and Convolution Neural Network[J].Journal of Nanjing University of Posts and Telecommunications (Natural Science Edition)(S1673-5439),2018,38(5):63-69.
[11]	Sun L H,Fu S,Wang F.Decision Tree SVM Model with Fisher Feature Selection for Speech Emotion Recognition[J].EURASIP Journal on Audio,Speech,and Music Processing (S1687-4714),2019(1):1-14.
[12]	Li H H.Research on Speech Emotion Recognition Based on Feature Selection and Optimization[D].Xi'an:Xi'an University of Electronic Science and Technology,2019.
[13]	Koduru A,Valiveti H B,Budati A K.Feature Extraction Algorithms to Improve the Speech Emotion Recognition Rate[J].International Journal of Speech Technology (S1381-2416),2020,23(1):45-55.
[14]	Jiang P X,Fu H L,Tao H W,et al.A Speech Emotion Recognition Method Based on Convolutional Neural Network Feature Representation[J].Electronic Devices (S1005-9490),2019,42(4):998-1001.
[15]	Miao Y Q,Zou W,Liu T L,et al.Speech Emotion Recognition Based on Parameter Migration and Convolution Recurrent Neural Network[J].Computer Engineering and Application (S1002-8331),2019,55(10):135-140,198.
[16]	Zeng R H,Zhang S Q.Speech Emotion Recognition Method Based on Improved Convolutional Neural Network[J].Journal of Applied Sciences (S0255-8297),2018,36(5):837-844.
[17]	Tian L L,Zou J Z,Zhang J,et al.Emotion Recognition of EEG Signals Based on Improved Convolutional Neural Network[J].Computer Engineering and Application (S1002-8331),2019,55(22):99-105.
[18]	G Viswanatha Reddy,C V R Dharma Savarni,Snehasis Mukherjee.Facial Expression Recognition in the Wild,by Fusion of Deep Learnt and Hand-Crafted Features[J].Cognitive Systems Research (S1389-0417),2020,62:23-34.
[19]	Hou P W,Wang Y J.Continuous Emotion Recognition and Tracking Algorithm Based on Random Forest[J].Science and Technology and Engineering (S1671-1815),2018,18(22):77-83.
[20]	Li G L,Tie Y,Qi L.Multi Feature Speech Emotion Recognition Based on Stochastic Forest Classification Optimization[J].Microelectronics and Computer (S1000-7180),2019,36(1):70-73.
[21]	Chen P,Zhang J H,Wen Z Z,et al.EEG Emotion Recognition Based on Nuclear Spectrum Regression and Random Forest[J].Journal of East China University of science and Technology (Natural Science Edition)(S1006-3080),2018,44(5):744-751.
[22]	Zhong Q,Feng Y Q,Wang W.A Comparative Study of Speech Emotion Recognition Based on Cross Language Corpus[J].Journal of Nanjing University (Natural Science)(S0469-5097),2019,55(5):765-773.
[23]	Liu Chang,Zhang Yike,Zhang Pengyuan,et al.Neural Network Language Model Based on Improved Topic Distribution Characteristics[J].Acta electronics and information (S1009-5896),2018,40(1):219-225.
[24]	Jiang Xiaoqing,Tian Lan,Cui Guohui.Prosodic Feature Analysis and Emotion Recognition of Multilingual Emotional Speech[J].Acta acoustics Sinica (S0371-0025),2006(3):217-221.
[25]	Zhao J F,Mao X,Chen L J.Speech Emotion Recognition Using Deep 1D &2D CNN LSTM Networks[J].Biomedical Signal Processing and Control (S1746-8094),2019,47:312-323.
[26]	Bo Hongjian,Ma Lin,Kong Xianghao,et al.Research on Dimension Reduction Method of Speech Emotion Features Based on Convolutional Neural Network Learning[J].High tech communication (S1002-0470),2017,27(S2):889-898.
[27]	Man Hao,Wei-Hua Cao.Visual-Audio Emotion Recognition Based on Multi-Task and Ensemble Learning with Multiple Features[J].Pattern Analysis and Applications (S0925-2312),2020,391:42-51.
[28]	Sun B H,Wang H,Fang B F,et al.Task Allocation of Emotional Robot Pursuit Based on Self-Organizing Algorithm[J].Robot (S1002-0446),2017,39(5):680-687.
[29]	Ren Hao,ye Liang,Li Yue,et al.Speech Emotion Recognition Algorithm Based on Multilevel SVM Classification[J].Computer application research (S1001-3695),2017,34(6):1682-1684.
[30]	Han Wenjing,Li Haifeng,Ruan Huabin,et al.Review of Speech Emotion Recognition Research[J].Acta software Sinica (S1000-9825),2014,25(1):37-50.
[31]	Li Aijun,Shao Pengfei,Dang Jianwu.Cross Cultural Multimodal Perception of Emotional Expression[J].Journal of Tsinghua University (NATURAL SCIENCE EDITION)(S1000-0054),2009,49(S1):1393-1401.
[32]	Zhang Jian,Qu Dan,Li Zhen.Cyclic Neural Network Language Model Based on Word Vector Feature[J].Pattern recognition and artificial intelligence (S1003-6059),2015,28(4):299-305.
[33]	Zhang Ziheng,Sun Ying,Yao Hui.Research on Nonlinear Characteristics of Emotional Speech Based on Chaotic Characteristics[J].Microelectronics and computer (S0469-5097),2017,34(4):65-68,75.
[34]	Liang Ruiyu,Zhao Li,Tao Huawei,et al.Speech Emotion Recognition Algorithm Based on Selective Attention Mechanism[J].Acta acoustics (S0371-0025),2016,41(4):537-544.
[35]	Ke X X,Yun Y W,Qiu B R.Emotional Robot Expression Research with Different Personalities[J].Journal of Intelligent Systems (S1673-4785),2017,12(4):468-474.

基于随机森林和卷积特征学习的人机交互语音情感识别

Human-computer Interaction Speech Emotion Recognition Based on Random Forest and Convolution Feature Learning

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献 35

相关文章 8

编辑推荐

Metrics

本文评价

[1]	刘兴, 王艳, 纪志成. 基于随机森林的风电功率短期预测方法[J]. 系统仿真学报, 2021, 33(11): 2606-2614.
[2]	陈涛, 王艳, 纪志成. 基于经验小波变换的光伏功率组合预测模型[J]. 系统仿真学报, 2021, 33(11): 2627-2635.
[3]	杨文峰, 王艳, 纪志成. 基于RF-GA-BP神经网络的N-乙酰氨基葡萄糖含量预测[J]. 系统仿真学报, 2020, 32(10): 2034-2040.
[4]	吴敬兵, 唐汉卿, 胥军. 水泥窑协同处置生活垃圾的燃烧特性分析优化[J]. 系统仿真学报, 2020, 32(1): 35-43.
[5]	李坤伟, 游雄, 张欣, 汤奋. 面向越野通行分析的土壤数据分类方法研究[J]. 系统仿真学报, 2019, 31(1): 158-165.
[6]	唐超, 张苗辉, 李伟, 曹峰, 王晓峰, 童晓红. 融合局部与全局特征的人体动作识别[J]. 系统仿真学报, 2018, 30(7): 2497-2506.
[7]	叶继华, 时淑霞, 李汉曦, 左家莉, 王仕民. 一种结合深度学习和随机森林的地平线检测方法[J]. 系统仿真学报, 2018, 30(7): 2507-2514.
[8]	杨兴, 吴静, 周建国, 江昊, 朱劼. 基于仿真数据驱动的空间信息网络建模方法[J]. 系统仿真学报, 2018, 30(11): 4323-4334.