基于图像序列的学习表情识别

doi:10.16182/j.issn1004731x.joss.19-VR0470

系统仿真学报 ›› 2020, Vol. 32 ›› Issue (7): 1322-1330.doi: 10.16182/j.issn1004731x.joss.19-VR0470

基于图像序列的学习表情识别

王素琴¹, 张峰¹, 高宇豆², 石敏¹

1. 华北电力大学控制与计算机工程学院,北京 102206;
2. 云南电网有限责任公司信息中心,云南昆明 650200

收稿日期:2019-08-30 修回日期:2019-12-12 出版日期:2020-07-25 发布日期:2020-07-15
作者简介:王素琴(1970-),女,辽宁,硕士,副教授,硕导,研究方向为计算机视觉,软件工程;张峰(1996-),男,山西,硕士生,研究方向为计算机视觉,机器学习。

Learning Expression Recognition Based On Image Sequence

Wang Suqin¹, Zhang Feng¹, Gao Yudou², Shi Min¹

1. School of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China;
2. Information center of Yunnan Power Grid Co., Ltd, Kunming 650200, China

Received:2019-08-30 Revised:2019-12-12 Online:2020-07-25 Published:2020-07-15

摘要/Abstract

摘要： 对学习者面部表情进行识别,能够判断学习者的情绪状态,分析其学习效果。针对面部表情具有持续性和时序性的特点,采用表情图像序列作为表情识别对象。通过组合网络的方式,将长短期记忆网络(Long Short Term Memory Network, LSTM)与VGGNet组合成VGGNet-LSTM模型,在此基础上进行表情识别,显著提高了识别准确率。借鉴迁移学习方法,将VGGNet通过基本表情数据集CK+进行预训练后迁移到学习表情数据集下,避免了学习表情数据集数据量不足的缺陷,解决了模型过拟合问题。

关键词: 表情识别, 学业情绪, 图像序列, 迁移学习

Abstract: By recognizing the facial expressions, the emotional state of learner can be judged and their learning effect can be analyzed. Due to the persistence and timing of facial expressions, the sequence of facial images is adopted as the object of facial expression recognition. The Long Short Term Memory Network (LSTM) and VGGNet are combined into a VGGNET-LSTM model. On this basis, facial expression recognition is carried out, which significantly improves the accuracy of recognition. Based on the transfer learning method, VGGNet is transferred to the learning expression data set after being pre-trained through the basic expression data set CK+ and avoids the defect of insufficient data in the learning expression data set and solves the problem of overfitting the model.

Key words: expression recognition, academic emotion, image sequence, transfer learning

中图分类号:

TP391.4

王素琴, 张峰, 高宇豆, 石敏. 基于图像序列的学习表情识别[J]. 系统仿真学报, 2020, 32(7): 1322-1330.

Wang Suqin, Zhang Feng, Gao Yudou, Shi Min. Learning Expression Recognition Based On Image Sequence[J]. Journal of System Simulation, 2020, 32(7): 1322-1330.

参考文献 20

[1]	Mehrabian A, Russell J A.An approach to environmental psychology[M]. Cambridge: MIT Press, 1980.
[2]	Ekman P, Friesen W V.Constants across cultures in the face and emotion[J]. Journal of Personality and Social Psychology (S0022-3514), 1971, 17(2): 124.
[3]	Pekrun R, Linnenbrink-Garcia L.Academic Emotions and Student Engagement[M]. Springer US: Handbook of Research on Student Engagement, 2012.
[4]	苏世将, 马惠霞. 大学生的一般学业情绪现状及其与大学适应的关系研究[J]. 中国健康心理学杂志, 2009, 17(5): 591-593.Su Shijiang, Ma Huixia.A study on the general academic mood of college students and its relationship with university adaptation[J]. China Journal of Health Psychology, 2009,17(5): 591-593.
[5]	Sun B, Wei Q, He J, et al.BNU-LSVED: a multimodal spontaneous expression database in educational environment[C]. Optics and Photonics for Information Processing X. California: SPIE - International Society for Optics and Photonics, 2016, 9970: 16-17.
[6]	He J, Li D, Yang B, et al.Multi View Facial Action Unit Detection Based on CNN and BLSTM-RNN[C]. 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). Washington DC: IEEE Computer Society, 2017.
[7]	Lopes A T, Aguiar E D, Oliveira-Santos T.A Facial Expression Recognition System Using Convolutional Networks[C]. Graphics, Patterns and Images. Salvador: IEEE, 2015: 273-280.
[8]	Lopes A T, de Aguiar E, De Souza A F, et al. Facial expression recognition with convolutional neural networks: coping with few data and the training sample order[J]. Pattern Recognition (S0031-3203), 2017, 61(4): 610-628.
[9]	Lucey P, Cohn J F, Kanade T, et al.The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression[C]. Computer Vision and Pattern Recognition Workshops. San Francisco, CA: IEEE, 2010: 94-101.
[10]	Yin L, Wei X, Sun Y, et al.A 3D facial expression database for facial behavior research[C]. International Conference on Automatic Face & Gesture Recognition. Southampton: IEEE, 2006.
[11]	袁源. 远程教育中不良表情的识别研究[D]. 成都: 四川师范大学, 2014.Yuan Yuan.Recognition of bad expressions in Distance Education[D]. Chengdu: Sichuan Normal University, 2014.
[12]	唐康. 人脸检测和表情识别研究及其在课堂教学评价中的应用[D]. 重庆:重庆师范大学, 2019.Tang Kang.Research on face detection and expression recognition and its application in classroom teaching evaluation[D]. Chongqing: Chongqing Normal University, 2019.
[13]	Goodfellow I J, Erhan D, Carrier P L, et al.Challenges in representation learning: A report on three machine learning contests[J]. Neural Networks (S0893-6080), 2015, 64: 59-63.
[14]	Greff K, Srivastava R K, Koutnik J, et al.LSTM: A Search Space Odyssey[J]. IEEE Transactions on Neural Networks & Learning Systems (S1045-9227), 2016, 28(10): 2222-2232.
[15]	Chao L, Tao J, Yang M, et al.Long Short Term Memory Recurrent Neural Network based Multimodal Dimensional Emotion Recognition[C]. International Workshop. Brisbane, Australia: ACM, 2015: 65-72.
[16]	杨雨浓. 基于深度学习的人脸表情识别方法研究[D]. 西安: 西北大学, 2017.Yang Yunong.Research on facial expression recognition based on deep learning[D]. Xi’an: Northwest University, 2017.
[17]	Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[EB/OL]. Eprint arXiv: 1409.1556, 2014, http://arxiv.org/abs/ 1409.1556.
[18]	Huang G B, Mattar M, Berg T, et al.Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments[R]. University of Massachusetts Amherst Technical Report, 2008: 7-49.
[19]	Baltru Aitis T, Robinson P, Morency L P.OpenFace: An open source facial behavior analysis toolkit[C]. IEEE Winter Conference on Applications of Computer Vision. Lake Placid, NY: IEEE, 2016: 1-10.
[20]	Deng J, Dong W, Socher R, et al.Imagenet: A large-scale hierarchical image database[C]. 2009 IEEE conference on computer vision and pattern recognition. Miami, FL: IEEE, 2009: 248-255.

基于图像序列的学习表情识别

Learning Expression Recognition Based On Image Sequence

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献 20

相关文章 3

编辑推荐

Metrics

本文评价

[1]	曹建芳, 崔红艳, 张自邦, 赵爱迪. 结合迁移学习和底层特征的古代壁画分类模型[J]. 系统仿真学报, 2021, 33(5): 1095-1103.
[2]	石祥滨, 房雪键, 张德园, 郭忠强. 基于深度学习混合模型迁移学习的图像分类[J]. 系统仿真学报, 2016, 28(1): 167-173.
[3]	詹永杰, 龙飞, 卜轶坤. 基于独立子空间分析特征学习的表情识别[J]. 系统仿真学报, 2015, 27(10): 2316-2319.