一种音乐舞蹈视频关键帧提取方法

doi:10.16182/j.issn1004731x.joss.201807046

系统仿真学报 ›› 2018, Vol. 30 ›› Issue (7): 2801-2807.doi: 10.16182/j.issn1004731x.joss.201807046

一种音乐舞蹈视频关键帧提取方法

马楠¹, 石祥滨^1,2, 代钦³, 刘翠微¹, 刘芳¹

1. 沈阳航空航天大学,辽宁沈阳 110136;
2. 辽宁大学,辽宁沈阳 110036;
3. 沈阳工程学院信息学院,辽宁沈阳 110136

收稿日期:2017-07-31 出版日期:2018-07-10 发布日期:2019-01-08
作者简介:马楠(1992-),女,辽宁,硕士生,研究方向为视频分析,图像处理,音频处理。
基金资助:
国家自然科学基金(61170185,61602320),辽宁省博士启动基金(201601172),辽宁省教育厅一般项目(L201607,L2014070)

A Method of Key Frame Extraction for Music and Dance Video

Ma Nan¹, Shi Xiangbin^1,2, Dai Qin³, Liu cuiwei¹, Liu Fang¹

1. College of Computer Shenyang Aerospace University, Shenyang 110136, China;
2. College of Information Liaoning University, Shenyang 110036, China;
3. College of Information, Shenyang Institute of Engineering, Shenyang 110136, China

Received:2017-07-31 Online:2018-07-10 Published:2019-01-08

摘要/Abstract

摘要： 针对音乐舞蹈视频动作复杂,冗余动作多的问题,提出了一种提取舞蹈视频关键帧的方法。对舞蹈视频中相邻的两个图像帧进行光流计算,计算每张光流图的熵并提取舞蹈视频中音乐的特征,将音乐特征与光流图的熵值序列进行融合,得到一个与音乐相匹配的熵值序列,通过阈值比较得到舞蹈视频的关键帧集合。实验结果表明：该算法能够有效地提取出冗余少且概括视频内容的关键帧,对舞蹈视频分析和舞蹈动作编排等方面起到了重要的辅助作用。

关键词: 关键帧, 光流图, 图像熵, 音乐特征, 特征融合

Abstract: Aiming at the complicated and redundant action of music and dance video, a method of extracting dance video key frame is proposed. The optical flow of two adjacent image frames in the dance video is calculated. The entropy of each optical flow graph is calculated and the characteristics of the music in the dance video are extracted. The music characteristics are merged with the entropy sequence of the optical flow graph and a sequence of entropy matching with music is obtained. The key frame of dance video is achieved by threshold comparison. The experimental results show that the algorithm can effectively extract the key frames with less redundancy and summary of the video content, and plays an important role in the analysis of dance video and choreography.

Key words: key frame, optical flow graph, image entropy, musical characteristics, feature fusion

中图分类号:

TP391.4

马楠, 石祥滨, 代钦, 刘翠微, 刘芳. 一种音乐舞蹈视频关键帧提取方法[J]. 系统仿真学报, 2018, 30(7): 2801-2807.

Ma Nan, Shi Xiangbin, Dai Qin, Liu cuiwei, Liu Fang. A Method of Key Frame Extraction for Music and Dance Video[J]. Journal of System Simulation, 2018, 30(7): 2801-2807.

参考文献

[1] Cao J, Yu L, Chen M, et al.A Key Frame Selection Algorithm Based on Sliding Window and Image Features[C]//Parallel and Distributed Systems (ICPADS), 2016 IEEE 22nd International Conference on. IEEE(S1521-9097), 2016: 956-962.
[2] Chugh I, Gupta R, Kumar R, et al.Techniques for key frame extraction: Shot segmentation and feature trajectory computation[C]//Cloud System and Big Data Engineering (Confluence), 2016 6th International Conference. IEEE, 2016: 463-466.
[3] 王宇, 汪荣贵, 杨娟. 一种新的自适应的视频关键帧提取方法[J]. 合肥工业大学学报(自然科学版), 2016, 39(11): 1483-1487, 1542.
WANG Yu, WANG Ronggui, YANG Juan.A novel adaptive video key frame extraction method[J]. Journal of Hefei University of Technology (Natural Science), 2016, 39(11): 1483-1487, 1542.
[4] Zhang H J, Wu J, Zhong D, et al.n integrated system for content-based video retrieval and browsing[J]. Pattern Recognit.(S0031-3203), 1997, 30(4): 643-658.
[5] Fuentes J, Ruiz J, Rendon J M.Salient Point Tracking for Key Frames Selection in Outdoor Image Sequences[J]. IEEE Latin America Transactions(S1548-0992), 2016, 14(5): 2461-2469.
[6] 庞亚俊. 基于先验的动作视频关键帧提取[J]. 河南理工大学学报(自然科学版), 2016, 35(6): 862-868.
PANG Yajun.Key frames extraction of motion video based on prior knowledge[J]. Journal of Henan Polytechnic University (Natural Science), 2016, 35(6): 862-868.
[7] 应锐, 蔡瑾, 冯辉, 等. 基于运动块及关键帧的人体动作识别[J]. 复旦学报(自然科学版), 2014, 53(6): 815-822.
YING Rui, CAI Jin, FENG Hui, et al.Human Action Recognition Based on Motion Blocks and Key Frames. Journal of Fudan University (Natural Science), 2014, 53(6): 815-822.
[8] Chen C Y, Wang J C, Wang J F, et al.Motion entropy feature and its applications to event-based segmentation of sports video[J]. EURASIP Journal on Advances in Signal Processing (S1687-6172), 2008, 2008(1): 152.
[9] Horn B K P, Schunck B G. Determining optical flow[J]. Artificial intelligence (S0004-3702), 1981, 17(1/2/3): 185-203.
[10] Wolf W.Key frame selection by motion analysis[C]// Acoustics, Speech, and Signal Processing (S1520-6149), 1996. ICASSP-96. Conference Proceedings, 1996 IEEE International Conference on. IEEE, 1996: 1228-1231.
[11] Brox T, Malik J.Large displacement optical flow: descriptor matching in variational motion estimation[J]. IEEE transactions on pattern analysis and machine intelligence (S0162-8828), 2011, 33(3): 500-513.
[12] Mentzelopoulos M, Psarrou A.Key-frame extraction algorithm using entropy difference[C]//Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval. ACM, 2004: 39-45.
[13] Scheirer E D.Tempo and beat analysis of acoustic musical signals[J]. The Journal of the Acoustical Society of America (S0001-4966), 1998, 103(1): 588-601.
[14] Knop M, Kapuściński T, Mleczko W K.Video key frame detection based on the restricted Boltzmann machine[J]. Journal of Applied Mathematics and Computational Mechanics (S2299-9965), 2015, 14(3): 49-58.
[15] Fei M, Jiang W, Mao W, et al.New fusional framework combining sparse selection and clustering for key frame extraction[J]. IET Computer Vision (S1751-9632), 2016, 10(4): 280-287.
[16] Liu T, Zhang H J, Qi F.A novel video key-frame-extraction algorithm based on perceived motion energy model[J]. IEEE transactions on circuits and systems for video technology (S1051-8215), 2003, 13(10): 1006-1013.

一种音乐舞蹈视频关键帧提取方法

A Method of Key Frame Extraction for Music and Dance Video

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 11

编辑推荐

Metrics

本文评价

[1]	曹建芳, 崔红艳, 张自邦, 赵爱迪. 结合迁移学习和底层特征的古代壁画分类模型[J]. 系统仿真学报, 2021, 33(5): 1095-1103.
[2]	王远明, 张珺, 秦远辉, 柴秀娟. 基于多特征融合的指挥手势识别方法研究[J]. 系统仿真学报, 2019, 31(2): 346-352.
[3]	刘望, 孙金玉, 马世伟. 基于时空特征金字塔网络的动作时序检测方法[J]. 系统仿真学报, 2019, 31(11): 2382-2387.
[4]	王仕民, 叶继华, 王明文, 左家莉, 刘长红. 基于多尺度区域协方差的显著性特征提取方法[J]. 系统仿真学报, 2018, 30(7): 2767-2775.
[5]	李鸣晓, 庚琦川, 莫红, 吴威, 周忠. 基于片段关键帧的视频行为识别方法[J]. 系统仿真学报, 2018, 30(7): 2787-2793.
[6]	魏毅, 夏时洪. 物理逼真的人体空翻运动合成[J]. 系统仿真学报, 2018, 30(6): 2190-2197.
[7]	张新良, 李阳, 赵运基. 一种基于时空切片的SLAM关键帧提取方法[J]. 系统仿真学报, 2018, 30(5): 1724-1729.
[8]	胡凌燕, 曹禄, 熊鹏文, 辛勇, 谢泽坤. 基于RGB-D图像的三维同步定位与建图研究[J]. 系统仿真学报, 2017, 29(11): 2840-2846.
[9]	王仕民, 叶继华, 程柏良, 王明文. 基于多尺度张量空间的改进Itti视觉显著性检测[J]. 系统仿真学报, 2016, 28(9): 2138-2145.
[10]	叶继华, 兰清平, 刘长红, 王仕民. 结合互信息量和Log-Gabor特征的嵌入式人脸识别[J]. 系统仿真学报, 2016, 28(9): 2214-2219.
[11]	石祥滨, 刘拴朋, 张德园. 基于关键帧的人体动作识别方法[J]. 系统仿真学报, 2015, 27(10): 2401-2408.