A Method of Key Frame Extraction for Music and Dance Video

doi:10.16182/j.issn1004731x.joss.201807046

Abstract

Abstract: Aiming at the complicated and redundant action of music and dance video, a method of extracting dance video key frame is proposed. The optical flow of two adjacent image frames in the dance video is calculated. The entropy of each optical flow graph is calculated and the characteristics of the music in the dance video are extracted. The music characteristics are merged with the entropy sequence of the optical flow graph and a sequence of entropy matching with music is obtained. The key frame of dance video is achieved by threshold comparison. The experimental results show that the algorithm can effectively extract the key frames with less redundancy and summary of the video content, and plays an important role in the analysis of dance video and choreography.

Key words: key frame, optical flow graph, image entropy, musical characteristics, feature fusion

CLC Number:

TP391.4

Ma Nan, Shi Xiangbin, Dai Qin, Liu cuiwei, Liu Fang. A Method of Key Frame Extraction for Music and Dance Video[J]. Journal of System Simulation, 2018, 30(7): 2801-2807.

References

[1] Cao J, Yu L, Chen M, et al.A Key Frame Selection Algorithm Based on Sliding Window and Image Features[C]//Parallel and Distributed Systems (ICPADS), 2016 IEEE 22nd International Conference on. IEEE(S1521-9097), 2016: 956-962.
[2] Chugh I, Gupta R, Kumar R, et al.Techniques for key frame extraction: Shot segmentation and feature trajectory computation[C]//Cloud System and Big Data Engineering (Confluence), 2016 6th International Conference. IEEE, 2016: 463-466.
[3] 王宇, 汪荣贵, 杨娟. 一种新的自适应的视频关键帧提取方法[J]. 合肥工业大学学报(自然科学版), 2016, 39(11): 1483-1487, 1542.
WANG Yu, WANG Ronggui, YANG Juan.A novel adaptive video key frame extraction method[J]. Journal of Hefei University of Technology (Natural Science), 2016, 39(11): 1483-1487, 1542.
[4] Zhang H J, Wu J, Zhong D, et al.n integrated system for content-based video retrieval and browsing[J]. Pattern Recognit.(S0031-3203), 1997, 30(4): 643-658.
[5] Fuentes J, Ruiz J, Rendon J M.Salient Point Tracking for Key Frames Selection in Outdoor Image Sequences[J]. IEEE Latin America Transactions(S1548-0992), 2016, 14(5): 2461-2469.
[6] 庞亚俊. 基于先验的动作视频关键帧提取[J]. 河南理工大学学报(自然科学版), 2016, 35(6): 862-868.
PANG Yajun.Key frames extraction of motion video based on prior knowledge[J]. Journal of Henan Polytechnic University (Natural Science), 2016, 35(6): 862-868.
[7] 应锐, 蔡瑾, 冯辉, 等. 基于运动块及关键帧的人体动作识别[J]. 复旦学报(自然科学版), 2014, 53(6): 815-822.
YING Rui, CAI Jin, FENG Hui, et al.Human Action Recognition Based on Motion Blocks and Key Frames. Journal of Fudan University (Natural Science), 2014, 53(6): 815-822.
[8] Chen C Y, Wang J C, Wang J F, et al.Motion entropy feature and its applications to event-based segmentation of sports video[J]. EURASIP Journal on Advances in Signal Processing (S1687-6172), 2008, 2008(1): 152.
[9] Horn B K P, Schunck B G. Determining optical flow[J]. Artificial intelligence (S0004-3702), 1981, 17(1/2/3): 185-203.
[10] Wolf W.Key frame selection by motion analysis[C]// Acoustics, Speech, and Signal Processing (S1520-6149), 1996. ICASSP-96. Conference Proceedings, 1996 IEEE International Conference on. IEEE, 1996: 1228-1231.
[11] Brox T, Malik J.Large displacement optical flow: descriptor matching in variational motion estimation[J]. IEEE transactions on pattern analysis and machine intelligence (S0162-8828), 2011, 33(3): 500-513.
[12] Mentzelopoulos M, Psarrou A.Key-frame extraction algorithm using entropy difference[C]//Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval. ACM, 2004: 39-45.
[13] Scheirer E D.Tempo and beat analysis of acoustic musical signals[J]. The Journal of the Acoustical Society of America (S0001-4966), 1998, 103(1): 588-601.
[14] Knop M, Kapuściński T, Mleczko W K.Video key frame detection based on the restricted Boltzmann machine[J]. Journal of Applied Mathematics and Computational Mechanics (S2299-9965), 2015, 14(3): 49-58.
[15] Fei M, Jiang W, Mao W, et al.New fusional framework combining sparse selection and clustering for key frame extraction[J]. IET Computer Vision (S1751-9632), 2016, 10(4): 280-287.
[16] Liu T, Zhang H J, Qi F.A novel video key-frame-extraction algorithm based on perceived motion energy model[J]. IEEE transactions on circuits and systems for video technology (S1051-8215), 2003, 13(10): 1006-1013.