Application of gesture estimation method based on computer
List of Authors
  • S. B. Goyal , Wu Chen

Keyword
  • computer vision, pose estimation, fall detection. Azure Kinect DK, openpose

Abstract
  • This paper will study the computer vision pose estimation algorithm and apply it to fall detection. At present, the problem of aging cannot be ignored. There are more and more elderly people living alone. Due to their poor physical fitness, they are very easy to fall in daily life and bring great harm to the body and mind of the elderly. Therefore, they are in great need of means to monitor the daily life of the elderly in real time and make judgments on fall events. Driven by the rapid development of computer vision technology, human pose estimation has been widely used in fields such as motion recognition, motion posture scoring, and assisted rehabilitation therapy. The core of pose estimation is to detect the joint information and bone information of the human body from the image, and connect them to form a bone image to help the computer understand the human body information. The use of pose estimation for fall detection requires the extraction of human joint point information and bone information. In order to extract human joint point coordinates and bone images from video sequences, this paper has done a lot of research on computer vision pose estimation. First, the background subtraction method is studied. After that, the openpose pose estimation algorithm is studied. then studied the attitude estimation of Azure Kinect DK depth camera, focusing on its TOF principle and body tracking principle. After comparison, it is found that traditional pose estimation such as openpose is based on optical RGB images. In this paper, we mainly use Azure Kinect DK depth camera for fall detection. In the fall detection method, this paper uses the threshold method combined with pattern recognition method to detect falls. The thresholding method uses the head fall speed and the body tilt angle. The tilt angle of the upper body is used to solve the bending and squatting errors. The pattern recognition method transforms fall detection into two classification problems, fall and no-fall, and uses a support vector machine binary classification method combined with a directed gradient histogram to extract features from skeletal images for classification. The final experimental data show that this paper implements fall detection based on human posture estimation using Azure Kinect DK depth camera with high accuracy and practicality.

Reference
  • 1. Alan J Lipton, H Fujiyoshi, Raju S Patil. Moving target classification and tracking from real time video[J]. IEEE Transaction on Workshop Application of Computer Vision, 1998:8-14.

    2. Alwan M, Rajendran P J, Kell S, et al. A smart and passive floor-vibration based fall detector for elderly[C]// Information and Communication Technologies. 2006, 1: 1003-1007.

    3. Bai Yunfei, Li Qu, He Jinlong. Design of fall detection system based on embedded video surveillance [J]. Television Technology, 2014, 38(15):231-236.

    4. Bourke A K, Lyons G M. A threshold-based fall-detection algorithm using a bi-axial gyroscope sensor [J]. Medical Engineering & Physics, 2008, 30(1):84-90.

    5. CAO Z, SIMON T, WEI S-E, et al. Realtime multi-person 2d pose estimation using part affinity fields[C] //. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 7291–7299.

    6. Chen Wei, Tong Lina, Song Quanjun, et al. Design of fall detection system based on inertial sensor devices [J]. Sensors and Microsystems System, 2010, 29(8):117-119.

    7. Chinese Journal of Electronics, 2011, 39(001):195-200.

    8. Dai Science, Li Guohui, Tu Dan, et al. Research status and prospect of background technology for moving target detection in surveillance video. China Image Acta Graphics. 2006, 11(7):919-927

    9. Du Peng, Li Long. Long-term trend prediction of China's population aging in the new era [J]. Journal of Renmin University of China, 2021, 35(01): 96-109.

    10. Du Youtian, Chen Feng, Xu Wenli, et al. Review of Vision-Based Human Motion Recognition [J]. Chinese Journal of Electronics, 2007, 35(001):84-90.

    11. Harville M. A Framework for High-Level Feedback to Adaptive, Per-Pixel, Mixture-of- Gaussian Background Models[C]// Computer Vision, 2002: 543-560.

    12. Huang Jin, Han Dongqi, Chen Yineng, et al. Overview of Human-Computer Interaction in Mixed Reality [J]. Computer Aided Design and Graphics Journal, 2016, (06): 869-880.

    13. Huang Kaiqi, Chen Xiaotang, Kang Yunfeng, Tan Tieniu. Overview of Intelligent Video Surveillance Technology [J]. Journal of Computer, 2015,38(06):1093-1118.

    14. Huo Hongwei, Zhang Hongke, Youzhi XU. Research on fall detection of the elderly based on the radio frequency signal of indoor wireless sensor network

    15. James Charles, Tomas Pfister, Mark Everingham, Andrew Zisserman. Automatic and Efficient Human Pose Estimation for Sign Language Videos[J]. International Journal of Computer Vision, 2014, 110(1).

    16. Johansson G. Visual perception of biological motion and a model for its analysis[J]. Perception & Psychophysics, 1973, 14.

    17. Lai C F, Chang S Y, Chao H C, et al. Detection of cognitive injured body region using multiple triaxial accelerometers for elderly falling[J]. IEEE Sensors Journal, 2011, 11(3): 763-770.

    18. Marr D. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information[J]. Quarterly Review of Biology, 2010, 8.

    19. Mathie M J, Coster A C F, Lovell N H, et al. Accelerometry: providing an integrated, practical method for long-term, ambulatory monitoring of human movement[J]. Physiological measurement, 2004, 25(2): R1-R20.

    20. Miaou S G, Sung P H, Huang C Y. A Customized Human Fall Detection System Using Omni-Camera Images and Personal Information[C]//Distributed Diagnosis and Home Healthcare, 2006.D2h2. Transdisciplinary Conference on. IEEE, 2006:39-42.

    21. Mscoco keypoint evaluation metric. http://mscoco.org/dataset/keypoints-eval, 2017.

    22. Rougier C, Meunier J, St-Arnaud A, et al. Robust Video Surveillance for Fall Detection Based on Human Shape Deformation[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2011, 21(5):611-622.

    23. Shi Xin, Zhang Tao. Design of a wearable fall detection device [J]. Journal of Instrumentation, 2012,33(3):575-580.

    24. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C]. International Conference on Learning Representations 2015. San Diego: ICLR,2015：1150-1210.

    25. Song Fei, Xue Zhi. Design and implementation of fall detection for the elderly based on OpenCV [J]. Information Technology, 2015(11):137-139.

    26. Tian Lei, Wang Peng, Liang Guoqiang, Shen Chunhua. An adversarial human pose estimation network injected with graph structure[J]. Pattern Recognition, 2021, 115.

    27. Tinetti ME, Speechley M, Ginter SF. Risk factors for fall among elderly persons living in the community[J], New Engl J Med 1988, 319:1701-1707.

    28. Tomkun J, Nguyen B. Design of a Fall Detection and Prevention System for the Elderly[J].2010:565-569.

    29. Toshev A, Szegedy C. DeepPose: Human Pose Estimation via Deep Neural Networks[J]. 2013.

    30. Wan Weibing, Huo Hong, Zhao Yuming. Target Detection and Recognition in Intelligent Video Surveillance [M]. Shanghai Jiaotong University Press, 2010.

    31. WEI S-E, RAMAKRISHNA V, KANADE T, et al. Convolutional pose machines[C]// .Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2016: 4724–4732.

    32. Xu Guangyou. Computer Vision [M]. Beijing: Tsinghua University Press, 1999.

    33. Yu M, Naqvi S M, Rhuma A, et al. One Class Boundary Method Classifiers for Application in a Video-Based Fall Detection System [J]. IET Computer Vision, 2012, 6(2):90-100.

    34. Zhang C, Tian Y, Capezuti E. Privacy Preserving Automatic Fall Detection for ElderlyUsing RGBD Cameras[C]// International Conference on Computers for Handicapped Persons. Springer Berlin Heidelberg, 2012: 625-633.

    35. Zhuang X, Huang J, Potamianos G, et al. Acoustic fall detection using Gaussian mixture models and GMM supervectors[C]//IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE Computer Society, 2009:69-72.

    36. Zigel Y, Litvak D, Gannot I. A method for automatic fall detection of elderly people using floor vibrations and sound—Proof of concept on human mimicking doll falls[J]. IEEE Transactions on Biomedical Engineering, 2009, 56(12): 2858-2867.