This book offers a systematic, comprehensive, and timely review on V-HAR, and it covers the related tasks, cutting-edge technologies, and applications of V-HAR, especially the deep learning-based approaches. The field of Human Activity Recognition (HAR) has become one of the trendiest research topics due to the availability of various sensors, live streaming of data and the advancement in computer vision, machine learning, etc. HAR can be extensively used in many scenarios, for example, medical diagnosis, video surveillance, public governance, also in human-machine interaction applications. In HAR, various human activities such as walking, running, sitting, sleeping, standing, showering, cooking, driving, abnormal activities, etc., are recognized. The data can be collected from wearable sensors or accelerometer or through video frames or images; among all the sensors, vision-based sensors are now the most widely used sensors due to their low-cost, high-quality, and unintrusive characteristics. Therefore, vision-based human activity recognition (V-HAR) is the most important and commonly used category among all HAR technologies.
The addressed topics include hand gestures, head pose, body activity, eye gaze, attention modeling, etc. The latest advancements and the commonly used benchmark are given. Furthermore, this book also discusses the future directions and recommendations for the new researchers.
Autorentext
Zhongxu Hu is currently a research fellow within the Department of Mechanical and Aerospace Engineering of Nanyang Technological University in Singapore. His current research interests include human-machine interaction, computer vision, and deep learning applied to human behavior analysis and autonomous vehicles in multiple scenarios. He has contributed more than 20 papers in high-level international journals.
Dr. Hu serves as a lead guest editor for Computational Intelligence and Neuroscience and an academic editor/Editorial Board for Automotive Innovation, Journal of Electrical and Electronic Engineering, and is also an active reviewer for IEEE Transactions on Intelligent Transportation Systems, IEEE Transactions on Industrial Electronics, IEEE Intelligent Transportation Systems Magazine, Journal of Intelligent Manufacturing, and Journal of Advanced Transportation et al.
Chen Lv is currently an assistant professor at Nanyang Technology University, Singapore. His research focuses on advanced vehicle and human-machine systems, where he has contributed 2 books, 2 chapters, and over 100 papers and obtained 12 granted patents and 1 technology disclosure.
Dr. Lv serves as academic editor/editorial board member for IEEE Transactions on Intelligent Transportation Systems, SAE International Journal of Electrified Vehicles, International Journal of Vehicle Autonomous Systems, Automotive Innovation, etc., and guest editor for IEEE/ASME Transactions on Mechatronics, IEEE Transactions on Industrial Informatics, IEEE Intelligent Transportation Systems Magazine, Applied Energy, IEEE Sensors Journal, etc. He received many awards and honors, selectively including the Highly Commended Paper Award of IMechE UK in 2012, Japan NSK Outstanding Mechanical Engineering Paper Award in 2014, China SAE Outstanding Paper Award in 2015, the 1st Class Award of Automotive Industry Scientific and Technological Invention in 2015, Tsinghua University Outstanding Doctoral Thesis Award in 2016, Seal of Excellence of EU H2020 Marie Sklodowska-Curie Actions in 2017, Best Workshop/Special Session Paper Award of IEEE Intelligent Vehicle Symposium in 2018, Automotive Innovation Best Paper Award in 2020, and the winner of INTERPRET Challenge of NeurIPS 2020 competition.
Inhalt
1.Introduction.
1.1 Background of human activity recognition
1.2 Common used sensor types
1.3 Vision-based human activity related topics
This chapter will provide an overview of the V-HAR, introducing what types of topics are included in V- HAR and what types of application scenarios are available. A brief introduction of deep learning and computer vision will also be illustrated.
2. Vision-based hand gesture recognition
2.1 Introduction of hand gesture recognition
2.2 3d hand pose estimation based on depth sensor
2.3 Dynamic hand gesture recognition based on sequential images
2.4 Conclusion and and unsettled problems
This chapter will introduce the hand-related interaction topics including hand pose estimation, static and dynamic hand gesture recognition. The corresponding deep learning approach will be given. Some applications will also be introduced.
3. Vision-based facial state recognition
3.1 Introduction of facial related topics
3.2 Appearance-based head pose estimation
3.3 Eye gaze estimation based on facial feature
3.4 Conclusion and unsettled problems
This chapter mainly includes head pose estimation, eye gaze estimation, and eye fixation estimation. The corresponding deep learning approach and application will be elaborated. The research progress will be also presented.
4. Vision-based body activity recognition
4.1 Introduction of body activtiy recognition
4.2 Body pose estimation based on multi-scale network
4.3 Skeleton-based activity recognition
4.4 Emerging technique for human body acitivity
4.5 Conclusion and unsettled problems
This chapter will describe the body-related interaction topics including body pose estimation, dynamic action recognition, and body reconstruction. The research progress and approach will be introduced.
5. Human attention modelling
5.1 Visual saliency introduction
5.2 Data-driven human attention map estimation
5.3 Context-aware human focus estimation
5.4 Conclusion and unsettled problems
This chapter will illustrate how to estimate the human attention and discuss the relationship with the visual saliency of the environment. It will be introduced wherever it can be applied.
6. Conclusion and future work
6.1 Conclusion for vision-based human activity recognition
6.2 Recommendations for future work
This chapter will discuss the limitation and unsettled issues of the V-HAR, several recommendations will be given for the future direction.
