Model-based 3D Hand Posture Estimation
Haiying Guan and Chin Seng Chua
July 1998 - July 2000
School of Electrical and Electronic Engineering, Nanyang Technological University |  |
Overview
The research efforts of passive sensing of 3D human hand posture have been hampered by the computational complexity caused by inverse kinematics and 3D reconstruction. In this project, we adopts some hand constraints to reduce the search space and present a close-form solution for 3D hand joint angle estimation based on a single 2D image.
Summary
Hand gesture analysis has been studied extensively over the past decade in the field of computer vision. However, it has not advanced enough to provide sufficiently flexible and reliable performance for applications such as Computer Graphic Animation (CGA), Human-Computer Interaction (HCI), Sign Language Recognition (SLR) and Virtual Reality (VR).
The project focuses on the estimation of hand posture in 3D space from a single 2D image. This is a challenging work for the following two reasons: First, human hand is an articulated object that consists of 27 degrees of freedom (DOFs). Some degrees of freedom are constrained by static or dynamic kinematic equations. From the kinematic point of view, human hand has large state space and its appearance is very complicated. Second, it is inherently an ill-posed problem to recover the 3D information from a single 2D image.
Establishing the 3D human hand model is the first step in posture estimation. In this work, we analyze the human hand model constraints and simplify the existing hand configuration from 27 DOFs to 16 DOFs with good performance. The forward kinematic solution of the hand model is given by the Denavit-Hartenberg presentation.
The following hand constraints and assumptions are used:
Based on the kinematic model and its constraints, we proposed a new geometric algorithm to retrieve the posture of the hand in 3D space. The algorithm uses 2D positions of eight points namely, the wrist point, four finger tips, the metacarpal joint of the middle finger and the metacarpal joint and the tip of the thumb. Firstly, combined with the finger constraints, we analyze the geometrical relationship of the five points of the finger in the ``finger plane". The five points are the wrist point, the metacarpal joint, the proximal joint, the distal joint and the finger tip. Secondly, the ``finger plane" is transformed into the 3D space. The desired 3D information of these points is solved by a regression method. After that, we retrieve the 3D thumb posture with similar method. Finally, the whole hand posture is solved by a geometric method in 3D space.
The closed-form solution for θ3 is given by the following equations:
To locate the required eight 2D points in a given hand image, two kinds of feature extraction methods are studied in this project. First, hand contour is extracted to estimate hand model parameters in the initial frame. Second, color markers are used to identify the eight feature points in the estimation frame. The experimental results using real images illustrate the effectiveness of the algorithm in hand posture estimation.
References
[1] J. M. Rehg, “Visual Analysis of High DOF Articulated Objects with Application to Hand Tracking”, Ph.D. Thesis, Carnegie Mellon University, Dept. of Electrical and Computer Engineering, Technical Report CMU-CS-95-138, 1995.
[2] R. P. Paul, “Robot Manipulators: Mathematics, Programming, and Control”, The MIT PRESS, 1981.
[3] M. W. Spong and M. Vidyasagar, “Robot Dynamics and Control”, John Wiley and Sons, 1989.
Publications
[1] Chin-Seng Chua, Haiying Guan, and Yeong-Khing Ho, “Model-based 3D Hand Posture Estimation From A Single 2D Image”, In Image Vision and Computing, vol. 20, pp. 191-202, 2002. (PDF)
[2] Haiying Guan, Chin-Seng Chua and Yeong-Khing Ho, “3D Hand Pose Retrieval From A Single 2D Image”, In Proc. of the IEEE International Conference on Image Processing (ICIP'01), vol. 1, pp. 157-160, 2001. (PDF)(Poster)
[3] Haiying Guan, Chin-Seng Chua and Yeong-Khing Ho, “Model-based Finger Posture Estimation”, In Prof. of the IEEE Fourth Asian Conference on Computer Vision (ACCV'00), pp. 43-48, 2000.
[4] Haiying Guan, Chin-Seng Chua and Yeong-Khing Ho, “Hand Posture Estimation from 2D Monocular Image”, In Proc. of the IEEE Second International Conference on 3-D Digital Imaging and Modeling (3DIM'99), pp. 424-429, 1999.
[5] Haiying Guan, Chin-Seng Chua, Yeong-Khing Ho, Yang Yang, “Model-based Finger Posture Estimation from 2D Monocular Image”, In Proc. of Image and Vision Computing Conference, pp. 133-138, 1999.