Research Projects
| 
| Spine Pathology and Image Retrieval April 2008 – present U.S. National Library of Medicine, National Institutes of Health With increasing use of images in clinical medicine and biomedical research, there is a compelling need for efficient image retrieval techniques to support medical informatics applications. Content-Based Image Retrieval (CBIR) has been proposed as a possible solution for this problem. Previous research has mainly focused on extracting low-level visual features (e.g., color, texture, shape, spatial layout) and using them directly to compute image similarity. Extensive experiments have shown that such visual features cannot always capture the desired semantic concepts in an image. This poses a serious shortcoming in developing search and retrieval techniques where use of standardized biomedical concepts is routine. We present novel approaches for bridging the "semantic gap" between high-level semantic concepts and low-level image features to improve retrieval quality. |
| 
| Vision-based 3D Hand Posture Estimation using Hierarchical-ISOSOM Jan. 2005 - July 2007 CS Department, University of California, Santa Barbara Hand gesture is a potentially useful modality for visual-based interaction (VBI) in human-computer interaction (HCI), but estimating 3D hand posture and pose under camera viewpoint variations using 2D image is a very challenging problem due to the high intrinsic degrees of freedom (DOF) and the self-occlusion problem. The work mainly focuses on appearance-based 3D hand posture and pose estimation. We formulate the posture and pose problem to a nonlinear manifold learning, representation, and retrieval problem. With the 3D realistic hand model with commonly used hand postures, a large synthetic hand images database is built under different camera viewpoints. Each image with the shape features and the hand posture and pose ground-truth is a point on a highly nonlinear, complex hand pose and posture manifold. We apply the H-ISOSOM algorithm to learn the manifold with a concise, organized structure. Given the hand image, the features such as shape context based on 2D intensity edge and 2.5D depth edge are extracted. The relevant representatives with the hand posture and pose ground-truth on the manifold are retrieved by the learned map. Thus, the hand pose and posture are estimated. |
 | Multi-view Appearance-based 3D Hand Pose Estimation Jan. 2005 - July 2007 CS Department, University of California, Santa Barbara We describe a novel approach to appearance-based hand pose estimation which relies on multiple cameras to improve accuracy and resolve ambiguities caused by self-occlusions. Rather than estimating 3D geometry, our approach uses multiple views to extend current exemplar-based methods for estimating hand pose by matching a probe image with a large discrete set of labeled hand pose images. We formulate the problem in a MAP framework, where the information from multiple cameras is fused to provide reliable hand pose estimation. Our quantitative experimental results show that the correct estimation rate is much higher using our multi-view approach than using a single-view approach. |
| Research Across Disciplines: Human Body Tracking as Interface Device for Interactive Installations June 2005 - Sept. 2005 University of California, Santa Barbara The project is a collaborative work with the Experimental Visualization Lab, Media Arts and Technology. The project is realized as interactive art installation, which emphasis is on aesthetic research through the implementation of computer vision technologies for new forms of content, narratives, experiences and analysis. The system interactively changes the size, shape, and rotation of a graphic cylinder in the display by detecting and tracking the user’s face and hands’ locations using the skin color detection and mean-shift tracking technologies. The work addresses the poetics of presence, and interaction. |
| 
| Small Business Technology Transfer: Vision-Based Body Gesture Recognition Sept. 2003 - Jan 2004 CS Department, University of California, Santa Barbara Gesture recognition could be considered as a special case of motion pattern recognition, which plays an emergent role in Human-Computer-Interaction. Hand gesture is a potential candidate for the next generation interface. This project is a Small Business Technology Transfer (STTR) project. The objective is to automatically interpret director’s gestures to control (Taxing, Launch, Recovery) the Unmanned Air Vehicle (UAV) on the carrier deck. As the first phase of the project, we conceptualized, designed, and implemented a gesture recognition system, and proved the feasibility of taxiing UAV by vision-based gesture recognition approach accordingly. |
 | Human Body Detection and Tracking for Video Surveillance Aug. 2000 - Sept. 2001 Center for Signal Processing, Nanyang Technological University The project mainly focuses on the detection and tracking of a human body and head. We proposed a multi-modal algorithm with different kinds of features, such as color, head ellipse, and head-edge template. The human face is detected by the face detection technique, and then the head and body are tracked by the combinations of three tracking techniques: partial ellipse tracking, head color histogram tracking, and head edge template tracking. The hybrid multi-modal algorithm supports front-view, profile-view and rear-view of human head tracking. It can adapt to the scale and lighting changes, and it can also handle the occlusions to some extent. The multi-modal algorithm has been tested with a real-word image sequences. |
| 
| 3D Hand Posture Retrieval in a 2D Single Image July 1998 - July 2000 Nanyang Technological University The research efforts of passive sensing of 3D human hand posture have been hampered by the computational complexity caused by inverse kinematics and 3D reconstruction. In this project, we adopts some hand constraints to reduce the search space and present a close-form solution for 3D hand joint angle estimation based on a single 2D image. |