Real Time Hand Tracking and Pointing for Human Interaction

 Background: 

 Hand gesture is an efficient means for humans interacting with computers. Pointing gesture can resolve ambiguities derived from the verbal communication, thus opening up the possibility of humans interacting or communicating intuitively with computers or robots by indicating objects or pointed locations either in the three dimensional (3D) space or on the screen. However, it is a challenging task to estimate the 3D hand pointing direction automatically and reliably from the streams of video data due to the great variety and adaptability of hand movement and the undistinguishable hand features of the joint parts.

 Technology Overview: 

 The present technology provides a hand pointing estimation system based on two regular cameras, which includes hand region detection, hand finger estimation, two views' feature detection, and 3D pointing direction estimation. The hand detection system has similarities to a binary pattern face detector, in which a polar coordinate system is proposed to represent the hand region, and achieved a good result in terms of the robustness to hand orientation variation. To estimate the pointing direction, an Active Appearance Model (AAM) based approach was applied to detect and track 14 feature points along the hand contour from a top view and a side view. Combining two views of the hand features, the 3D pointing direction is estimated. 

 Advantages: 

• Real time robust tracking of hand using two orthogonal cameras without any intrusive glove or marks.
• Accurate pointing in a cursor resolution.
• Intuitive drawing in a 3D place.
• Feasible for finger pointing in a long distance.

 Applications: 

 

• Medical systems and assistive technologies
• Entertainment (i.e. computer and console video games)
• Crisis management and disaster relief
• Computer animation techniques

 Intellectual Property Summary: 

• U.S. 8,971,572 
• U.S. 9,128,530

Patent rights available for licensing