Image quality enhancement and fourier boundary decomposition for effective hand pose classification

Abstract

Advancements in computer vision facilitate new interactions between parts of the human body with technological devices. This research builds upon past technologies by introducing new digital image processing concepts and algorithms that improve hand based human-computer synergies. At present, hand gesture recognition systems are limited by the environmental conditions in which the signalled gesture is captured and the number of gestures that it can understand. Hand gestures are further characterised by the position and orientation of the fingers, however the lack of image features to recognise and track fingers makes it a challenging problem to solve.

In order to extract more precise hand boundaries, two image quality enhancement techniques that rely on the manipulation of its intensity channel values were adapted into a skin-pixel detector. Experiments were then conducted to study the effects of image quality enhancement techniques on skin pixel detection in a newly collected hand skin dataset. The new dataset gives focus to a range of human hand skin types under different illumination conditions and varies from traditional skin datasets that consist of face images and mainly focus on background distractions. From this, an improved Mixture of Gaussian based hand skin detector is trained.

Next, a method based on intuitive finger counting was introduced to simplify the hand gesture classification process. To achieve this, a new Fourier-based boundary decomposition method was introduced to divide hand boundaries into smaller sub-segments. Features were designed to evaluate the resemblance of each boundary segment to fingers and the wrist. Once recognised, finger tip and web points were identified by applying geometric manipulation.

The effectiveness of the proposed system were validated against state-of-the-art algorithms on natural scene images, skin pixel extraction datasets and an American Sign Language datasets. On a proof of concept example, where shapes drawn using tracked fingers are used as supplementary inputs to content-based image retrieval system, the proposed content-based image retrieval by hand gesture system was shown to possess potential to further enhance image retrieval performance.