makerfairerome
Home   >   News   >   Open-Source Code for Hand Gesture Recognition – Sign Language Translation

Open-Source Code for Hand Gesture Recognition – Sign Language Translation

By on August 23, 2019
Pin It

MediaPipe is a framework for building multimodal (eg. video, audio, any time series data) applied ML pipelines. With MediaPipe, a perception pipeline can be built as a graph of modular components, including, for instance, inference models (e.g., TensorFlow, TFLite) and media processing functions.

hand_tracking_android_gpu.gif

In the visualization above, the red dots represent the localized hand landmarks, and the green lines are simply connections between selected landmark pairs for visualization of the hand skeleton. The red box represents a hand rectangle that covers the entire hand, derived either from hand detection (see hand detection example) or from the pervious round of hand landmark localization using an ML model (see also model card). Hand landmark localization is performed only within the hand rectangle for computational efficiency and accuracy, and hand detection is only invoked when landmark localization could not identify hand presence in the previous iteration.

The example can also run in a mode that localizes hand landmarks in 3D (i.e., estimating an extra z coordinate):

hand_tracking_3d_android_gpu.gif

In the visualization above, the localized hand landmarks are represented by dots in different shades, with the brighter ones denoting landmarks closer to the camera.

 

More info

OpenStore

About Boris Landoni

Boris Landoni is the technical manager of Open-Electronics.org. Skilled in the GSM field, embraces the Open Source philosophy and its projects are available to the community.

Leave a Reply

Your email address will not be published. Required fields are marked *