Yoga training with YogAI and a Raspberry Pi smart mirror | The MagPi issue 80

By Boris Landoni on March 28, 2019

Running on a smart mirror, YogAI uses a database of postures, image recognition software, and the magic of mirrors to not only show users their current posture but to also teach them how to correct their posture to reach peak yogi-ness. Here’s Rob Zwetsloot from The MagPi magazine with more.

We’ve seen many ‘magic mirror’ projects over the past few years, featuring a TV screen behind the glass to show useful information, but YogAI takes the concept to a whole new level by providing an AI personal trainer to guide and correct your yoga positions.

Self-confessed fitness nuts Salma Mayorquin and Terry Rodriguez thought that having a personal trainer could be a way to keep track of their fitness progress, so why not try to make a virtual one? “With [deep learning] models like pose estimation, we figured there was a way we could make a program that could track how we were exercising and started experimenting from there,” says Terry.

“YogAI guides users through a flow of yoga poses, offering generally helpful advice when the camera senses a user not in the correct pose,” explains Salma. “At the heart, YogAI uses pose estimation to find reference key points on the body. This is used to understand and classify common yoga poses.”

Users interact with YogAI through both visual feedback via the mirror display, and a voice interface — using the Snips AIR voice assistant — which enables the user to give spoken commands to start, stop, pause, and restart a yoga session. YogAI also talks back through the Flite voice synthesiser to guide the yogi to achieve the correct poses.

While a prototype magic mirror only took the experienced makers a week to build, training the AI to recognise yoga poses in real time was a trickier task. “We need our computer vision models to run quickly so that we have enough resolution in time to identify the move,” reveals Terry.

Strike a pose

A Raspberry Pi 3 interprets the camera images in real time, detecting key body points to display the pose on the mirror and classify it using a deep-learning model trained with a dataset of around 35000 samples.

However, the pair found that the Pi could only run image inference at one frame every 4–5 seconds, resulting in lag. A workaround was soon found: “Shrinking our pose estimation models down using TensorFlow Lite, we were able to bring our frame rate from 0.2 fps to 2.5 fps,” says Salma. “For faster inference, we will look for ways to reduce the model further. We also believe upgrading to the Raspberry Pi Compute Module 3 will increase the performance significantly.”

“Overall, the accuracy across a dozen common poses is roughly 80%,” divulges Terry. “Not surprisingly, we find similar pose variants, e.g. warrior poses, can be a source of confusion. When the head/face is blocked, the pose estimates degrade, which impacts our classification of poses like downward dog.”

More intense exercise

As well as using the system for yoga, Salma and Terry are planning to adapt YogAI to monitor more energetic workouts. “We’re interested in strength training, and others have suggested dance and karate katas,” says Terry. “We think YogAI is well-positioned to perform more general health and personal wellness tasks.”

“We want to integrate with popular health wearables,” adds Salma. “A smart watch with an accelerometer and heart rate monitor can introduce a lot of important context to bring YogAI closer to our vision for a smart mirror yoga instructor and toward a personal wellness platform.”

More from The MagPi magazine

The MagPi magazine issue 80 is out today. Buy your copy now from the Raspberry Pi Press store, major newsagents in the UK, or Barnes & Noble, Fry’s, or Micro Center in the US. Or, download your free PDF copy from The MagPi magazine website.

Subscribe now

Subscribe to The MagPi magazine on a monthly, quarterly, or twelve-month basis to save money against newsstand prices!

Twelve-month print subscribers get a free Raspberry Pi 3A+, the perfect Raspberry Pi to try your hand at some of the latest projects covered in The MagPi magazine.

The post Yoga training with YogAI and a Raspberry Pi smart mirror | The MagPi issue 80 appeared first on Raspberry Pi.

About Boris Landoni

Boris Landoni is the technical manager of Open-Electronics.org. Skilled in the GSM field, embraces the Open Source philosophy and its projects are available to the community.