How it Works...

All Workouts Welcome

FitMonkey uses computer vision to accurately count repetitions and provide feedback on exercise form.

  1. Modular Code means new exercises can be trained in short cycles.
  2. Add routines as your gym expands.
  3. New workouts are always being added.

Our Proof of concept demo has been trained on pushups, pullups and squats, with the following performance results

Pushups Validation Test
F1 Accuracy F1 Accuracy
Pushup Detector 0.95 0.95 0.89 0.81
Bad Back Form 0.97 0.97 0.79 0.76

Squats Validation Test
F1 Accuracy F1 Accuracy
Squats Detector 0.98 0.98 0.93 0.93
Bad Form 0.99 0.99 0.81 0.75

Pullups Validation Test
F1 Accuracy F1 Accuracy
Pullup Detector 0.95 0.95 0.97 0.99

State of the Art Keypoint Tracking with Human2D Pose Estimation

We use Human Pose Estimation to generate input into our form trackers.
Human Pose Estimation localizes and captures human body parts, such as shoulders, elbows and ankles from the input video.

  • First we identify the confidence maps to identify body parts (fig 1a)
  • Then we identify the affinity fields (fig 1b) which represent the degree of association between the body parts
  • The confidence maps and affinity fields are used to infer 2d keypoints (fig 1c)
  • These keyopints map to CoCo bodyparts (fig 1d)
  • An aditional step is to calculate the angles between body part pairs, such as wrist and elbow

The keypoints, and angle coordinates are used to train exercise detection and posture models

Counting Reps with Optical Flow

We use Optical flow to count reps.
Optical flow is the pattern of apparent motion of image objects between two consecutive frames in a sequence (e.g. frames in a video). It shows the movement of points fetween frames using a 2D vector field, where each vector represents the displacement (difference) of frames.

We use Open CV's implementation of Dense Optical Flow using Gunner Farneback’s algorithm to track the movement of the user performing the exercise. The algorithm calculates the optical flow for every pixel in the image.

Because anarobic exercises are Home consistent and repetitive motion, we can track the change in direction of movement as an example of a repetition And because optical flow tracks movement - we can use the same algorithm for all exercises, negating the need to train multiple models.
Fig.2 - Optical Flow field

Optimized for Edge, Works on a desktop

Our models have been trained to run on edge devices, which can be deployed throughout the gym

  • Each station can have it's own inexpensive device
  • Can scale as your gym grows
  • Try out on new equipment
  • No expensive hosting costs, all the compute is done locally
FitMonkey will also run on a standard desktop computer - see table right for the performance of a 21 second pushup routine
APPLICATION RUN TIME (secs)
EDGE DEVICE - ARM GPU DESKTOP - INTEL CPU
full pipeline 16.43 46.91
reading 17.01 5.93
get pose feature 5.97 31.22
pre-process 0.54 0.14
optical flow 5.50 14.65
push up prediction 2.86 0.42
post-process 0.12 0.00
back prediction 0.03 0.00
Optical flow FPS 228.09 85.62
Full pipeline FPS 76.38 26.75

Works at home, when you can't get to the gym

Fitmap can work for home routines just as well as at the gym. By simply using the cell phone to capture the workout video in real time, Fitmap will count your reps and tell you when form needs improving