Accurate full body tracking has benefits for everything from fitness applications, to gaming, to virtual reality, but no one wants to squeeze into one of those form-fitting body tracking suits that Hollywood uses to create visual effects. It turns out there’s a less awkward solution, and it leverages the devices you’re already carrying around all day, like earbuds or an iPhone.
Carnegie Mellon University’s Future Interfaces Group (FIG) has shared some new research into full-body tracking, and while it can’t begin to compete with the accuracy of a bodysuit and a million dollar studio, this new approach could still be precise enough to improve the accuracy of fitness tracking, or allow users to have their movements translated to a virtual reality world without the need for any special hardware.
The FIG research is actually reminiscent of patents Apple filed in the past few years, because the only equipment it relies on is a smartwatch, a smartphone, and wireless earbuds, which all now feature accelerometers, gyroscopes, and other motion-detecting sensors to facilitate more advanced features like fitness tracking, or even the Spacial Audio offered by Apple’s AirPods. The data from these sensors are made available through SDKs for developers to use in third-party apps and accessories, but the FIG researchers are harvesting it as a way to intelligently calculate a user’s full-body pose using a tool called IMUPoser.
Capturing the performance of an actor and translating it to a video game character or a CG-generated movie character requires the movements of hundreds of tracking points to be captured. But here, a smartphone in a pants pocket, a smartwatch on a wrist, and an earbud in the ear provide enough data to make an intelligent guess about a user’s body pose, including the position of all their limbs, or whether they’re standing tall or crouched down.
The approach doesn’t promise anywhere near 100% accuracy, because it makes a lot of assumptions to compensate for the minimal amount of tracking data it’s working with. For example, if it senses a person is walking, it will assume the arm and leg opposite the one tracked by the smartphone and smartwatch are also moving, just out of sync (which is the natural human walking gait). Of course, accuracy drops even more when IMUPoser is forced to rely on data from just two devices, or even one, but many of us can’t imagine not having our smart devices on our persons at all time.
Will this tool replace more robust body-tracking tools? Definitely not any time soon, but it can be used to improve and refine the data being collected from the motion sensors already used in smart devices. Many fitness trackers will make an attempt to guess what activity you’re doing, and this tool could help reduce the number of times users need to correct them, without the need for modifying or upgrading the physical hardware in any way.