Samsung: Augmented Reality Spacesuit

Samsung, Cheil and the Science Museum Group teamed up with UNIT9 to help students step into the world of astronaut Tim Peake, using a giant augmented reality installation combining face and body tracking. As the European Space Agency’s first British astronaut, Tim Peake spent six incredible months aboard the International Space Station. We helped deliver the technology behind it.

Inspired by Tim Peake’s out-of-this-world journey to space, the augmented reality spacesuit combines face-tracking and body-tracking technology to mirror user’s actions in real time. It then projection maps the user into the shell of Tim Peake’s 200 lb spacesuit. Users can then take selfies and create a shareable video for social media.

We developed the project in Unity and used Microsoft's Kinect to track the user's body movements. We then transferred the skeleton data from the sensor onto a rigged 3D model of the spacesuit. In order to avoid glitches when tracking is briefly lost, we implemented custom joint restrictions that prevented the limbs from twisting unnaturally.

The augmented reality spacesuit worked by capturing a live video feed of the user’s face, and projecting it onto a 3D model of a human head inside the helmet. Using a HQ webcam we captured the user’s image, extracted it using face-tracking technology and applied it onto a 3D mesh.

One of the creative challenges was to deliver clear and true-to-life assets that could work on a giant 65’’ 4K Samsung screen, just half a metre away from the user. Not to mention the user experience design which had to make them feel like they were wearing a big 200lb spacesuit.