Disney Research’s mixed reality Magic Bench allows users to interact with digital characters

Imagine sitting down in front of a screen when suddenly, you hear increasingly louder “ribbit” sounds and a large frog hops onto the seat beside you, shaking the seat as it lands. This is one of the demonstrations for Disney Research‘s Magic Bench, a mixed reality experience that aims to let users see, hear and interact with animals and other digital characters in the real world.

The idea for the project came from the fact that mixed reality experiences can offer great experiences, but are often limited to cumbersome head-mounted hardware or other devices, such as Microsoft’s HoloLens. “This can be an isolating experience because you need the hardware to participate,” says Mo Mahler, principal digital artist on the project, in the demo video. “What we’re proposing instead of instrumenting a person, what can we accomplish by instrumenting the environment — creating a walk-up-and-play experience?”

Created by a Disney R&D team in Pittsburgh, Magic Bench portrays a mirrored image using an RGB camera on a large display in front of the user, creating a third-person point of view. The scene is reconstructed using a depth sensor, allowing participants to occupy the same digital 3D space as computer-generated characters or objects.

The mantra for the project, according to Mahler, for you to “hear a character coming, see them entering the space and feel them sit next you.” He says a “multi-sensory approach” would be achieved through the various technology included in the Magic Bench setup.

An example of this is shown in the video, with a snoring elephant sitting on the bench beside a man. To enhance this scene, there are haptic actuators underneath the bench which determine where a user is situated on the bench. These sensors then create vibrations that run through the bench to where the user is seated to simulate the feeling of a deep, reverberating snore. This sensation, along with the sounds emitted from the screen, are intended to add to the immersion. Another example would be the elephant walking up with a small digital ball in its hand that the man could then grab and rotate through mixed reality.

While the demonstrations only feature generic cartoon animals, it’s easy to imagine Disney eventually incorporating some of its own popular animated characters into Magic Bench, such as Mickey Mouse or Buzz Lightyear.

There’s no word as of yet on commercial availability or pricing, although researchers will present their paper on the project at the Siggraph 2017 computer graphics and interactive tech conference next week in Los Angeles.

Via: Variety