Facebook Reality Labs (FRL), a research unit responsible AR and VR headsets, is now looking into technology that will let users ‘zoom in’ on and enhance sounds in real-life and dim out loud background noises that might cause an issue in clear conversations.
FRL, led by Ravish Mehra, calls the tech “perceptual superpowers.” Currently, the technology is only a prototype in-ear monitor and is paired with an off-the-shelf eye-movement tracking device that uses eye movement to detect what the user takes an interest in before turning up the volume.
This technology seems crazy, but Mehra and the team want to make AR glasses with the ability to integrate audio capabilities with the visual environment on a single device. Facebook could implement this technology with LiveMaps to create an augmented reality map of objects and sounds that surround the user.
Furthermore, this technology could also help users suffering from hearing loss, and has welcomed hearing scientist Thomas Lunner to look into this path.
FRL also revealed what the company is calling ‘audio presence,’ which lies at the core of Facebook’s AR and VR projects. Facebook’s goal is to essentially let a user feel like they’re in the same room as someone they hear virtually.
However, doing this is difficult. One way is to implement a head-related transfer function (HRTF), which lets the user digitally represent an individual’s personal experience of hearing. But HRTF is currently too complex, so FRL is looking into developing an algorithm that’ll approximate HRTF from a photograph of the user’s ear.
This feature could also make VR headsets like Facebook’s Oculus more hyper-realistic, which will fool users’ brains into believing that the sounds they hear in their headsets are coming from the room they’re sitting in.
Reportedly, this technology is still in its early stages, but this project may “redefine human hearing.”
Source: ZDNet
