How Microsoft’s ‘Seeing AI’ app is helping thousands of the visually-impaired

Artificial intelligence can help out with a lot of things — booking tickets to an event, setting reminders in a calendar and checking the weather, just to name a few. That said, while this is all useful, it’s not necessarily essential to carrying out one’s everyday life.

However, for some, AI is capable of offering truly life-changing possibilities, depending on the circumstance. Such is the case with Microsoft’s ‘Seeing AI’ app, which leverages artificial intelligence to assist the visually impaired with a variety of different tasks. In essence, Seeing AI scans the user’s surroundings to recognize text, objects, product barcodes and even people to provide various audio descriptors.

With Seeing AI, users will be able to do anything from scan the barcode of a food item, have written text read out to them, recognize the age, gender and emotions of nearby people and more.

Microsoft designed Seeing AI to help the nearly 253 million people around the world who live with some sort of visual impairment — 36 million of whom have completely lost their sight.

Since launching in July, the app has been downloaded over 100,000 times, and has completed more than three million tasks. To continue to help the visually-impaired, Microsoft has rolled out version 2.0 of Seeing AI, which adds new features to the app.

First off, Seeing AI can now recognize U.S. Dollars, Canadian Dollars, Euros and British Pounds, with support for additional currencies to be added at a later date.

The app now also features an audible light detector that scans the area for bright lights and emits an increasingly loud noise the closer the user gets to the source. The idea is to prevent someone from touching an active lightbulb or any other objects that may be too hot. Moreover, Seeing AI will be able to distinguish between colours to relay to the user accordingly, making tasks like picking the right clothing much more easier.

According to Anirudh Koul, a senior data scientist at Microsoft and original creator of the Seeing AI app, the response from visually-impaired users has been overwhelmingly positive. “We’re truly humbled by the feedback,” he said, going over a list of the praiseful tweets and Facebook posts he’s seen from those who have used the app.

“For the first time since going total[ly blind] 15 years ago I read my own menu in a restaurant today using my iPhone and Seeing AI app,” wrote Facebook user Karen Parsegian. “Game changer. Have not stopped smiling since.”

“I just used #SeeingAI to find and Mark pages in books with beautiful pictures, I will never see, so I can show them to kids while I’m telling them stories the next four Sundays,” said Kim Rickmon on Twitter.

Other users said Seeing AI proved particularly useful when travelling to places that didn’t have braille. Koul also mentioned one visually-impaired salesperson who told him that the app helped him get a read on what his customers looked like and how happy they seemed with his services, allowing him to change his sales tactics accordingly.

Interestingly, though, despite having a large and ever-growing userbase, Seeing AI has a rather humble history. Koul said the app initially started out as a Microsoft hackathon project that was intended to offer an easy and cost-efficient way for the visually-impaired to get the assistance they need.

“For example, people would previously have had to buy a $1,300 barcode scanner; now, they can get that same kind of functionality with a free smartphone app,” said Koul.

Because Microsoft is constantly iterating on Seeing AI, Koul and the company’s accessibility team ask that users continue to offer as much feedback as possible.

This can be done by e-mailing seeingai@microsoft.com or by contacting the Disability Answer Desk. More information can be found here.

Seeing AI can be downloaded for free on iOS. There is currently no word on whether the app will make its way to Android.