Wearable Device That Allows The Visually Impaired To Read

With a new phone, tablet or a laptop launching every other day worldwide, we often wonder whether we are actually moving forward in the terms of technology or are we just standstill. But then come certain advances that blow our mind, remove the illusion of being ‘standstill’ and make us proud. The researchers in MIT Media Lab have built one such device. These researchers have developed prototype of a device that can be mounted on the fingers, has a built-in camera and converts text into audio for users that cannot see.

In short, this is how it works. The user has to move their fingers over the text, the camera reads the word that the user is pointing to and converts it into audio which the user can hear. However, all this is easier said than done. The key essence of the working is the algorithm that converts camera’s feed to audio.


When the user keep their finger at a start of a new line, the algorithm will try and make a number of guesses about the baseline (the point of reference) of letters. There are two reasons that these guesses will differ (and so will the baseline) for every user: one, there are many letters whose bottoms descend below the baseline and two, the orientation or the manner in which each user will keep their finger on the text will differ from person to person. Thus the algorithm will tend to make a number of guesses and hence the median value of the cluster of guesses will be selected by the algorithm to convert into audio. Now that a baseline has been established, the computational burden on the algorithm will reduce as the user will move their finger to the right.

As the reader will move further along the line, the camera will try to capture each and every word that they are reading. When the camera will realize that a particular word is near to centre of camera’s field of view, it will crop that word out of the image and provide it to the algorithm. The algorithm will realign the word and by thus understanding the distortion and the angle of reader’s finger, the algorithm will improve its baseline and the guesses of the next word that the user will read.

The elephant in the room of course is that since the user is blind, how will they know that their finger is actually on the word and that now they should start moving towards the right. The researchers have provided two solutions for this conundrum. One solution is that the device on the reader’s fingers will have two haptic sensors, one on the top of the finger and the other below it. The vibration of the respective motors will indicate whether the reader should move their hand up or down to reach the actual line of text.


The second solution is in the form of audio. The volume of musical tone that will play will increase if the reader’s finger moves away from the text. If the finger moves towards the text, the volume will decrease. The advantage of the musical solution is that it has a smaller light-weight sensor than the haptic sensors used in solution one.

MIT's Undergraduate Research Opportunities Program are working on building a software of the program for the Android phones. While working on the device, the researchers realized that besides the visually impaired, the device will also be helpful for other specially-abled people who have problem in reading. Needless to say, if this device ever makes its way into the market, it will be a big boon for many.

Source: #-Link-Snipped-#
Via: #-Link-Snipped-#