Sensory substitution devices (SSDs) use the auditory or tactile senses to help the visually impaired perceive the world around them. In a recent study, researchers attempted to teach people to move quickly and accurately, using a SSD called EyeMusic.

Ideally, SSDs would be able to help visual impaired people navigate their surroundings in highly specific and perhaps sensitive ways, like shaking a friend’s hand, or picking up a cup of coffee.

The device was developed by a team in Israel at the Hebrew University of Jerusalem. In a study published in Restorative Neurology and Neuroscience, EyeMusic was easy to teach and training took as little as thirty minutes. The SSD uses music to identify where things are in relation to a person, and could potentially be used for rehabilitative purposes.

EyeMusic works by using high-pitched musical notes to identify objects at a high vertical location, low-pitched notes are used to identify objects at low vertical locations. The image in front of the person is continuously scanned horizontally, with a noise prompt indicating that a scan has begun. The horizontal location of the perceived object is indicated by the length of time between the start of the scan and the following notes; the further right an object is, the later in time the next note will play. The brighter the object is, the more loudly the note plays.

EyeMusic uses five different musical instruments to indicate colors. Red objects are indicated with a reggae organ; yellow is a violin; green, a synthesized reed; blue, a trumpet; and white, vocals. Black is represented with a silence. Researchers say that the notes were specifically chosen in order to be pleasing to the ear.

The study was conducted on 18 people who did not have any visual impairment, but were blindfolded, or whose hand was unable to be seen. They used the device on a tablet with a stylus.

While this device is quite exciting and an important first step for this type of technology, it remains to be seen if the SSD could be used in a room to find something specific, with its many differently-colored objects, as opposed to a tablet screen.