It's long been known that blind people are able to compensate for their loss of sight by using other senses, relying on sound and touch to help them “see” the world. Neuroimaging studies have backed this up, showing that in blind people brain regions devoted to sight become rewired to process touch and sound as visual information.
Now, in the age of Google Glass, smartphones and self-driving cars, new technology offers ever more advanced ways of substituting one sensory experience for another. These exciting new devices can restore sight to the blind in ways never before thought possible.
Seeing with the Ears
One approach is to use sound as a stand-in for vision. In a study published in Current Biology, neuroscientists at the Hebrew University of Jerusalem used a “sensory substitution device” dubbed "the vOICe" (Oh, I See!) to enable congenitally blind patients to see using sound.
The device translates visual images into brief bursts of music, which the participants then learn to decode. Over a series of training sessions they learn, for example, that a short, loud synthesizer sound signifies a vertical line, while a longer burst equates to a horizontal one. Ascending and descending tones reflect the corresponding directions, and pitch and volume relay details about elevation and brightness. Layering these sound qualities and playing several in sequence (each burst lasts about one second) thus gradually builds an image as simple as a basic shape or as complex as a landscape.
The concept has tried and true analogs in the animal world, says Dr. Amir Amedi, the lead researcher on the study.
“The idea is to replace information from a missing sense by using input from a different sense. It’s just like bats and dolphins use sounds and echolocation to ‘see’ using their ears.”
Starting with sounds for basic shapes and lines, the participants in the recent study worked their way up to being able to identify bodily forms and positions – “seeing” them in their mind even though they had never before caught sight of a human form.
Remarkably, after approximately 70 hours of training, the blind participants were able to convert the sounds into visual activity in their brains, tapping into the same areas as control subjects who had seen the target images. This included a specific part of the visual cortex – the extrastriate body area (EBA) – that is specially activated when perceiving human shapes and positions. These participants had never before been able to perceive an entire human body. They could grab a hand, touch a cheek, even wrap someone in a bear hug, but they didn’t know what a person looked like squatting down, standing on one leg, or pumping their fists in the air.
Yet despite this, these individuals were able to activate a very specific area of the visual cortex just by hearing certain sounds, correctly identifying these positions. According to Dr. Amedi, the brain is able to transfer the initial activation in the auditory cortex into the “correct” visual area of the brain. “What is going on is neuronal recycling, in the sense that the same neurons [are] doing the same task, but now with a different sensory modality.”
Technological advancements are also allowing blind individuals to do something many never dreamed possible: driving a car. Research being done at the Robotics and Mechanisms Laboratory at Virginia Tech, led by Dr. Dennis Hong, has led to a prototype car for blind drivers.
To accomplish this, Hong and his team tapped into the visual system through touch, rather than sound. Employing technology used in the self-driving car, Hong and his team modified a computer-driven vehicle to allow drivers to pilot it solely using touch, relying on instructional feedback transmitted through sensors in the car.
Lasers in the front of the car serve as the automobile’s eyes, collecting information about obstacles and the boundaries of the road. Specialized gloves then relay this information to the driver to help them steer, gradually vibrating the fingertips on either hand in the direction the car should be turned. A vibrating chair provides guidance on optimal speed: the placement and intensity of the vibrations tell the driver to speed up, slow down or come to an emergency stop.
Finally, air puffs coming out of a tablet-like device located next to the driver create a map on their palms and fingertips to help them navigate the road ahead. This allows the driver to make advanced decisions, giving them greater independence over the automated feedback from the car.
Although he’s gotten flack for the project – blind drivers making some people nervous for obvious reasons – Dr. Hong has demonstrated that the car is safe and effective, and it’s been successfully taken out for several flawless test drives by blind individuals. The National Federation of the Blind has also lauded the project for enabling the drivers to be actively engaged in the process, rather than passive passengers being shuttled around on autopilot.
Dr. Hong and his team are still refining the process, improving both the car’s sensory input and tactile output systems, and at the moment the car has only been tested on a closed track, but the ultimate ambition is to one day have it out on the open road.
Mountain-Climbing with the Tongue
A more unusual part of the body has also been recruited to help blind individuals see through touch: the tongue. Your mouth is a highly sensitive area with a disproportionate number of neurons dedicated to it; the homunculus, the carnival-like depiction of our body as expressed in our brain, shows it as having greater sensory significance than the entire torso.
The inventors of BrainPort have put this lingual sensitivity to use, developing a device that converts visual stimuli into small electrical bursts on the tongue. An array of 400 electrodes on an area a little larger than a postage stamp sits on the tongue and receives input from a video camera hooked up to a set of snazzy sunglasses, ala Google Glass. The visual signal is processed through a small computer connected to the device, with the camera pixels corresponding to different electrodes in the array. This visual information is thus translated and spit out as electrical pulses on the tongue, varying in intensity, duration, location and number depending on the incoming signal. The researchers describe the pulses as feeling like bubbles or sparkling water on the tongue.
In perhaps the most incredible demonstration of this sensory substitution technology, one man has been using BrainPort to climb mountains. Extreme athlete Erik Weihenmayer, the only blind person to ever scale Mount Everest, uses the tactile information on his tongue to navigate mountainous terrain. With the device, he is able to judge the size, distance and depth of his next target on a climbing wall or cliff face, seeing with the help of the vibrations on his tongue.
Like the vOICe, BrainPort activates the visual cortex, bypassing the damaged sensory circuitry in the eye. These technological advancements raise the possibility not only for these temporary sensory stand-ins, but also for potential permanent treatment options, circumventing the malfunctioning perceptual organ and accessing sensory regions in the brain that had previously been thought to lie dormant.
For example, laser sensors implanted in the eye could replace bulky cameras, relaying visual information to a microchip that translates it into tactile or auditory sensations. Or electrodes inserted in the brain could be used to directly activate the visual cortex, negating the need to access these other sensory systems at all. For as neuroscientist Dr. Paul Bach-y-Rita, the pioneer of sensory substitution technology and BrainPort’s co-creator, once said, “we see with our brains, not with our eyes.”
