Seeing with the ear and the tongue
Sensory substitution turns images into sound or touch. With training, the brain learns to read that signal as a new kind of sight — and even activates the visual cortex while doing it.
Image: a session in the network · Beyond Sight
In the late 1960s, neuroscientist Paul Bach-y-Rita built a strange chair: a camera sent an image to a grid of vibrating points resting on a blind person's back. After hours of practice, users stopped feeling "vibrations on the skin" and began to perceive objects out there, in space. They recognized silhouettes, and even flinched when something seemed to loom closer. Sensory substitution was born: giving the brain the information of one sense through another.
His famous line: "we don't see with the eyes, we see with the brain." Decades later the idea lives on in devices anyone can try. The vOICe, created by physicist Peter Meijer, turns a camera image into a "soundscape": height becomes pitch, brightness becomes loudness, and the scene is scanned left to right. BrainPort brings the image to a plate of gentle electrodes on the tongue. In both cases, with training, the user stops decoding a code and starts to perceive.
The most striking part is what happens inside. Neuroimaging shows that, in trained blind people, these sound or touch signals activate the visual cortex. The brain cares less about "which door" the information comes through than about "what kind" of information it is. This is the metamodal brain: regions devoted to a task —locating, recognizing a shape— rather than to a single sense.
Sensory substitution doesn't cure blindness or replace the cane or guide dog. But it's tangible proof that perception is flexible and trainable — exactly the ground Beyond Sight wants to pool between schools, alongside echolocation, interoception and attention.
In brief
- Sensory substitution: giving the brain visual information through hearing or touch.
- Paul Bach-y-Rita pioneered it in the 1960s: "we see with the brain, not the eyes".
- The vOICe (image→sound) and BrainPort (image→tongue) put it within everyone's reach today.
- In trained users, these signals activate the visual cortex (metamodal brain).
The vOICe · Seeing with Sound
Peter Meijer's project on vision through sound.
Read on seeingwithsound.com →An explanatory summary written by Beyond Sight from public sources. Sensory substitution complements —does not replace— the cane, guide dog, or orientation and mobility.
← Back to Perception today