I can see what you can see

Enabling blind people to see again is a noble goal that science and medicine have been pursuing for centuries. Currently, 43 million people worldwide are blind and 295 million suffer from severe visual impairments. The magnitude of the wish to help them is matched by the magnitude of the related challenges. Initial approaches to emulating the miracle of sight by means of technology already exist. Artificial retinas have been implanted in patients with retinal damage for several years. A chip does the job of the defective photoreceptors and converts light signals into electrical impulses which are then transmitted to the brain via the patient’s natural optical nerve. Initial progress has been achieved in this area both in terms of image sharpness and use of the human eye’s optical system instead of camera glasses. However, the patients’ visual perception is still poor. Rough outlines are recognized in shades of gray, which is not surprising since the sense of sight is not only regarded as the most important one for humans but also the most complex one. The human eye can distinguish between 600,000 colors and absorbs more than ten million items of information per second that are transmitted to the brain where they’re filtered and processed.

Two years ago, an international research team presented a complete bionic eye with greater visual acuity than its human counterpart and the ability to see color spectrums such as infrared light. Its core element is a spherical artificial retina whose ultra-delicate nano-wire sensors are more sensitive to light than human photoreceptors. Thus, the technical foundation for a high-resolution ocular prosthesis has been laid. Yet a lot more work remains to be done to integrate it into the human visual system, particularly to connect it with the visual cortex. Apropos connectivity: The artificial eye could also send real-time signals to friends or post things on the internet. For all these visions to become true, a feasible solution must be found for permanently satisfying the high energy requirement of such an artificial visual system. Due to those hurdles, it’s possible that the bionic super eyes will initially be used in machines.

When conventional behind-the-ear devices are no longer adequate people with hearing loss receive cochlear implants (CI). These hearing prostheses electrically stimulate the acoustic nerve. Their prehistory dates to the 18th century when battery pioneer Alessandro Volta stuck two electrically charged metal rods into his ears and heard a sound similar to that of boiling soup. The first implant in 1957 substituting the functions of the pea-sized cochlea by converting sound into electrical impulses via electrodes was followed by numerous evolutions culminating in today’s CI generation. However, there’s a problem that’s common to all those hearing aids: The electric current extensively spreads in the saline solution of the cochlea, stimulating a large number of cells that are responsible for various frequencies. “That makes Mozart sound as if a pianist is hitting the keys with both fists,” says Marcus Jeschke from Göttingen University, where hearing researchers have been working on a CI version using light instead of electric current. It has the advantage that light can be focused more effectively. The method involves a genetic modification of the cochlear nerve cells, causing them to respond to precise light signals instead of to mushy sounds. As a result, people with hearing loss are ­supposed to be able to disentangle even a confusing mishmash of voices and recognize subtle nuances such as irony.

Feel the squeeze

People in Ancient Egypt already used effective walking aids 3,000 years ago. Today, thanks to sensors and tiny electric motors, high-tech prosthetic devices enable complex movements. However, there’s one thing people still don’t have after amputations: a sense of touch. Researchers are now raising hopes of enabling tactile sensation especially for artificial hands. Scientists at Stanford University in California have developed a sensitive rubber skin with electrically conductive mini-tubes as sensors. The stronger the squeeze (of the hand) the more tubes contact each other and the more current is transmitted to the nervous system by the artificial sense of touch. A UK research team at the University of Bristol has developed artificial fingertips produced by 3D printing. The printed papillae mimicking the dermal papillae underneath the outer layer of human tactile skin can “feel” shapes and transmit this information via artificial neural signals. Such an artificial yet sensitive tactile sensation could help people with nerve damage as well as open up new opportunities in robotics, another area to which research is dedicated around the globe. For instance, researchers at the Max-Planck Institute for Intelligent Systems in Stuttgart have developed a thumb-like sensor that processes light patterns created by pressure-dependent surface deformation and transmits them to neural networks. Although quite a bit of research work is yet to be done robots may soon have something that used to be a human privilege: a delicate sense of touch.

Artificial noses can already sniff out explosives, for instance during security checks, or identify chemicals in laboratories. Researchers are even working on “super noses” that can smell diseases such as dementia, cancer or covid. But can an artificial sense of smell also help people suffering from anosmia, i.e., a partial or total loss of the ability to smell? Eric Holbrook at Harvard Medical School in Boston wants to eliminate such impairments. He’s already succeeded in manipulating olfactory nerves with electrical impulses so that a study participant’s brain structure known as the olfactory bulb, which is responsible for the sense of smell, detected an onion-like odor that didn’t really exist. However, a few more issues must be resolved before an artificial sense of smell will be able to substitute the natural human sense: for instance, the precise positioning of the electrodes on the olfactory bulb and sensitive sensors that can absorb, analyze and convert into electrical impulses a wide range of odors. By comparison, restoring an impaired sense of taste is a lot more difficult because, from a scientific perspective, it involves a complex interaction of taste, odor, tactile and temperature perception in the oral cavity.

Giving a voice to those who have lost it

Although people who have lost their ability to speak can already be assisted in various ways today, such artificial verbal communication does sound somewhat clumsy. The best-known example is the world-famous physicist Stephen Hawking, who died in 2018 and during the last 20 years of his life “talked” by using a computer interface, which sounded rather robot-like. Also, entering words on a PC keyboard makes speech dialog difficult.

Scientists of various disciplines are working on convenient ways to provide those who’ve lost it with an authentic voice again, for instance at Tsinghua University in China. The research team there has recently developed flexible, stretchable sensors with carbon nanofiber membranes placed on the face where they capture movements that typically occur during the formation of phonetic sounds. The signals are transmitted to a microcomputer and an AI unit converts them into audio signals for the loudspeaker. The sensors also capture the person’s entire facial expression, temperature fluctuations and pulse, and can even integrate emotions in the voice output. Initial tests showed promising outcomes but there’s still a long way to go before the technology is fit for field use.

Researchers at the University of California are still in the early stages of their development project as well. They’re working on tapping electrical impulses that the brain transmits to the articulatory system for speech output into an artificial voice. The brain formulates, the technology articulates. Using this method, initial study participants managed to audibly produce simple test sentences such as “Tina Turner is a pop singer.” Joseph Makin from the University of California says, “We are not there yet but we think this could be the basis of a speech prosthesis.”