Ford turns to neuroscience to make driverless cars

Ford Motor Co. could make its next autonomous car breakthrough with the help of a New York neuroscientist who’s attempting to restore sight to the blind.

The Dearborn automaker last month announced a licensing agreement with Nirenberg Neuroscience to help it bring a fully autonomous car to market in 2021. The New York company was founded by Dr. Sheila Nirenberg, who four years ago cracked the code to how the brain receives and processes visual images through electrical impulses.

Humans view images — whether it’s a baby’s face or the words on this page — through photoreceptors in the retina that convert light into a form of code, which is then sent to output cells that transmit that code through specific electrical impulses to the brain. In certain degenerative eye diseases, the photoreceptors in the retina die but the output cells remain. Nirenberg found a way to send signals to the output cells through a device that acts as an artificial photoreceptor.

While she never engineered cars before, she has been developing a computer-vision platform based on this code. In the last few months she’s worked exclusively with Ford to direct it toward car applications. Ford hopes her findings can allow its vehicles to see and understand the road ahead by working with a wide range of cameras, radar and LiDAR (light detection and ranging) sensors.

“You can find innovations in other industries, including the medical industries,” said Dr. Ken Washington, Ford’s vice president of research and advanced engineering. “If you look outside of your own domain, you might find some technical gems you can apply to your problems.”

Nirenberg, who has been working at Cornell University for the past decade, said she’s always been drawn to the field of neuroscience.

“It’s the brain. It’s inherently intriguing,” she said. “It’s everything we are.”

Nirenberg was able to translate the eye’s electrical impulses into a sort of mathematical code that determines what the brain sees. After finding the code, she had to find a way to get that code from the retina’s output cells to the brain.

After talking one night with a colleague, Nirenberg realized she could create an “encoder device” — worn like goggles — that replicates what the retina does by sending flashes of light into the eye. Once the eye is injected with channelrhodopsin — a gene found in algae — the genetic material interacts with those light pulses and sends the proper electrical impulses to the brain, replicating what a normal retina would do.

“That was the moment I jumped out of my chair,” she said. “I haven’t sat down since. It’s exciting.”

Nirenberg said she’s been doing safety studies with the technology and is hoping to get permission to do the first clinical trial within the next year.

“What Sheila Nirenberg and her team are doing just holds incredible promise for society, much less autonomous vehicles,” said Raj Nair, Ford’s executive vice president of product development and chief technical officer, when the licensing agreement was announced. “If that device can restore sight, imagine what it could do in an autonomous vehicle.”

Nirenberg adopted the technology for small robots, but didn’t realize the potential her discovery could have in the automotive field until one of her investors introduced her to Ford President and CEO Mark Fields and other executives.

“If we can make mobile robots, one can do self-driving vehicles of a bigger sort,” she said. “A car is a giant robot in a sense. It’s kind of a natural fit.”

Washington said it’s still too early to tell exactly how Nirenberg’s technology will work in Ford’s cars of the future, but said it will likely help enhance what the company’s sensors already see. Ford uses LiDAR sensors from Velodyne, and is also working with 3-D mapmaker Civil Maps and machine-learning company SAIPS.

He described Nirenberg’s technology as a “filtering mechanism” that could potentially show the car only what it needs to see, helping it make decisions like where to turn, how fast to drive and when to brake more efficiently.

“Humans see the world around us and we ignore the stuff that doesn’t matter,” Washington said. “Dr. Nirenberg’s technology holds the promise to apply that kind of filtering to our machine-learning algorithms.”

Washington said Nirenberg has given talks on her work to Ford’s employees and has spent the past few months teaching them about her findings. Their licensing agreement gives Ford exclusive access to her work in automotive applications.

“We’re delighted,” Washington said. “We think this could be game-changing and provide a capability that will really be pretty important for our vehicles.”

mmartinez@detroitnews.com

(313) 222-2401

Twitter: @MikeMartinez_DN