For years, neuroscientists have been monitoring the brain activity of mice as they looked at a wide range of images — including the film-noir classic “Touch of Evil” — in hopes of discovering deep insights about the workings of the visual system. Now they’ve come upon a plot twist worthy of director Orson Welles himself.
The latest results, reported today in the journal Nature Neuroscience by researchers at Seattle’s Allen Institute for Brain Science and at the University of Washington, suggest that more than 90% of the neurons in the visual cortex don’t work the way scientists thought.
“We thought that there are simple principles according to which these neurons process visual information, and those principles are all in the textbooks,” Christof Koch, the brain institute’s chief scientist and president, said in a news release. ”But now that we can survey tens of thousands of cells at once, we get a more subtle — and much more complicated picture.”
Not that there’s anything wrong with that.
“To me, that’s the business. In some sense, that’s the exciting thing,” Michael Buice, an associate investigator at the Allen Institute and one of the study’s lead authors, told GeekWire. “We’re in a more interesting place than we thought.”
For more than a half-century, neuroscientists have assumed that different sets of neurons responded to specific patterns — for example, alternating bars of light and dark, or objects moving from left to right — and that the brain stitched together all those inputs to produce a mental picture of the world. Two Harvard researchers won a Nobel Prize in 1981 for describing the process, based on studies with cats.
Over the past few years, the mouse experiments at the Allen Institute have been documenting the process as well. Researchers set up a screen in front of the mice, and showed them pictures of abstract patterns, photographs of natural scenes — and movies. The first three minutes of “Touch of Evil,” Welles’ 1958 film, was one of the research team’s favorites because it’s one continuous scene with no cuts to confuse the mice.
Since 2016, the institute has increased its database for activity in the mouse visual cortex from 18,000 neurons to nearly 60,000 neurons. As a result, the institute’s researchers have gotten a more complete picture of what cells in the visual cortex are doing while the mice are processing the pictures they’re seeing.
They found that the overwhelming majority of cells didn’t match well very with the textbook model as seen in cats and monkeys. “The number of cells we get that work in that ‘very well’ sense … depending on how you measure it, is between 2 and 10 percent of the data set — closer to 2,” Buice said.
Roughly 60% of the neurons showed a reliable response that could be associated with specific stimuli, but the response was more specialized than the textbook model would have predicted. The remaining 30% or so showed some activity, but didn’t light up reliably in response to any of the stimuli in the experiment.
The Allen Institute’s Saskia de Vries, another lead author of the study, said the fact that so many neurons are working in mysterious ways doesn’t mean the previous brain-cell studies were wrong. “It’s just that those cells turn out to be a very small fraction of all the neurons in the cortex,” she said.
Further experiments will be required to try to figure out why the data from mice don’t match the data from cats and monkeys, and decide whether the textbook model will have to be dramatically revised.
Buice suspects part of the reason for the difference may have to do with the way the eyes of different species are built. The eyes of monkeys, humans and other primates have a central retinal area known as the fovea that’s responsible for fine-focus vision. Cat retinas have a visual streak that appears to serve a similar function. But mouse retinas don’t have as much specialization — which means mice basically have 20/2000 vision, much worse than the 20/20 norm for humans.
Mouse brains are also more limited in their processing capacity, which may require their visual system to work in a markedly different way. “It’s been programmed, if you will, by evolutionary development to do a very different thing from what our vision system does,” Buice said.
Buice said such differences could explain what’s going on with the 30% of neurons that pose a complete mystery. Perhaps those neurons haven’t been triggered by the kinds of imagery they’re programmed to respond to. “The thing we’re trying to do to fix this is to do another set of experiments where we just show as many features as we can,” he said.
The results could have implications for brains that aren’t made of flesh and blood, Figuring out the basics of biological vision systems, and how they differ in mice and humans, could help scientists build better machine-based vision systems as well. “We can see what’s common, in the sense of having a universal neural computation, versus what is task-specific,” Buice explained.
Will future AI agents see the world as a mouse sees it, as a human sees it — or as something totally alien? If only Orson Welles were around to direct that movie…
Saskia de Vries, Jerome Lecoq and Michael Buice are lead authors of the study published by Nature Neuroscience, “A Large-Scale Standardized Physiological Survey Reveals Functional Organization of the Mouse Visual Cortex.” Christof Koch and R. Clay Reid are the senior authors. Sixty-seven other researchers are listed as co-authors. A preprint version of the paper is freely available via BioRxiv.
