By Daniel Dunaief

Most behaviors involve a combination of cues and reactions. That’s as true for humans awaiting a response to a gesture like buying flowers as it is for a male fruit fly watching for visual cues from a female during courtship. 

The process is often a combination of behaviors and signals, which the visual system often processes as a way of determining the next move in a courtship ritual.

At Cold Spring Harbor Laboratory, Assistant Professor Benjamin Cowley recently published research in the prestigious journal Nature in which he used a so-called deep neural network to mirror the neurons involved in a male fly’s vision as it interacts with a potential female mate.

Working with a deep neural network that reflects the fly’s nerve cells, Cowley created a knockout training process, in which he altered one set of neurons in the model at a time and determined their effect on the model and, with partners who conduct experiments with flies, on the flies themselves.

Cowley worked closely with his former colleagues at the Princeton Neuroscience Institute, including Professor Jonathan Pillow and Professor Mala Murthy. His collaborators genetically silenced a fruit fly’s neuron type, observing the changes in behavior. Cowley, meanwhile, trained his deep neural network on this silenced behavior while also “knocking out” model neurons, teaching the model by perturbing it in a similar way to the changes in the fruit fly circuitry.

This approach proved effective, enhancing the ability of these models not only to understand the wiring involved in processing visual information and translating that into behavior, but also to provide potential clues in future experiments about similar cellular dysfunction that could be involved in visual problems for humans.

What researchers can infer about the human visual system is limited because it has hundreds of millions of neurons. The field has taken decades to build artificial visual systems that recognize objects in images. The systems are complex, containing millions of parameters that make them as difficult to explain as the brain itself.

The fly visual system, which is the dominant focus of the fly’s brain, occupying about 70 percent of its 130,000 neurons, provides a model system that could reveal details about how these systems work. By comparison, the human retina has 100 million neurons.

“To build a better artificial visual system, we need to know the underlying mechanisms,” which could start with the fly, Cowley said. “That’s why the fruit fly is so amenable.”

Researchers need to know the step-by-step computations going from an image to neural response and, eventually, behavior. They can use these same computations in the artificial visual system.

‘A suite of tools’

The fly’s visual system is still robust and capable, contributing to a range of behaviors from courtship to aggression to foraging for food and navigating on a surface or through the air as it flies.

The fly “gives us a whole suite of tools we can use to dissect these circuits,” Cowley said.

The fly visual system looks similar to what the human eye has, albeit through fewer neurons and circuits. The fruit fly visual system has strong similarities to the early processing of the human visual system, from the human eye to the thalamus, before it reaches the visual cortex in the occipital lobe.

Interpreting the visual system for the fly will “help us in understanding disorders and diseases in human visual systems,” Cowley said. “Blindness, for the most part, occurs in the retina.”

Blindness may have many causes; a large part of them affect the retina and optic nerve. This could include macular degeneration, cataracts, diabetic retinopathy and glaucoma.

In its own right, understanding the way the visual processing system works in the fly could also prove beneficial in reacting to the threat of invasive species like mosquitoes, which pass along diseases such as malaria to humans.

Visual channels

Anatomists had mapped the fly’s 50 visual channels, called optical glomeruli. In the past decade, researchers have started to record from them. Except in limited cases, such as for escape reflex behaviors, it was unknown what each channel encoded.

Cowley started the research while a postdoctoral researcher at Princeton Neuroscience Institute in Jonathan Pillow’s lab and finished the work while he was starting his own lab at CSHL. Mala Murthy’s lab, who is also at Princeton, performed the silencing experiments on fruit flies, while Cowley modeled the data.

Through hundreds of interactions between the flies in which some part of the fly’s visual system was silenced, Cowley created a model that predicted neuronal response and the behavior of the fly.

The deep neural network model he used deploys a new, flexible algorithm that can learn its rules based on data. This approach can be particularly helpful in situations when researchers have the tools to perturb the system, but they can’t recover or observe every working part.

In some of the experiments, the males became super courters, continuing to engage in courtship activities for 30 minutes, which, given that the fly lives only three weeks, is akin to a date that lasts 25 days.

It is unclear why these flies become super courters. The scientists speculate that silencing a neuron type may keep the male from being distracted by other visual features.

In the experimental part of the experiments, the researchers, including Dr. Adam Calhoun and Nivedita Rangarajan, who both work in Murthy’s lab, tried to control for as many variables as possible, keeping the temperature at 72 degrees throughout the experiment.

“These flies live in nature, they are encountering so much more” than another fly for potential courtship, said Cowley, including the search for food and water.

This research addressed one small part of a behavioral repertoire that reveals details about the way the fly’s visual system works.

A resident of Huntington, Cowley grew up in West Virginia and completed his undergraduate work and PhD at Carnegie Mellon in Pittsburgh.

An avid chess player, which is a field that has included artificial intelligence, Cowley, who spent much of his life in a city, appreciates having a backyard. He has learned to do some landscaping and gardening.

Cowley had been interested in robotics in college, until he listened to some lectures about neuroscience.

As for the next steps in his work, Cowley hopes to add more complex information to his computational system, suppressing combinations of cells to gather a more complete understanding of a complex system in action.