By Daniel Dunaief
Throw a giant, twisted multi-colored ball of yarn on the floor, each strand of which contains several different colored parts. Now, imagine that the yarn, instead of being easy to grasp, has small, thin, short intertwined strings. It would be somewhere between difficult and impossible to tease apart each string.
Instead of holding the strings and looking at each one, you might want to construct a computer program that sorted through the pile.
That’s what Partha Mitra, a professor at Cold Spring Harbor Laboratory, is doing, although he has constructed an artificial intelligence program to look for different parts of neurons, such as axons, dendrites and soma, in high resolution images.
Working with two dimensional images which form a three dimensional stock, he and a team of scientists have performed a process called semantic segmentation, in which they delineated all the different neuronal compartments in an image.
Scientists who design machine learning programs generally take two approaches: they either train the machine to learn from data or they tailor them based on prior knowledge. “There is a larger debate going on in the machine learning community,” Mitra said.
His effort attempts to take this puzzle to the next step, which hybridizes the earlier efforts, attempting to learn from the data with some prior knowledge structure built in. “We are moving away from the purely data driven” approach, he explained.
Mitra and his colleagues recently published a paper about their artificial intelligence-driven neuroanatomy work in the journal Nature Machine Intelligence.
For postmortem human brains, one challenge is that few whole-brain light microscopic data sets exist. For those that do exist, the amount of data is large enough to tax available resources.
Indeed, the total amount of storage to study one brain at light microscope resolution is one petabyte of data, which amounts to a million megapixel images.
“We need an automated method,” Mitra said. “We are on the threshold of where we are getting data a cellular resolution of the human brain. You need these techniques” for that discovery. Researchers are on the verge of getting more whole-brain data sets more routinely.
Mitra is interested in the meso-scale architecture, or the way groups of neurons are laid out in the brain. This is the scale at which species-typical structures are visible. Individual cells would show strong variation from one individual to another. At the mesoscale, however, researchers expect the same architecture in brains of different neurotypical individuals of the same species.
Trained as a physicist, Mitra likes the concreteness of the data and the fact that neuroanatomical structure is not as contingent on subtle experimental protocol differences.
He said behavioral and neural activity measurements can depend on how researchers set up their study and appreciates the way anatomy provides physical and architectural maps of brain cells.
The amount of data neuroanatomists have collected exceeds the ability of these specialists to interpret it, in part because of the reduction in cost of storing the information. In 1989, a human brain worth of light microscope data would have cost approximately the entire budget for the National Institutes of Health based on the expense of hard disk storage at the time. Today, Mitra can buy that much data storage every year with a small fraction of his NIH grant.
“There has been a very big change in our ability to store and digitize data,” he said. “What we don’t have is a million neuroanatomists looking at this. The data has exploded in a systematic way. We can’t [interpret and understand] it unaided by the computer.”
Mitra described the work as a “small technical piece of a larger enterprise,” as the group tries to address whether it’s possible to automate what a neuroanatomist does. Through this work, he hopes computers might discover common principals of the anatomy and construction of neurons in the brain.
While the algorithms and artificial intelligence will aid in the process, Mitra doesn’t expect the research to lead to a fully automated process. Rather, this work has the potential to accelerate the process of studying neuroanatomy.
Down the road, this kind of understanding could enable researchers and ultimately health care professionals to compare the architecture and circuitry of brains from people with various diseases or conditions with those of people who aren’t battling any neurological or cognitive issues.
“There’s real potential to looking at” the brains of people who have various challenges, Mitra said.
The paper in Nature Machine Intelligence reflected a couple of years of work that Mitra and others did in parallel with other research pursuits.
A resident of Midtown, Mitra, his wife Tatiana and their seven-year-old daughter have done considerable walking around the city during the pandemic.
The couple created a virtual exhibit for the New York Hall of Science in the Children’s Science Museum in which they described amazing brains. A figurative sculptor, Tatiana provided the artwork for the exhibition.
Mitra, who has been at Cold Spring Harbor Laboratory since 2003, said neuroanatomy has become increasingly popular over the last several years. He would like to enhance the ability of the artificial intelligence program in this field.
“I would like to eliminate the human proofreading,” he said. “We are still actively working on the methodology.”
Using topological methods, Mitra has also traced single neurons. He has published that work through a preprint in bioRxiv.
