New research shows that previous studies of mental illness using brain scans may not be reliable.

Andrew Brookes/Getty Images/Image Source

Researchers can see inside the human brain with the help of magnetic resonance scans. The technology can reveal damage from a stroke or areas that light up when we see a face.

Brain scans don't offer much insight into the underlying causes of mental health conditions like anxiety and depression.

A team reported in the March 16 issue of the journal Nature that studies need to include scans of thousands of brains, instead of the dozens typically used.

The author of the study says that bigger samples are better.

Paul Thompson, a neuroscientist at the University of Southern California who was not involved in the research, says that the field of genetics has already learned a lesson.

Twenty years ago you would hear someone say they had discovered a gene for criminality or for psychosis, or another group would say they had found another.

Thompson says geneticists fixed the problem by expanding their studies from hundreds to millions of people. It appears that neuroscientists are in the same situation, one that will require them to reexamine the results of many small studies.

An illuminating search for intelligence

A new paper on brain scans was written in order to understand how children develop cognitive abilities.

The data from the federal study that was scanning the brains of thousands of adolescents was to be used by the team.

We wanted to ask the question with the huge sample: How is cognitive ability represented in the brain?

Intelligence is linked to the thickness of the brain and the strength of connections between brain regions, according to previous research.

The team analyzed the brain scans from the federal study. They used 1,000 different scans to check their work.

We noticed that we could not replicate everything.

An area that seemed important in one set of scans may not be so important in the other. The results became more reliable when the sample size was increased to thousands of brains.

The team wondered if this was the case with other studies that looked at the entire brain for differences associated with complex problems.

They got brain scans from about 50,000 people and then used a computer to conduct lots of simulations. It took thousands of scans to get reliable results.

For years, smaller samples have been used to produce a stream of scientific papers on mental illness and behavioral disorders.

So far, that research hasn't really translated to tangibles for patients, and I think these results give us a clue as to why.

The perils of small samples

Small studies can only find brain features that produce large effects on mood, behavior, or mental abilities. It is easy to show that the hippocampus is damaged in Alzheimer's disease.

The differences in the brain that are associated with mental illness are more controversial. Some studies have shown that people with major depression have less activity in the brain. The correlation strength varies from study to study. There is no way to know how a person is feeling because they don't have a way to look at the activity in their frontal cortex.

publication bias is a problem with small studies.

If multiple groups are doing similar research using small samples, one of them will have a significant result.

A misleading finding can become the conventional wisdom when enough studies are published. Small studies are not necessarily wrong.

He says that even a tiny study could hold true.

The public should be wary of headlines that make assumptions about the findings of a small study.

A study with "aftershocks"

The news that human behavior studies may require thousands of scans is still being digest by many brain scientists.

It was similar to an earthquake in Los Angeles, according to UCLA's Thompson.

Thompson says the solution is to combine the scans from many small studies into one or more large databases and check the results.

Thompson helped create the ENIGMA Consortium to make this easier. The database has more than 50,000 scans. Scientists have been able to identify brain differences associated with schizophrenia.

There are huge differences in the brains of people with schizophrenia. There are changes in memory and vision systems.

Smaller studies may not be able to find the brain areas and connections associated with mental illnesses like depression.

Some of the studies are already under way.

More than 11,000 young people have been Enrolled in the National Institutes of Health study on adolescent brain development.

Terry Jernigan, a brain scientist at the University of California, San Diego and one of the study's principal investigators, says that the large size is an effort to address the problems found in smaller studies.

It is not enough for brain scans to include thousands of people, the studies must also be more diverse.

She says that you want to know what extent your observations are generalizable to all the groups in our society.