Are forensic experts biased? Scientist’s claims spark outrage

In February of 2021, cognitive psychologist Itiel Dror set off a fire in the forensics community. If the child was black and brought to the hospital by the mother's boyfriend, they would be more likely to be found dead than if they were white. The latest experiment by Dror suggests that forensic scientists are influenced by cognitive biases that can put innocent people in jail.

Dror, a researcher at University College London, has spent decades using real-world cases and data to show how experts in fields as diverse as hospital care and aviation can reverse themselves when presented with the same evidence in different contexts. His most public work has been in forensic science, a field that has a history of unscientific methods. In 2009, the National Research Council published a report that said forensic sciences are more based on tradition than on quantifiable science. Hundreds of studies and legal cases have shown flaws in forensic sciences.

The work of Dror forms a tissue. Most problems with forensics are not caused by bad apple technicians. They come from the same kind of subconscious bias that affects everyone's daily decisions.

In the span of a decade, cognitive bias went from being almost totally unheard of in forensics to common knowledge in the lab, according to a Duke University School of Law professor.

Dror travels the world testifying in trials, taking part in commissions, and offering training to police departments, forensic laboratories, judges, militaries, corporations, government agencies, and hospitals. His approach of shielding experts from information that could bias them has been adopted by national agencies.

The chief justice of the Michigan Supreme Court, who worked with Dror, says she doesn't know anyone else who is doing everything that Itiel is doing. It is critical to the future of the rule of law.

Dror's previous studies on bias in forensics caused grumbling, but nothing like the reaction to the paper of 2021. He used a survey to see if bias could affect decision-making. He concluded that the race of the dead person and their relation to the person who cared for them were not medical evidence.

Eighty-five of the country's most prominent pathologists demanded its removal. The National Association of Medical Examiners demanded that the employer of Dror stop his research. The editor of the Journal of Forensic Sciences wrote that he hadn't seen many arguments in the 65-year history of the journal. Dror got into a fight that threatened his career after challenging forensic experts for decades.

Dror appears to be a nice man, with salt-and-pepper hair and wire-rimmed glasses, but that impression disappears when he starts talking. He gains steam like a runaway train, detailing his latest study, making a quick detour to bring up an example, and then circling around to put a cap on his original point. He speaks with a mixture of accents from his upbringing in Israel, his graduate work in the United States, and his professional life in the United Kingdom.

There are two diamond studs in his left ear. As a child, Dror attended five elementary schools on three continents, but he didn't know the language very well. It was very difficult, but it gave me a lot of freedom of thought.

Dror disliked the discipline of school. He broke his back in the Israeli army at the age of 19 and things have changed ever since. He was in a body cast for 7 months and had ignored the books he had read in high school. He home-tested and got As in the courses he had failed. He studied philosophy at Tel Aviv University, took a year off to work on a kibbutz, and did a PhD in psychology at Harvard University.

One of his projects looked at how the U.S. Air Force pilots use mental imagery. The work caught the attention of David Charlton, a prominent British fingerprint examiner who had started to have doubts about his field.

I wondered if my eyes were the same from one day to the next. I wondered if it applies to fingerprints.

Fingerprints don’t lie. … But it’s also true that fingerprints don’t speak. It’s the human examiner who makes the judgment, and humans are fallible.

• Itiel Dror
• University College London

He had reason to be concerned. The scandal of a Scottish police officer who was charged with perjury after investigators claimed to find her thumbprint at a murder scene shook the United Kingdom. Two American experts testified that the thumbprint could not have been hers. The Americans had a scandal of their own in 2004, when the FBI arrested an American lawyer as a suspect in the Madrid train station bombing. The Portland, Oregon, resident with Taliban connections was provided legal defense by a man who had converted to Islam. The U.S. government agreed to a $2 million settlement when they found the real bomber.

The cases deepened his doubts about his objectivity. Dror suggested they do some research together. Five fingerprints experts who knew about the case did not see the fingerprints. Dror and Charlton sent each expert a pair of prints from one of the experts own previous cases, which they had personally verified, but told them the prints came from the FBI case.

Four of the five experts disagreed with their previous decision, three of them concluded the pair was a mismatch and one felt he needed more information. They were influenced by the passage of time.

The study was simple and elegant, says Peter Neufeld, co-founder of the Innocence Project.

In a follow-up study, Dror and Charlton gave six experts sets of prints they had previously examined, along with biasing information, that the suspect had either confessed or had an alibi. Four of the experts changed their findings.

Some of his colleagues thought he was trying to destroy the profession. There were angry letters to the professional journal. The chair of the Fingerprint Society wrote that any fingerprints that could be swayed by images or stories should seek employment at Disneyland.

He considered abandoning his career because he was so upset by the reactions. Let's do more research and see how we can improve things.

Some of the factors Dror looked at were innocuous. When police recover a print from a crime scene, they use an FBI computer database containing millions of fingerprints to find the most likely matches. Dror found that experts were likely to pick a match near the top of the list even after he had changed his mind.

Dror says that people would say that fingerprints don't speak. Humans are fallible and it is the human examiner who makes the judgement.

Dror and his colleagues are quick to point out that bias does not always equal prejudice. Studies show that black children are punished more often for the same behavior than white children. In forensic science, bias can subconsciously influence experts to incriminate a suspect.

What could happen if fingerprints were biased? Dror wanted to find out if there was a link between the two. When the authors of the National Research Council study criticized forensic sciences, they made an exception for DNA analysis, a method developed in the lab that was statistically verifiable and scientifically sound.

Human interpretation has come to be used in DNA analysis as it has gotten more sensitive and sophisticated. When investigators find a mixture of several people's DNA at a crime scene, it's up to the analyst to tease apart the contributors. It is a complicated and subtle process that can be influenced by context. Kerry Robinson was accused of taking part in a gang rape in Georgia in 2002. The state's case was based on the testimony of White, who admitted to being the main culprit and who had a vendetta against Robinson. The jury found him guilty after the state's two DNA experts found that Robinson's DNA could not be excluded from the mixture of DNA found at the crime scene.

Dror shared the data from the case with 17 DNA analysts who were unfamiliar with it. Only one agreed with Georgia's analysts and the other 16 either excluded Robinson's genes or said they couldn't come up with a result. The conclusion of Dror was that the gold standard of forensic science was subject to human bias. Robinson was not released by the state until 2020, when other exonerating information was submitted. Robinson had served 18 years of his sentence.

Dror and other researchers have found bias just about everywhere they've looked, in toxicologists, forensic anthropologists, and others who must make judgments about often ambiguous crime scene evidence. Dror and others have found that juries find compelling forensic evidence.

Saul Kassin, a psychologist at the John Jay College of Criminal Justice, says that many examiners feel like they are not human. Almost three-quarters of the examiners saw bias as a general problem, but just over half thought it was a concern in their own specialty, and only 26% thought bias would affect them personally.

The best way to fight bias is to shield experts from extraneous information. The analyst only sees the evidence relevant to their task in the process of Linear Sequential Unmasking. Authorities have endorsed the approach. The most powerful means of protecting against contextual bias is recommended by the United Kingdom's forensic science regulators.

John Riemen, the police force's lead biometrics specialist, says that after consultation with Dror, police in the Netherlands began to blind fingerprint examiners to details of a crime investigation that might influence their analysis. He says that the approach ensures that you are looking at fingerprints and not at your biases.

Dror was in hot water because of the attempt to win medical examiners over to this approach. He got a message from Daniel Atherton, a Pathologist at the University of Alabama, who wanted him to look at some data he had collected. A survey was sent to 713 pathologists across the country by Atherton, which showed two scenarios in which a toddler with a skull fracture and brain hemorrhage could have died in the emergency room. The child was white and brought in by the grandmother. The child was black and brought in by the mother's boyfriend. The manner of death was the subject of a survey.

Of the 133 people who answered the survey, 32 said the death was a homicide. A disproportionate number of those had received the scenario with the Black child and the boyfriend. The participants who read the black condition were five times more likely to conclude homicide than the participants who read the white condition.

Their decisions were affected by medically irrelevant contextual information, according to a paper published in the Journal of Forensic Sciences.

The paper included a survey of 10 years of Nevada death certificates showing an apparent correlation between Black deaths and findings of homicide versus accident.

He got more than he anticipated. The journal received angry letters from medical examiners. The study was derided as "rank pseudoscience" by one and "fatally flawed" by the other. Michael Peat, editor of the journal, said that the article had been peer reviewed before it was published and that it had been re-reviewed by a biostatistician.

The experimental design linked two unrelated variables, the race of the child and their relationship to the caretakers. When they had reason to suspect that the caretakers, not the race, was the reason, they were even more enraged by Dror's labeling of the scenarios. Statistics show that a boyfriend is more likely to harm a child in his care than a grandmother is.

The letter from the 85 practitioners said that introducing race appears to be an effort to label the survey responders and their colleagues as racist. Other factors could have played a role in the decisions, such as their level of experience, local crime statistics, and office policies, none of which Dror had considered.

Stephen Soumerai, an expert in research design at Harvard Medical School, agrees that linking a known risk factor for homicide to a nonwhite race is problematic. He says the survey of Nevada death certificates failed to investigate other possible explanations beyond race.

The two experimental groups would have been designated using neutral terms. He doesn't concede that the study is flawed. According to Dror, statistics show that a grandmother is more likely to harm a child than an unrelated care giver. It shouldn't affect how examiners diagnose individual cases.

There are too many cases where innocent caregivers are prosecuted for accidental child deaths because forensic pathologists make assumptions based on larger trends.

Antibias strategies are important to medical examiners because many of them work with the police. One solution is to have a laboratory's case manager safeguard details about an investigation and reveal them to a medical examiner only as needed to determine the manner of death.

William Oliver, a retired professor of pathology at East Carolina University, said that the approach represents aclueless understanding of how medical examiners work. Medical examiners need to gather all the information about the case that they can make a correct diagnosis, unlike other forensic examiners who only look at a particular type of evidence.

They determine the cause of death and the manner of death. If a dead man is found sitting in his car with the engine running, the garage door closed, and high levels of carbon monoxide in his blood, the autopsy would conclude that he died of carbon monoxide poisoning. Unless investigators found signs of suicide, such as a note, recent job loss or divorce, the manner of death would remain unclear.

Manner is not a scientific determination, and it is not meant to be. He says that the rates at which grandmothers and unrelated caretakers harm children are crucial to making that judgement.

There was acrimony around Dror's paper. The association received a complaint against the four pathologists who collaborated with Dror from the chief medical examiner for Milwaukee County, Wisconsin. The paper would expose every medical examiner to withering cross-examination at trials, he wrote.

Joye Carter, a forensic pathology consultant and co-author of the study, was named in the complaint.

The Legal Defense Fund came to the defense of the pathologists, and their complaint was dismissed in May of 2021. Carter was the first Black chief medical examiner in the United States and resigned from NAME.

Dror was attacked by NAME's leadership. James Gill and Mary Ann Sens, the NAME president and executive vice president, accused him of intentional ethics violations in a letter to the University College London. Dror says the letter could have ended in his dismissal. At one point he became so emotional that he had to leave the room to regain his composure, he argued that the study's nature would have biased the results. The board ruled that the allegation was incorrect.

The Minneapolis police officer who killed George Floyd was accused of bias in autopsies. The local medical examiner in Minnesota testified that the manner of death was homicides. The manner of Floyd's death was not determined by an expert hired by Chauvin's defense team.

Chauvin was found guilty, but Fowler's testimony upset other pathologists and physicians, who saw in his conclusions a pro- police bias. More than 400 people signed a petition demanding an investigation into all the death-in- police-custody cases during Fowler's 17 years in office. Dror was recruited by Frosh to design the study. Dror agreed to participate despite the blowback he has received for his actions in the field of forensic pathology.

It is worth all the grief I have been getting if my work results in one person not being wrongly convicted or one person not going free. Hopefully this will change the domain.