Google is using a new way to measure skin tones to make search results more inclusive

A new scale for measuring skin tones is being promoted by a Harvard professor in hopes of fixing problems of bias and diversity in the company's products.

Ellis Monk, an assistant professor of sociology at Harvard and creator of the Monk Skin Tone scale, is working with the tech giant. The skin tone scales that are biased towards lighter skin have been replaced by the MST scale. Tech companies use older scales to categorize skin color, which can lead to products that perform worse for people with darker coloring.

Monk says that they can integrate the differences in skin tone into products to make sure they are more inclusive.

Tech products that use artificial intelligence perform worse with darker skin tones. There are apps designed to detect skin cancer, facial recognition software, and even machine vision systems used by self-driving cars.

The use of outdated skin tone scales is one of the ways bias is programmed into these systems. The Fitzpatrick scale is the most popular skin tone scale. The scale was originally designed in the 70s to classify how people with paler skin burn or tan in the sun, but was later expanded to include darker skin.

The Fitzpatrick scale fails to capture a full range of skin tones and may mean that when machine vision software is trained on Fitzpatrick data, it is biased towards lighter skin types.

Image: Ellis Monk / Google

The Fitzpatrick scale is comprised of six categories, but the MST scale expands this to 10 different skin tones. The number was chosen because of Monk's research to balance diversity and ease of use. Too much choice can lead to inconsistent results, but some skin tone scales offer more than a hundred different categories.

If you get past 10 or 12 points on these scales, you can ask someone else to pick out the same tones, but the more you increase that scale, the less people are able to do that.

A new skin tone scale is only the first step in integrating this work into real-world applications. The skintone.google website was created to explain the research and best practices for its use in artificial intelligence. The company is working to apply the scale to a number of its own products. The Real Tone photo filters are designed to work better with darker skin tones.

Image: Google

The new image search feature will allow users to refine searches based on skin tones that are classified by the MST scale. If you want to find eye makeup orbridal makeup looks, you can use skin tone as a filter. In the future, the company will use the MST scale to check the diversity of its results so that if you search for images of cute babies or doctors, you will win.

One of the things we're doing is taking a set of results, understanding when they are particularly homogenous across a few sets of tones, and improving the diversity of the results. The updates were in a very early stage of development and hadn't yet been rolled out across the company's services.

This should be taken with a pinch of caution, not just for this specific change, but also for the approach to fixing bias in its products more generally. The company has a spotty history when it comes to ethical guidelines, and the industry as a whole has a tendency to promise ethical guidelines and then fail on the follow-through.

This mistake was first noticed in 2015, but it was not noticed before.

These changes can be culturally and politically challenging, reflecting broader difficulties in how we integrate this sort of tech into society. In the case of image search results, diversity may look different in different countries, and if the image results are adjusted based on skin tone, it may have to change them.

What diversity means is going to be inherently different when we are looking at results in different parts of the world.

The introduction of a new and more inclusive scale for measuring skin tones is a step in the right direction.