Palette.fm does AI photo colorization using text prompts for refinement.
Enlarge / Palette.fm does AI photo colorization using text prompts for refinement.

A Swedish machine learning researcher has released a tool that uses artificial intelligence to colorize photos. Users can use a written description to refine the colors after uploading a photo.

A deep learning model is used to classify images by guiding its initial guesses for the colors of objects. Wallner wrote in a message to Ars that he made a custom model using the image and text. One model creates the text while the other takes the image and the text to create the colorization.

The site's sleek interface provides an estimated caption for what it sees in the picture after you've uploaded it. If you don't like any of the preset filters, you can click the pencil icon to change the caption on your own.

We took a picture of a small pumpkin and used a program to remove the color. After uploading the black-and-white version, we tried out the pre-made filters on the website. We changed the caption to describe the objects in the scene after finding a good filters. The pumpkin was thought to be a "claw" by the website. The colors made more sense after we wrote those terms in the prompt. We added green leaves in the background to further refine the image.

Advertisement
An original Ars test photo (left), one with color removed (center), and one colorized by Palette.fm (right).
Enlarge / An original Ars test photo (left), one with color removed (center), and one colorized by Palette.fm (right).

Wallner wants to add a paid option to the service. The images are uploaded into the cloud. "We don't store your images" is what the website says about the privacy of uploaded photos. It's a good idea to take that with a grain of salt. There is no requirement for user account registration at the moment.

People on Hacker News have been using the tool to colorize photos of loved ones. A variation of Wallner's colorization technology can be used as a bot on the social networking site. Have a good time coloring.