Did an AI Really Invent Its Own 'Secret Language'? Here's What We Know

There is a new generation of artificial intelligence models that can make "creative" images on demand. The likes of Imagen, MidJourney, and DALL-E 2 are changing the way creative content is made with implications for intellectual property in mind.

It is difficult to know how these models produce their results. Researchers in the US claimed last week that the DALL-E 2 model may have invented its own language to talk about objects.

DALLE-2 has a secret language. "Apoploe vesrreaitais" means birds. "Contarra ccetnxniams luryca tanniounons" means bugs or pests. The prompt: "Apoploe vesrreaitais eating Contarra ccetnxniams luryca tanniounons" gives images of birds eating bugs.

A thread (1/n)🧵 pic.twitter.com/VzWfsCFnZo

— Giannis Daras (@giannis_daras) May 31, 2022

By prompting DALL-E 2 to create images containing text caption, then feeding the resulting (gibberish) caption back into the system, the researchers concluded

The security and interpretability implications of these claims are intriguing. What is happening?

Does DALL-E 2 have a secret language?

DALL-E 2 isn't likely to have a secret language. We don't know if it has its own vocabulary or not.

Only a few researchers and creative practitioners have access to DALL-E 2 and other large artificial intelligence models, so it's difficult to verify claims.

Any images that are publicly shared should be taken with a grain of salt, because they have been "cherry- picked" by a human from among many output images generated by the artificial intelligence.

The models can only be used in limited ways by people with access. DALL-E 2 users can modify images, but can't interact with the system in a deeper way.

The methods for understanding how these systems work are difficult to apply.

What's going on then?

There is a chance that the "gibberish" phrases are related to non-English language words. The Latin Apodidae is a family of bird species and Apoploe is similar to it.

This appears to be a plausible explanation. Many non-English words were included in the data that DALL-E2 was trained on.

Artificial intelligence models have been able to write computer code without being trained.

Is it all about the tokens?

The fact that language models don't read text the same way as we do supports this theory. The input text is broken into "tokens" before it's processed.

There are different approaches to tokenization. It's easy to treat each word as a token, but it can cause trouble when the same token has different meanings.

On the other hand, if you treat each character as a token, you can get a smaller number of possible token, but they don't mean much.

DALL-E 2 uses an in-between approach. This could be an important factor in understanding the "secret language".

that "secret language" seems like mostly tokenizer effects. you can do the inverse too: 1) i picked two families of fish "Actinopterygii" and "Placodermi" from wikipedia2) prompted dalle with "placoactin knunfidg"

3) dalle consistently generates fish images https://t.co/ndAe7MURyg pic.twitter.com/1kHk5NWJb3

— rapha gontijo lopes (@iraphas13) June 3, 2022

Not the whole picture

The "secret language" is an example of the principle of garbage in and garbage out. DALL-E 2 can't say "I don't know what you're talking about" because it can't say "I don't know what you're writing".

None of these options give a complete explanation of what's happening. The generated images appear to have been corrupted by removing individual characters from gibberish. If there was a secret "language" under the covers, individual gibberish words might combine to produce coherent compound images.

Why this is important

You may be wondering if any of this is important.

Yes, the answer is affirmative. DALL-E's "secret language" is an example of an "adversarial attack" against a machine learning system, a way to break the intended behavior of the system.

The model is challenged by the attacks. It is possible that the artificial intelligence interprets gibberish words in a way that also interprets meaningful words in a different way.

Security concerns are raised byversarial attacks. DALL-E 2 filters input text to prevent users from generating harmful or abusive content, but a secret language of words could allow users to circumvent these filters.

The research shows that short phrases such as "zoning tapping fiennes" can reliably cause the models to spew out racist or biased content. Understanding and controlling how deep learning systems learn from data is part of the ongoing effort.

DALL-E 2's secret language raises interpretability concerns. We want these models to behave in line with our expectations, but we can't see structured output because of gibberish.

Shining a light on existing concerns

The hullabaloo over some Facebook chat-Bots that " invented their own language" was a big deal. The results are concerning, but not like "Skynet is coming to take over the world."

DALL-E 2's secret language highlights concerns about the robustness, security, and interpretability of deep learning systems.

We won't be able to really know what's going on until these systems are more widely available.

The DALL-E mini is a smaller model that you can check out if you want to try generating some of your own images. Don't use English or gibberish to prompt the model.

Snoswell is a post-doctoral research fellow at the university.

Under a Creative Commons license, this article is re-posted. The original article is worth a read.