Adversarial images are pictures that contain carefully crafted patterns designed to fool computer vision systems. The patterns cause otherwise powerful face or object recognition systems to misidentify things or faces they would normally recognize.
This kind of deliberate trickery has important implications since malicious users could use it to bypass security systems.
It also raises interesting questions about other kinds of computational intelligence, such as text-to-image systems. Users type in a word or phrase and a specially trained neural network uses it to conjure up a photorealistic image. But are these systems also susceptible to adversarial attack and if so, how?
Today we get an answer thanks to the work of Raphaël Millière, an artificial intelligence researcher at Columbia University in New York city. Millière has discovered a way to trick text-to-image generators using made up words designed to trigger specific responses.
Adverse Consequences
The work again raises security issues. “Adversarial attacks can be intentionally and maliciously deployed to trick neural networks into misclassifying inputs or generating problematic outputs, which may have real-life adverse consequences,” says Millière.
In recent months, text-to-image systems have advanced to the point that users can type in a phrase, such as an astronaut riding a horse, and receive a surprisingly realistic image in response. These systems are not perfect but nevertheless impressive.
Nonsense words can trick humans into imagining certain scenes. A famous example is the Lewis Carroll poem Jabberwocky: “’Twas brillig, and the slithy toves, Did gyre and gimble in the wabe…” For most people, reading it conjures up fantastical images.
Millière wondered whether text-to-image systems could be similarly vulnerable. He used a technique called “macaroni prompting” to create nonsense words by combining parts of real words from different languages. So the word “cliff” is Klippe in German, scogliera in Italian, falaise in French and acantilado in Spanish. Millière took parts of these words to create the nonsense term “falaiscoglieklippantilado”.
To his surprise, putting this word into the DALL-E 2 text-to-image generator produced a set of images of cliffs. He created other words in the same way with comparable results: insekafetti for bugs, farpapmaripterling for butterfly, coniglapkaninc for rabbit and so on. In each case, the generator produced realistic images of the English word.
Millière even produced sentences of these made-up words. For example, the sentence “An eidelucertlagarzard eating a maripofarterling” produced images of a lizard devouring a butterfly. “The preliminary experiments suggest that hybridized nonce strings can be methodically crafted to generate images of virtually any subject as needed, and even combined together to generate more complex scenes,” he says.
A farpapmaripterling lands on a feuerpompbomber, as imagined by the text-to-image generator DALL-E 2 (Source; https://arxiv.org/abs/2208.04135)
Millière thinks is possible because text-to-image generators are trained on a wide variety of pictures, some of which must have been labelled in foreign languages. This allows the made-up words to encode information that the machine can understand.
The ability to fool text-to-image generators raises a number of concerns. Millière points out that technology companies put great care into preventing illicit use of their technologies.
“An obvious concern with this method is the circumvention of content filters based on blacklisted prompts,” says Millière. “In principle, macaronic prompting could provide an easy and seemingly reliable way to bypass such filters in order to generate harmful, offensive, illegal, or otherwise sensitive content, including violent, hateful, racist, sexist, or pornographic images, and perhaps images infringing on intellectual property or depicting real individuals.”
Unwanted Imagery?
He suggests that one way of preventing the creation of unwanted imagery would be to remove any examples of it from the data sets used to train the AI system. Another option is to check all the images it creates by feeding them into an image-to-text system before making them public and filter out any that produce unwanted text descriptions.
For the moment, opportunities to interact with text-to-image generators is limited. Of the three most advanced, Google has developed two, Parti and Imagen, and is not making them available to the public because of various biases it has discovered in their inputs and outputs.
The third system, DALL-E 2, was developed by the Open AI Initiative and is available to limited numbers of researchers, journalists and others. This is the one Millière used.
One way or another, these systems or other similar ones, are bound to become more widely used, so understanding their limitations and weaknesses is important for informing public debate. A key question for technology companies, and more broadly for society, is how these systems should be used and regulated. Such debate is urgently needed.
Ref: Adversarial Attacks on Image Generation With Made-Up Words : arxiv.org/abs/2208.04135
Made-Up Words Trick AI Text-To-Image Generators & Latest News Update
Made-Up Words Trick AI Text-To-Image Generators & More Live News
All this news that I have made and shared for you people, you will like it very much and in it we keep bringing topics for you people like every time so that you keep getting news information like trending topics and you It is our goal to be able to get
all kinds of news without going through us so that we can reach you the latest and best news for free so that you can move ahead further by getting the information of that news together with you. Later on, we will continue
to give information about more today world news update types of latest news through posts on our website so that you always keep moving forward in that news and whatever kind of information will be there, it will definitely be conveyed to you people.
Made-Up Words Trick AI Text-To-Image Generators & More News Today
All this news that I have brought up to you or will be the most different and best news that you people are not going to get anywhere, along with the information Trending News, Breaking News, Health News, Science News, Sports News, Entertainment News, Technology News, Business News, World News of this made available to all of you so that you are always connected with the news, stay ahead in the matter and keep getting today news all types of news for free till today so that you can get the news by getting it. Always take two steps forward
Credit Goes To News Website – This Original Content Owner News Website . This Is Not My Content So If You Want To Read Original Content You Can Follow Below Links