top of page
  • Writer's picturedanieleproietto92

Nightshade: someone has started poisoning the AI

Generative artificial intelligence represents a groundbreaking milestone in the world of technology, opening new horizons for content creation. However, this innovation brings with it several shadows that encompass the realms of privacy, intellectual property, and the spread of fake news.



An emblematic example is the case of images generated by AI: their ability to mimic reality is such that it can confuse the average user, lacking the appropriate tools to distinguish the true from the false.


The implications for categories of artists, photographers, and designers are particularly concerning.


A critical point concerns the issue of copyright. Often, AI models draw from online images without distinction, violating the intellectual property of artists. This fuels a system that rewards copying at the expense of originality. Many artists have discovered that their works have been taken and used without any authorization in datasets such as, for example, LAION-5B.


The poisoning of AI


Faced with this threat, some artists have decided to react. The University of Chicago has developed two tools, Nightshade and Glaze, which allow actively "poisoning" AI.


Glaze is a defensive tool: it can insert pixels that confuse AI about the style of the image, preventing imitations. Many AIs, starting from a work of art, mimic its style. Thanks to Glaze, this style mimicry can be stopped.


Nightshade, on the other hand, is an offensive and more aggressive weapon: it alters pixels in images imperceptibly to the human eye, but it can confuse the AI and make it learn incorrect information. By using Nightshade, models will be poisoned with inaccurate information, causing them to produce unpredictable outputs


Wikitrivia: Nightshade is a plant that in Italy we refer to as (Atropa) Belladonna. The name Atropo was indeed the name (in Greek: Ἄτροπος, the unchangeable, the inevitable) of one of the three Fates who, in Greek mythology, cuts the thread of life, serving as a reminder that the ingestion of the berries of this plant causes death.
The specific epithet 'belladonna' refers to a practice dating back to the Renaissance: ladies used an eye drop based on this plant to enhance and brighten their eyes due to its ability to dilate the pupil.

In an experiment, researchers 'poisoned' the AI with images of dogs modified by Nightshade, making it believe they were cats. The AI then generated deformed images of dogs and, after a hundred iterations, began producing cats instead of dogs.


The team of researchers led by Prof. Ben Zhao clearly states that Nightshade should be used ethically and was not designed to break AI models but to protect users from the rampant scraping by large companies (a method in which entire websites and their contents are 'scanned' indiscriminately), as there are currently no suitable tools to ensure respect for copyright and intellectual property rights.


Only through open and constructive dialogue among artists, developers, and policymakers can guidelines be established for a future of art in which artificial intelligence is an ally rather than a threat.




Comments


bottom of page