AI art is not generative art (clickbait title). While the technical definition says that one is a subset of the other, I think it’s useful to distinguish between these two categories.
Why I’m writing this in the first place — Starting 2022, the term “generative art” had been progressively becoming synonymous with art produced by AI text-to-image systems. As a consumer and producer of (traditional) generative art, it was becoming a bit annoying to browse generative art content on the internet. Whether through tags like #generativeart or communities like r/generative, spaces are being flooded with AI-generated images which I and many others are not interested in. End rant.
In 2024, things are a bit different. AI art is now commonly referred to as ‘AI art’. I shouldn’t have procrastinated writing this post for so long.
There are also cases where generative artists are pressured to relabel their art, so as to not be mistaken for being AI (and avoid things associated with it). It’s an unfortunate niche pressured to be nichey-er.
It’s kinda like animals vs humans. While technically & scientifically, humans are (a subset of) animals; in daily life “animals” usually mean non-human animals, as in “animal lover” and “animal rights”. I digress.
This article looks at the similarities and differences between (traditional) generative art and text-to-image AI art in different angles. Skip to different sections if you want.
Tracing the history of each practice may offer insights and nuance. Don’t worry, there are pictures! And interactive things! (too lazy to implement interactive things for now)
This is not a comprehensive history of either field. Btw, this timeline layout is better viewed on a large screen.
Frank Rosenblatt built a relatively simple neural network that can be trained, called the perceptron.
The perceptron was designed for image recognition. The perceptron looks at some input (an image) and decides from a set of pre-learned classes of images, which kind of image it is.
Artist Sol LeWitt started doing something called wall drawings, which are sets of instructions that produce abstract drawings. These can be thought of as precursors to generative art.
Example Wall Drawing:
“Wall Drawing #797. The first drafter has a black marker and makes an irregular horizontal line near the top of the wall. Then the second drafter tries to copy it (without touching it) using a red marker. The third drafter does the same, using a yellow marker. The fourth drafter does the same using a blue marker. Then the second drafter followed by the third and fourth copies the last line drawn until the bottom of the wall is reached.”
Product of Wall Drawing #797, executed from instructions written by LeWitt.
"The idea becomes a machine that makes the art." (Paragraphs on Conceptual Art, 1967) It’s not quite ‘generative art’ but it’s close. Besides, LeWitt called it something else: ‘conceptual art’.
Vera Molnár brought us closer to modern generative art by actually using computers and writing code to generate artworks.
Vera Molnár
The limitations of relatively primitive AI and weak computing power back then resulted in massive disappointment about AI. Budgets were cut. R&D slowed down. Hype was lost.
The Perceptron did not achieve self-awareness as promised.
Artists began meeting in generative art conferences and the artistic community started converging on a shared definition of generative art.
Generative art — work that has been produced by the execution, usually by a computer system, of a set of rules or procedures determined by the artist.
The intersection of advancements in computing power, improved machine learning algorithms, and huge ever-growing datasets of digital things resulted in bigger and better neural networks. The great potential of AI was reignited. It’s the start of the deep learning revolution.
Image recognition, picture captioning, automated language translation, text-to-speech, speech-to-text, etc. got seriously buffed.
Google’s DeepDream exploits image recognition feeding into digital pareidolia. It can be considered early AI art, but it’s quite different from modern AI art.
Due to deep learning, we had advancements in generative networks, which are networks that can generate new data on their own instead of simply being a complex input-output mapping.
This appears to be the start of the image synthesis wave.
Here is a fake person generated by a generative adversarial network (GAN) trained specifically on portraits. Tap to regenerate a new one from thispersondoesnotexist.com.
The first modern text-to-image model was introduced.
It was called alignDRAW.
It can generate images from natural language descriptions, making these machines massively accessible to the general public. Anyone could potentially create any image!
alignDRAW’s output for “A toilet seat sits open in
the grass field.”
DALL·E was revealed by OpenAI. Another text-to-image model, and it became famous. (Or, rather it was made famous by “DALL-E mini” which was viral.)
DALL·E’s output for “A toilet seat sits open in the grass field.”
Meanwhile, 2022 saw a burst of NFTs, most of which have been associated with generative art.
Honestly, some of them are on the borderline shallow end of generative art, Mad Libs-style images generated from shufflings of pre-drawn hairstyles, sunglasses, and clothes. Automated dress-up. They are more akin to the musical dice game.
Typical images associated with NFTs
Text-to-image models were approaching the quality of real images. Generated artworks were approaching the quality of human art.
AI-generated painting
Today, we see different techniques and tools to make generative art, from straight up canvas coding, to frameworks like p5.js, to full-blown authoring software like TouchDesigner and Houdini.
An output of the QQL algorithm which uses the p5.js library
Tools to make AI art and images have been heavily commercialised, with online image generation services provided by OpenAI/Microsoft, Google, and Midjourney. There are also local models for advanced users and tinkerers.
Microsoft’s image creator which at the time used the DALL·E 3 model
So. AI art is a relatively new movement borne out of general image synthesizers. This was made possible since visual artworks are images too. Meanwhile, there have been decades of development and refinement in (traditional) generative art, techniques, and algorithms.
The terminology may overlap (generative networks ~ generative art), but these are just labels. You may call them whatever you want, but they’re still separate art movements that cannot be historically reconciled.
By the way, I don’t care if you want to call them ‘art’ or not. That’s not the point of this post. If it bothers you, please imagine that all instances of the word ‘art’ were replaced with ‘image’. Same for the word ‘artist’.
OK so, whatever sure, history is in the past. Why don’t we look at it in terms of what artists are doing today. What do they do? What do they practise to improve?
Both practices involve some skills that are not in common with the other. They each have their own defining skills.
| Generative artists | AI artists |
|---|---|
|
programming math / geometry / trigonometry |
prompting curation / training data science |
Generative art involves programming and uses math (usually geometry or trigonometry) to create graphics. A good undertanding of functions and their domains and ranges and how they compose together is essential, especially when utilising random number generators.
Typical generative art math
Beyond these core skills, there are a lot of areas generative artists could expand into, like graph theory, simulations, fractals, image processing, and other algorithms.
Scattered Pathways by Nate Nolting.AI art, on the other hand, is heavy on prompt engineering. It could be as simple as writing English descriptions, or as precise as inputting parameterised labels if the tool allows for it.
Beyond writing prompts, there are several tools that allow more control like influencing certain compositional elements. AI artists could also train or fine-tune models by curating image-label datasets as extra training data.
Images generated with ControlNet, controlling the general shape of the subject.
There seems to be little to no transferable skills between these practices. Sure, you can program some scripts to automate parts of an AI art workflow, or add an AI filter step to an otherwise procedurally-generated image. But there is no inherent overlap.
At the highest level, it is the difference between computer science vs data science. At the lowest, it is the difference between coding and writing English.
That’s why you can’t just ask an AI artist to please create an interactive audio-reactive visualisation for your music video, or a public art installation that reacts to the weather. Likewise, you can’t expect a generative artist to easily generate photorealistic art in seconds.
One big thing that’s common between these practices is the need to curate. Since there is an element of chaos and randomness in both processes, a curation step is almost always required before finishing a work. Curation requires artistic vision, which could be considered a skill in itself.
|
Generative artists |
curation
artistic vision |
AI artists |
What’s curation? As one finishes a piece, they usually generate multiple outputs by running the process with different seeds or starting parameters. These produce different variations, from which the best one(s) are selected for publishing, discarding the rest. That’s curation.
Alternatively, one might refine and iterate their prompt or program until the outputs satisfy a certain artistic vision.
Uncurated set of outputs from a generative art algorithm
AI image generators usually generate multiple outputs at a time for you to curate.
It could be argued that having artistic vision and doing iterations are pretty common to all arts, and so those alone don’t make generative art and AI art any more similar to each other than any other art.
Overall, the significant differences in skillsets mean that generative artists and AI artists are not interchangeable, and the same should go for their respective arts.
Finally, let’s attack the subject from the POV of processes and systems.
Setting aside the question of whether Art is Product or Process, let us indulge in these animations of art in the process of being products.
Most modern AI image generators are based on diffusion models — they form images via ‘denoising’. The animation above (left) shows the sequence of steps to progressively denoise a pure noise image into something coherent based on a prompt. The prompt for the above generation specifically was ‘cherry blossom branches against a clear blue sky’.
The denoising process is a black box, an inscrutable network of billions of artificial neurons trained to generate coherent arrays of pixels.
Generative art algorithms, on the other hand, are relatively more hand-crafted, less magical. A generative art algorithm consists of instructions to explicitly draw every element that will be in the piece. This is apparent in the above animation (right) where each drawn stroke can be seen. This particular animation was from The Soul of Flowers by Che-Yu Wu.
There is large contrast in the transparency of these processes. One is a black box. The other is literally the instructions to create the art.
Beyond the processes themselves, art interacts with the context of its creation. And there is a lot to unpack here, especially for AI art. I’ll use diagrams to illustrate the context of the creation process.
In the generative art process, I see 3 components at the minimum: the artist, the program, and the rendered output.
The generative artist writes the program that renders the piece of art
They may use libraries or frameworks as part of the program, but the artist ultimately writes, at a certain level of abstraction, the specific steps or rules which the program would execute.
The medium is code.
In AI art, there are 4 components to the process: the artist, the model, the rendered output, and the dataset.
The AI artist prompts the model trained from the dataset to produce the piece of art
There is an immediate difference. Unlike AI art, generative art don’t need datasets.
And it’s also a controversial thing, the dataset. The data used to train a text-to-image model can consist of billions of images, usually scraped from the internet and includes other artworks by other artists.
If you consider that the model has been influenced by a lot of other images on the web, then the question of how important the AI artist’s influence over the final piece is raised. Was that chiaroscuro an artistic choice, or an emergent byproduct of the model? Were those bold paint strokes an explicit expression of the artist, or just some typical style chosen by the model?
In AI art, there is confusion over which elements were expressed by the artist and which by the machine.
Leaving the contentious topic of datasets aside, there is another point, but not as essential, of who authored the main program. Who created the most crucial piece of the system?
Generative art programs are usually created by the artists themselves, while AI art programs are usually created by AI researchers, only to be used as a product by AI artists.
One makes the machine that makes the art, the other uses a machine to make the art.
Of course, AI artists could train (not fine-tune!) their own models from scratch, if they have a big(!) enough dataset of their own and some machine learning know-how. But I don't think that’s a common occurrence.
In summary, one is a direct expression from the artist where the medium is code, and the other is a complicated web of datasets including other artworks, machine learning, and just-add-water kind of products, and I don’t even know what the medium is.
The point is that these are separate art mediums. Or art movements, if you will. The histories, the crafts, and the processes point to the same conclusion — they are no more similar than oil painting is to photography, or a film director to an animator.
So let’s not confuse one with the other. :)
Some aspects of the post have probably been outdated by the time I publish this post. I’ll still include them in the post because why not.
P.S. a similar thing: cryptoisnotcryptocurrency.com, Wikipedia article.
Now, I should go back to regular programming for this blog.
Thanks for reading! I like making interactive visualisations for my programming blog. Sometimes I do projects too. Oh, and before you go, sign the guestbook! See you around! —Lean
