This topic may be a bit different than what I usually deal with, but it still falls within my area of interest. I always liked art. Back in the days mainly pencil art, then pen and pencil art, and then later colorful paintings as well. Only in the past few years did this list also include digital paintings and digital art, being a conservative and old-school guy.

My main interest has always been fantasy art as you probably could have guessed. Growing up I had collections upon collections of pictures stored on my PC, neatly filed under the names of different artists. For a short time, I was obsessed with collecting everything from everyone that could be found on the net. With a lot of different online apps, and services popping up and with the advent of ubiquitous, broadband internet, this hobby (or obsession) of collecting things offline made little sense after some time. If I would like to see something, I’ll just conduct a very quick Google search and I am sure in a matter of minutes I’ll find the art I am looking for.

I consider the emergence of Artificial Intelligence in the art world as the next step in history or maybe in evolution? I think this is a controversial topic. Will AI-generated “art” displace art and artists as we know them today? Is that the goal at all and is that actually possible? Some of the most brilliant artists were known for trying to innovate and come up with new, surprising, sometimes even shocking ways to create and perform. I guess one of the other questions is – is AI a tool for the artist or is AI the artist itself? If you could tell a brush to paint you a female human portrait and it did just that – is that really created by You or the brush? Even if You consider it only a tool, a brush, you need to give it far more credit than you would to a real brush, as even the unskilled can create nice, pleasing, I might even say beautiful images using it.

How difficult is using an AI to create art? Where and how can you do this? Who is the owner of art created using Artificial Intelligence? Is it the one tweaking the settings and feeding it with keywords or the AI itself? Or the programmer of the AI? What does licensing look like? I’ll try to tackle some of these…

What is the “artist” AI?

A Story of GAN and DALL-E

This is going to be a rather short summary, not a technical deep dive. There are millions of scientific, in-depth guides and explanations that you can find in case you want to know more, but that is not my goal here.

As I found during my investigation, most artists use GAN to create masterpieces. GAN stands for Generative Adversarial Network, which is a so-called “machine learning” framework that was introduced by Ian Goodfellow in 2014. According to Wikipedia “Machine learning algorithms build a model based on sample data, known as training data, in order to make predictions or decisions without being explicitly programmed to do so.”

So GAN does exactly this – it learns and then generates new data. The “sample data” in this case may be photos or paintings for example, and using those, GAN will attempt to create something original. This alone is not everything. GAN is actually using two parallel, dynamically updated processes, both of which utilize machine learning. One process is the creator, the “dreamer” or the “painter” called the generator, and the other is the “curator”, the “critic” called the discriminator that is checking how “realistic” the creation seems. The goal of the generator process is to fool the discriminator and create something original and realistic. You could say that the generator is running in competition with the discriminator. If it succeeds (wins), and the discriminator cannot tell whether the creation is real or fake, a piece of art is created and the generator learns something new (about what is considered “realistic“) and updates itself. All of this is unsupervised learning.

In 2021, another AI, DALL-E appeared, developed by OpenAI, which is supposedly the new, advanced, and more flexible way of creating art, and which was already superseded by (or updated to) DALL-E 2 in 2022.

TRIVIA. DALL-E is a blend of two words: It's compiled out of WALL-E, which is the title of a Pixar animated movie, and the name of Salvador Dalí, a Spanish artist.

DALL-E is using CLIP (also developed by OpenAI) to do its magic. Another strange-sounding technology. CLIP stands for Contrastive Language-Image Pre-training. Basically, this is the reverse process of what we want to do with DALL-E – CLIP returns the text (caption) describing an image; it matches the image to words. The name “Language-Image Pre-training” pretty much already betrays that this is trained in image-text pairs. If you want to get deeper into understanding how DALL-E 2 works with CLIP, you can do that by reading (and watching) THIS AssemblyAI post. DALL-E 2 Is I believe not fully available at this point, but there is a waiting list you can get on.

One thing to note is that with almost all of these AI Art Generators, you can or are required to add keywords. There is a difference in how well they can interpret these. Can they (try to) understand natural human language and the context of words in a sentence or are just the keywords themselves thrown in one after the other? Some understand real, intelligent text, and some just take into account the words you typed in and try to make something out of them.

All this being said, there are a lot of other combinations and variations of these, and I can see different flavors and matrixes emerging. These combinations are around how they handle text-to-image learning for example or how they construct the images themselves. An example would be Nightcafé (an AI Art generator site I like) which claims to use Neural Style Transfer, VQGAN+CLIP, and CLIP-Guided Diffusion.

How Can You Use AI to Create Art?

Anyone can try their hands at creating art using Artificial Intelligence through the help of publicly available webpages, that offer some free services to evaluate them. Most of the ones I’ve seen you can try for free and if you like them, you can subscribe to additional processor cycles or run-time; which is the same as saying you can get a monthly subscription.

There is an interesting thing I found after trying five or six of these and it’s that the user is also required to learn. This is also continuous learning for the creator as well, not just the AI. You need to learn how to “use these tools” or I should say “how to talk to these AIs”. How do they react to keywords, and keyword orders; what do they do if you repeatedly tell them to do the same thing? What is the proper combination of styles and text that will give you the result that you like the most? I believe I had to attune to these Artificial Intelligence engines. The site that will work the best for you is where this is easy and you can get in sync with the AI using the least amount of effort.

I tried Starryai, Dream (at app.wombo.art), craiyon, Hotpot (at hotpot.ai), Nightcafé, and Midjourney. Altogether I created about 1500 images so far and my favorites are Nightcafé és Midjourney.

Nightcafé has a very user-friendly GUI and it is very clean and intuitive. You can try to create a handful of pictures but after that, it has a subscription. One of its strongest points is the large resolution it can enhance your creations, all the way up to 34-35 megapixels.

You can see a couple of samples here of what I’ve created or go to my profile directly. This is great for color images in huge sizes. I’d recommend checking out the profile not just to see great I am at using the AI, but you can see much more. Try it.

The other tool, Midjourney is a chat-based tool. You can use it by being invited to a Discord server and then supplying the AI with dumb CLI (Command Line Interface) commands. I actually enjoy this very much, this being closer to my work (network engineer).

The maximum size of art created here currently is around 3 megapixels. It is fast and clever and the whole experience is HIGHLY addictive… Before I knew it I created roughly 900 images. Some of them definitely look like any pencil or ink drawing created by actual human artists… Just a handful of examples below:

Closing Thoughts

Back to the question of whether these will replace human artists… I cannot see the future. What I could say is that probably not yet, but they are really good at what they are doing. What the emergence of Artificial Intelligence does do, however, is that it makes creating art pieces far more accessible to the artistically unskilled public. I do not know what consequences this will have in the long run. Could this take a portion of the market share from smaller, less known artists, or from those who create Stock Art or commissioned artworks? This technology is so new that I do not think it found its place yet. There are a lot of unanswered questions and a lack of legislation in general around things done by AIs or created by AIs.

So what exactly can you do with the created art and who owns the rights? Currently, only humans can own rights; AIs can’t. Some sites claim that what you create is yours and that you can sell it (might even encourage it), but there are sites that make you accept an agreement when you join, saying that everything you create falls under Creative Commons. Be sure to check contracts, agreements, and FAQs when you sign up for these places!

For all I know, I could illustrate a fantasy publication using the pictures I created (as decorative, supporting illustrations – not as the product itself, but as part of the product) and sell it through LuLu or DTRPG. I’d go as far as saying if I did this right, no one would know that the pieces were created by Artificial Intelligence.

EDIT/UPDATE: A stock art/ illustration publisher just sent out a newsletter that I also got. I’ll quote from that, without adding the name:

So, in the last month, the influx of AI art is really hitting us illustrators hard. Not only did I lose some smaller clients, but the stock art market is being flooded with these images.

My above assumptions seem to be correct, unfortunately…