“Could you draw me a bowl of soup that looks like a monster knitted out of wool?”
I'll bet that most good artists and photographers couldn't handle such a strange request. When I hear it, I can't picture anything remotely like it either. Dalí, however, pictures something very interesting:
This artist certainly deserves respect, so from here on we'll take care to spell his name properly. Because DALL·E may lack the waxed mustache and the eccentricity of its namesake Salvador, but it does thrilling things. It is an algorithm that uses artificial intelligence to turn a description into an image. And with such precision!
“Draw me a sheep,” says a little golden-haired boy to a pilot stranded among the scorching dunes of the Sahara. We all know what happens next, but the pilot's answer is beautifully childlike.
DALL·E's answers are anything but childlike. Its name is spelled like the name of one of Musk's children, and it comes from an association with its namesake painter and the animated hero WALL·E, Pixar's little robot. Since the first days of May this year, the algorithm has caught the attention of people interested in artificial intelligence with some stunning examples.
Dalí or Not Dalí, Who Can Tell?
DALL·E is quite active online. It even regularly posts its new works on Instagram.
Last year the first version of the project hinted at its impressive prospects, and the second version came along not only to show that algorithms are capable of creating, but also to reveal much more important things about what they can do.
The project is the work of one of the two largest groups of scientists today doing the high-wire acrobatics of artificial intelligence development. OpenAI is the company backed by Musk as a rival to Google, and it created the GPT-3 technology. It is precisely this pretrained neural network, the most impressive one so far, that also stands behind the electronic painter.
Our hero is definitely considered a serious breakthrough in its field. As it's worth mentioning again, today's artificial intelligence algorithms are at their strongest when they have one of two tasks: processing a huge volume of information to predict what the next unit will be, or classifying it. They are doing more and more impressively in some of the areas where we expect miracles, such as facial recognition, they lag behind in others, but on the whole they keep developing within these basic roles.
Rain or Dalí
Meanwhile our new friend DALL·E is changing the paradigm. It analyzes the connections between human speech and images. And as a result it creates unexpected pictures, sometimes impressive in their mastery.
Which brings us back to the big question. Can we call them art?
It really is astonishing. Let's try again.
Someone: Dear DALL·E, how about drawing us some teddy bears mixing bubbling chemicals like mad scientists from a 1990s Saturday morning cartoon…
DALL·E:
That's right: we can tell it not only what we want to see, but also in what style. We can describe and explain, and it will do its best to complete the task. On top of that, it gives us many possible solutions to choose from
Well, sometimes the results may resemble, in terms of realism, those from its namesake's oeuvre. It is no accident that for now the algorithm still hasn't been released for free “testing.” To try it yourself, you sign up for a waitlist (you can be sure I've been on it since the moment I heard about it; if you like, try it yourself here). But this is a standard approach in the world of artificial intelligence, where things are still rather unclear and not always controllable. Sometimes the attempt to imitate natural intelligence ends with us bringing its ugliest sides to the fore.
A Dog Drags a Bus, Leaves No Trace
I promise that this—a line of Bulgarian nonsense—will be the first task I give it. Keep your fingers crossed that my turn comes, and I'll share the result afterward!
But in the meantime, jokes aside, it's worth saying more about DALL·E's merits.
As mentioned, the new version of the artist uses the GPT-3 algorithm. It was trained on 650 million images and the texts that accompany them, searching for patterns.
That is exactly where its unconventional success lies. It shows that AI can turn out to be surprisingly powerful if we show it various patterns rather than simply channeling it into classifying or predicting. And this may very well turn out to be the road to creating general artificial intelligence: the next generation of algorithm, one that will be able to solve different tasks with one and the same program.
And all of this takes us further and further down the road toward a machine “smarter” than humans.
Fool Me Dalí
There are, of course, many steps before we get there, and it's far from certain that the winding path won't peter out. Still, DALL·E's success is impressive.
Let's take a stroll through the electronic master's exhibition. It can do more than generate an entirely new painting or photograph. It is also capable of retouching. You tell it to add a duckling somewhere in the middle of the chaos, and it finds a spot for it.
Then it can offer you different variants. I'm no specialist in this field, but… do you notice how the algorithm handles light and shadow? Even if it draws some criticism from the real masters, it will certainly save a lot of work for the people who do retouching.
At least they called it “retouching” before the Photoshop years. After that we stopped being so sure of everything we see in a picture… Alas, we didn't stop, though we should have. Even today we are often far too easily convinced, and photo manipulation really can be flawless, with video following in its footsteps.
But what will happen when the algorithm starts managing on its own? And if we add the looming general AI, capable of performing different tasks…
DALL·E 2 understands very well what “style” means. If you tell it what to draw, it will show more creativity, giving you quite varied options, but if you restrict it to an era, a movement or a mood, it will try to help you within the given limits.
Here's Hoping, but Dalí Doubt It
Well, in the modern American tradition, it stands accused of carrying out its tasks by drawing mostly the faces of white men. From the point of view of a white man, I'm inclined to back the attacks, and also those from the other side, about the fact that it rarely draws naked bodies.
But all these are biases, and in the world of artificial intelligence they are often considered the most serious problem: the ways in which an algorithm can deviate from its task for some subjective reason.
DALL·E is evidently prone to this, which is why its testing is restricted, and there are countless caveats on that front. And in that sense it is rather likable: it ticks more than a few boxes for an artist… who, it seems, is subjective by definition. And it lives up to its name, which obliges it to a certain originality.
They are even preparing a kind of censorship for it. Some of the specialists who worked on it insist that it should not be allowed to generate faces at all (because when it was asked to draw a prisoner or an angry person, it always returned people of color).
DALL·E, however, is not to blame, and it is capable of impressive results. And before we steer it toward the erotic genre (wonder why), let me hasten to tell you that its creators censored all spicier images during its training. So if you give it a steamy scenario, it might show you a hammer instead of the act we verbally associate with “nailing.”
Bread and Circuses
Every image fed to it passes through a serious filter, so that there are no political symbols and none of the messages considered inappropriate today. Knowing how well social networks handle this kind of filter today, we can only guess how good an idea it is to apply a similar approach to the next generations of algorithms.
It relies on flawless descriptions of the images in its attempt to find the connections between them. To do its job, the algorithm receives the images and the text in parallel, encoded in symbols. While training on the enormous number of images, it discovers patterns in the descriptions. Then it tries to put them into practice within the limits set by the user.
Besides helping people express themselves visually, our friend's other big goal is to give us more knowledge about how AI perceives and processes information. As discussed at length in the piece on
the spectacular dreams of algorithms, we still know far too little about what happens between the input and the final result.
Many see yet another future application: DALL·E may turn out to be the long-awaited tool that brings to life the much-touted, far more perfect and realistic metaverse. According to its first users, today's solutions are rather imperfect, while future versions of the program, a painter on commission, could seriously help the new virtual world become not only realistic but also quick and cheap to build.
Give and Take Dalí
In February, before the algorithm's launch, 23 outside researchers were brought into the team as a so-called “red team”: scientists with serious experience in machine learning, tasked with checking for every possible gap and potential problem. It was they who proposed that the project should initially be placed only in the hands of a select few. So far 400 people have had that luck, mostly OpenAI employees, along with selected artists and scientists.
According to people on the “red team,” our friend isn't capable enough of drawing realistic faces. Well, the real Dalí wasn't exactly strong on Mona Lisas either.
More criticism comes from the claim that the algorithm simply does what any reasonably good Photoshop artist with a bit of imagination can do.
But then, it's an algorithm, isn't it? In any field whatsoever, the phrase “does what anyone can do”… is a sign that the profession is on its way out.
Besides, it depends on what it is fed. The more original the task you give it, the more unexpected the image you get in return. Isn't this the long-awaited form of symbiosis when it comes to creativity?
Imagine all the AI…
And just as this piece of journalism was barely ready for publication, it turned out that our friend is… no longer alone! A company of AI painters is taking shape; they may well form a school of painting, like the Impressionists.
More likely, though, it's competition that's taking shape, because Imagen is the “offspring” of the other biggest player in the world of artificial intelligence today—Google. This painter can handle quite similar tasks, but… it definitely has a rather different style of expression.
Here are some examples of its work too:
It will be very interesting if one day we get free access and can check how the two algorithms would respond to the same original tasks. If the difference turns out to be significant, won't that be yet another argument for talking about… creativity?!
Fight On, Dalí!
The images have definitely drawn attention, and it is no accident that the progress has been achieved by several projects at once. Quite a few opinions are skeptical, because few people have access to the algorithms and can test them. That means only the best results are selected, we're only shown the pictures that turned out well… But didn't human painters do the same?
“DALL·E is a handy assistant that amplifies what people can do on their own. But it is also highly dependent on the imagination of the people who use it. Creators and artists can achieve unexpected results with its help,” says Aditya Ramesh, one of the project's lead engineers.
The last serious skill of DALL·E really turns out to be variations. The opposite of its main task: rather than ordering a drawing and getting it, you give it a picture to draw inspiration from. And it creates a similar one at its own discretion
That is probably something even its namesake couldn't manage.
And what will happen when we let in the advertisers, marketers and influencers? “DALL·E, I know you're an artist, but make me an ad for…”? Every creator of modern times knows how unpleasant that will be for their electronic colleague.
And in just a few days we'll learn more on the subject, when we talk to one of Bulgaria's established photographers, who has had the chance to try out (and show) the incredible capabilities of precisely these algorithms!
“DALL·E, draw me a sheep!” I promise you, if I get the chance, I'll ask it for that too.
Coming soon: another story, only not quite so digital, about the man who wrote the original of that line, Antoine de Saint-Exupéry. After that, a piece on general artificial intelligence. Oh, wouldn't life be beautiful, and wouldn't humanity's future be far more colorful, if we managed to bring them together…!
