Artificial intelligence has many dimensions these days. It is often talked about with unwarranted optimism, while it is already capable of wonders in many fields. One of the main challenges that will certainly determine its place in our lives is whether it can create something of its own, something different. Original or not, but something that could be called art.
One of the key technologies that have offered a new perspective on the subject in recent years proposes a particularly interesting solution. It pits two sides of neural networks—the next generation of computer algorithms—against each other. The first tries to complete its task, and the second does its best to guess how well the first has done.
How does all this happen? Why are GAN networks among the favorites in the race to the moment when artificial intelligence will be able to create human art? What other wonders… and problems might they bring us?
Good Fellow
Ian Goodfellow was only 30, a junior specialist at Google, when in 2014 he and his colleagues published the fundamental paper in which he proposed the idea of GAN. In the years that followed he came up with other impressive innovations in artificial intelligence. After gradually climbing the career ladder at Google, in 2019 he accepted an offer to head the artificial intelligence unit at Apple.
The young scientist, whose surname translates as “good colleague” or “good friend,” is today one of the best-known scientists in the world of artificial intelligence, and he remains most recognizable precisely for his idea of GAN.
The abbreviation stands for Generative Adversarial Networks.
What do they generate? What don’t they!
And whom do they compete with? Only with themselves!
The Good, the Bad and the Algorithm
Neural networks are combinations of algorithms arranged in layers, which process the units of information fed to them one by one and pass them on to the next layers. This makes it possible to detect all kinds of patterns with extraordinary precision.
In some directions the field advances at astonishing speed, in others it fails miserably to live up to unwarranted expectations. And yet, over the past decades it has certainly been one of the most dynamic and dizzyingly fast-developing areas of science.
The big breakthroughs did not begin as soon as deep neural networks saw the light of day. Their real beginning is considered to be Yann LeCun’s famous prototype from 1989. Back then, however, computers were far from having the necessary capabilities, and the revolution had to wait some 20 years. Over the last decade it has become a fact, with new large-scale directions being born one after another. We have already written about the curious hallucinogenic “dreams” of algorithms, about recommendation systems, about facial recognition.
Well, it is safe to say that the concept of generative networks opened up entirely new paradigms.
The idea is this. The neural network receives a sufficiently large volume of data, on the basis of which it trains two models in parallel. One is generative—it analyzes the data it has received—and the other is discriminative: its job is to evaluate how well the first model manages to capture and recreate the patterns. It accepts only the part of the results that it considers so precise it could “mistake” them for ones from the real database.
A GANius Idea
In other words, within the model we have two figures known since the dawn of humanity—the creator and the critic. The first part of the algorithm does everything in its power to discover the patterns in the database and recreate them in its own way, and the second judges what has come out right and “throws out” the rest.
Today human faces are recognized beautifully by deep neural networks. But what happens if the task gets harder and changes into creating, on the basis of the patterns found, a new human face that does not exist?
Here is one. A GAN network “made it up.” And here is another:
If you’re having fun, you can come here and carry on yourself. Every refresh of the page will show you a new face. At some point you will notice small inaccuracies that give away that something is unreal, but on the whole they are exceptions.
They happen mostly because the algorithm does not perceive, for example, jewelry such as earrings as a whole and “chops them into pieces” in its search for larger patterns.
Either way, in the vast majority of cases you will see for yourself that the GAN network does a flawless job.
And how do you like this kitten? You have probably guessed by now: it doesn’t exist either.
The same principle can be used for all sorts of things. Try it with kittens here, with horses here. The results will definitely surprise you.
The GANg of Cats
All of them were produced in the way proposed by Goodfellow and his colleagues. The neural network received a huge number of photos of cats. The generator tried to analyze the patterns and create new cats on its own, and the discriminator managed to judge precisely which ones met the criteria and really looked like the cats in the main database.
Seen up close, this is a series of algorithms that analyze the images layer by layer and carry out the two assigned tasks. Seen from a distance: a program that can successfully generate units of information—not necessarily photos—that resemble the ones it was fed with considerable success, without repeating them.
This breakthrough in artificial intelligence is certainly impressive. We won’t get into whether the newly created kitten is a work of art and who holds the copyright to it, because we have already dwelt on that subject here.
Vincent van GAN
What is more interesting here is what GAN networks are capable of and in which directions they are developing. Among the series of sites with faces and kittens there are also ones for images of chemical compounds… and of works of art. Yes, the latter look nothing like what they are supposed to be, unless we reduce art to some of its most abstract and modernist forms, known in northwestern Bulgaria by an earthy dialect word that roughly means “arse-scribbles.”
In its original application, the GAN network successfully imitates images that are relatively standardized. True art, however, turns out to be too big a bite for it, whether because there aren’t billions of brilliant paintings to learn from or, more likely, because the patterns there cannot be reduced to something so one-dimensional.
After the publication by Goodfellow and his colleagues, GAN networks became a popular solution in the world of machine learning. As often happens in technology, some time had to pass before it became clear which applications were the most promising.
Networks for Newbies
Images really are the first and obvious application. And the big thrill is certainly in creating art. Still, it seems to me that the serious breakthroughs in this area have not yet arrived. There are exhibitions of art generated by GAN networks. They often emphasize that images are generated automatically at the exhibition itself, shown only once and then gone. And yet the paintings are still far from convincing; whether they ever will be depends on the technology’s next steps.
No less interesting, and considerably more successful, are other applications, for example in astronomy. The technology is used to process images from space, for instance to simulate gravitational effects in the study of dark matter. It helps in other areas of physics as well: GANs are used to simulate expensive and complex experiments in the world of elementary particles.
In biology, for more than 5 years now, GAN has been helping to simulate cellular processes and to generate drug molecules that can be used in the fight against cancer.
GAN Over
Back in 2018 another interesting idea was put forward: using the networks to turn the cult low-resolution computer games of the ’90s into a 4K spectacle with far more realism. Of course, this is not an application vital to humanity, but it is quite useful to the genre’s many fans.
Again in the world of pictures, GAN networks make it possible to create photorealistic images related to industrial design, furnishings, even fashion. So that realistic prototypes can be created far more successfully, without wasting time making each one physically.
These technologies have also turned out to be very well suited to photography. Photoshop may have changed forever our attitude toward photo editing and how faithfully photos depict reality. But there are still directions in which it is powerless. And not only when it comes to enlarging small images, as happens with computer games. Prototype GAN projects that can restore badly damaged pictures are impressive. Or ones that colorize black-and-white photos hyperrealistically.
Also—realistically turning photos into paintings in the style of a particular artist, and the reverse: turning cartoon characters or paintings into realistic photographs.
A curious idea: the GAN model Speech2Face tries to recreate a person’s face just by hearing their voice. Quite human, I would say…
PropaGANda
And so we come to the dark side of this technology. For which, of course, neither the technology itself nor the “good fellow” who gave it to us is to blame. We have already written in detail about deepfake technology, which today can even spark wars. It was GAN networks that proved to be the platform that delivered the most serious progress in this direction. That, however, opened many social, political and psychological questions. Because you can see ever more perfectly on video how people say things they would never even think of. Not to mention… realistic pornographic content in which the participants have been flawlessly swapped.
Among the other potential problems is the easier creation of fake profiles online, thanks to the faces of nonexistent people. Compared with the other risk, though, this one looks rather more harmless; at the very least, it is hard to maintain a profile with more than one photo of the “person” in question.
GANstradamus
Anyone with a somewhat deeper interest in programming and neural networks can “play” with their own GAN. Here, for example, you will find my experiments with a simple network whose task is to create handwritten digits. Given the modest power of my computer, the result is actually pretty decent:
In the world of the looming metaverses, everything should very soon become more virtual, and GAN networks are finding their place here too. Many projects rely on building environments out of details created by this type of technology. They are fed real footage captured by multiple cameras, similar to Google Street View. The same goes for the increasingly realistic avatars through which every visitor will embody themselves.
What comes next? Ask a GAN network built to analyze forecasts of developments and events based on history. What, there is no such project yet? Maybe no one has just thought of it… As Goodfellow and his colleagues’ project shows, in the world of artificial intelligence, not all the good ideas have been thought up yet—far from it.
