What happens when artificial intelligence takes on its next big job: creating… artificial intelligence? What if we asked the famous ChatGPT to “create” a new neural network to solve a specific problem? Is it possible for complex algorithms to create their own successors? What would change as a result? Might this not lead us to the next step: an absolute AI, capable of solving any task at all?
These questions have many possible answers, but while we are asking them we may miss something important: systems already exist in which AI oversees and assists the process of creating its own “kin.” What is more, it does so in order to build solutions that are more secure and safer for people.
The Good, the Bad and the Ugly Robot
Ever since the Terminator arrived—and in fact several millennia earlier, thanks to the god from the machine—humanity has had one and the same ultimate worry about its relationship with machines. That they will take control. That they will manage to make conscious decisions, and it will not be within our power to stop them. The archetype not only survived but blossomed in the years when the famous term “artificial intelligence” took hold.
We have already asked whether the day is approaching when algorithms will learn to create software on their own. In one way or another, this is already happening: they spare programmers quite a few tedious tasks. Even if they are still far from real software engineering in all its fullness of multilayered problems.
Then comes the next important and interesting frontier. What happens if algorithms are capable of setting themselves the next task? Of programming and developing artificial intelligence.
Even Grandma Knows
Self-replication is a phenomenon we always associate with biological organisms. In other words: if artificial intelligence achieves it in one form or another, that will certainly be one more sign that our artificial creation keeps scoring points. That it keeps taking steps forward in its evolution.
The singularity is that moment when computers turn out to be “smarter” than us. Naturally, whether we ever reach that stage is a subject of serious debate, and we have already discussed it in detail.
Both of these—self-replicating artificial intelligence and the singularity—are concepts from a possible future. In this field the enthusiasm sometimes definitely gets out of hand, so let us step back from fears and dreams to the present day. To tell you about several interesting projects showing that algorithms can not only help create their kin but are already doing it. And not instead of people, but by helping us where they are stronger than we are: in their impartiality.
Who Will Guard the Guards
Once again we have to start with a short definition of what the famous AI actually is. Yes, when we talk about intelligence, the first and natural association is always with the human mind. The system that may one day resemble it, however, has a specific name: “general” or “absolute artificial intelligence.” Without that important first adjective, what we have is simply software that in some way resembles human thinking, ready to solve a specific task. Over the past few years this has most often been done through machine learning and the famous neural networks.
But even that term is rather misleading. The machine sits there teaching itself poetry by reading Goethe… Alas, nothing of the sort. We are talking about an algorithm that can process enormous volumes of information and look for patterns. Through carefully chosen parameters, it finds out which indicators matter. And on the basis of all this it solves one specific task, which always boils down to two options: to predict what the next unit of information will be, or to classify the next unit according to various criteria.
At first glance a simple solution, yet in some areas it has genuinely achieved the kind of perfection that earns it the label “intelligence.” For example, when it comes to facial recognition, driving autonomous cars, even creating works of art.
In some form, neural networks do exactly the same thing. The difference is that they involve a series of algorithms arranged in layers, which can discover far subtler patterns. But once again, you cannot say that one algorithm creates the next.
We Will Build a Factory
How, then, do we get from these principles to the next step? To artificial intelligence helping create its own successors?
One of the most serious problems in developing algorithms of this kind is reducing prejudice, which you will also encounter under the English term “bias.” More or less, how well an algorithm can do its job depends on how its initial parameters are fed in and how the individual elements or layers of the neural network are chosen. This is also where the big ethical questions arise.
This is precisely the main and hardest work of specialists in the field today. In the early stages, before actual machine learning begins, the creators of artificial intelligence very often proceed by trial and error. So what would happen if other algorithms replaced them in this foundational task?
Caught in the Net
With AI, everything turns into numbers. Whether we are talking about video or simulations in drug development. Sometimes training the models requires millions or even billions of parameters to be processed precisely and repeatedly before the software finds the right patterns.
Sometimes, if the volume of data to be processed is large, this can take even months, despite the enormous computing power of modern machines. That is why one of the first directions in which next-level AI projects are expected is creating systems that can themselves skip many steps. That can optimize existing algorithms and simplify them. The field is called “hypernetworks,” and it has definitely been one of the popular directions lately. Hypernetworks analyze the data and the corresponding untrained neural networks and can make the learning process considerably easier.
Among the pioneers in this direction is a team led by Boris Knyazev of the University of Guelph in Ontario, Canada. In 2018, he and his colleagues developed a graph-based hypernetwork whose goal is precisely to find the best neural network architecture for a specific task. The next generation, GHN-2, appeared a year ago and is able to take in and analyze the parameters of 500 different networks and match them with the possible next tasks for which a solution is being sought. In other words, if the system works as well as it promises, it saves 500 network trainings that developers would otherwise do one by one, and working time shrinks drastically.
I Have Raised Myself a Monument Not Made by Hands
All this has many dimensions, not only in terms of time and effort but also in terms of saving the huge energy resources that supercomputers often need to run.
The truth is that we know far too little about what actually makes deep neural networks so effective. As we have mentioned before, we feed them data and rejoice at ever more successful results, while they remain a black box—we do not know exactly what happens in the middle. But things get more complicated if we add hypernetworks. If they happen not to do their job well, we have no way of knowing that it happened, and above all why. We will have bias, raised to the next level…
A Robot Weaves a Fence
Let us go back to the good old Terminator for a second. After all, what we had there was an encounter between two absolute artificial intelligences—one wanted to destroy humans, the other to protect them. Well, we now have something similar as a solution in today’s AI, too. A startup with Bulgarian involvement is developing a neural network that watches over others of its kind and protects us from them getting out of control…
The startup is called Tenyks, and at the start of 2022 it raised $3.4 million in funding with its unconventional proposition. Serious names appear among the investors, for instance John Taysom, the first person to back Yahoo financially, in 1995.
The founder of the company and author of the idea is Botty Dimanov, a doctoral student at Cambridge. The platform developed by him and his two partners is aimed at developers in machine learning. With the help of their application, they can build their projects faster and more safely.
The platform allows AI-based systems to be tested much more easily and reliably. To get through the initial analysis of the data they receive more quickly, so as to create more targeted solutions in the chosen field. The goal is also to develop models that are more independent of human error, of being misled by the initial information depending, for example, on our personal preferences.
Hooked on Bias
Tenyks takes the models developed by programmers and evaluates them according to various parameters. It looks for their weak spots and for where that notorious prejudice shows up—the “bias” specialists so dislike. In other words… once again we have an AI that directly helps create its kin.
“Every time technology reaches the limits of the impossible, the next category of software solutions appears. Computers developed their graphical interfaces, and the internet its search engines. That is why I believe AI will have similar platforms that will allow us to take its development to the next level,” Botty Dimanov told the site The Recursive.
One of the reasons the subject is ever more topical is the way the data itself is processed. “AI development today is shifting from work centered on the model by which information is processed to work focused on the information itself,” says Rick Hao, one of Dimanov’s colleagues. This is exactly where the team’s application comes in—it helps engineers with the initial processing and with analyzing the steps to take in order to improve how the algorithms work.
Robotic Zen
Robots don’t understand poetry? We could argue about that for a long time (and we have, here). “Paired Open-Ended Trailblazer” sounds more like Vogon poetry, but it abbreviates to POET. This is the project of Uber AI specialist Rui Wang, and once again it is connected to our subject.
In short: the idea is again to create a hypernetwork, but one that tests and builds on algorithms in a different way.
Since the author comes from the East, he describes the way his system works as a kung fu contest. The algorithms are given tasks and pitted against each other, and not only does it become clear which is better, but each of the “contestants” progresses and develops in what it does. “In the end, a complete, true kung fu master walks out of the training dojo,” Wang says.
The system starts with a simple task, moving on to each next stage with new knowledge and facing different obstacles. When it succeeds or fails at a given stage, the “competitor” moves into an entirely new environment. The decision to change the environment at random turns out to be a particularly successful factor.
“POET is based on a paradox,” Wang says. “In life it happens that if you try to solve a given problem, you will fail, and if you don’t try, you are much more likely to succeed. Sometimes that is exactly how evolution happens in nature, so it is possible to achieve something similar with artificial intelligence.”
Darwin Makes Darwin
According to the team, it is exactly projects like these that lie on our path toward creating the long-awaited absolute AI. One that will be able to solve various tasks, not just one specific task, as, we have said, is the case today. So however distant the future of such developments may seem, their roots are already here.
“It is time to start creating algorithms that contain a certain form of intelligence from the outset and to watch them create the next ones on their own until, eventually, we reach absolute AI,” says Jeff Clune of OpenAI.
“The truth is that for now it is just a fantasy. But that is mainly because no one knows exactly how to create it. Progress in this field depends on people and happens step by step. Solutions that have already been discovered and are familiar get combined in different ways, gradually improving their success rate.” Clune says that in this way we are creating ever better individual building blocks, from which we will struggle to construct the building of absolute AI on our own. “And perhaps it falls to the algorithms to take on the hard task of putting it all together.”
Small Children, Small Troubles…
And what happens after that? As Jane Wang of DeepMind says, one of the main advantages of involving AI in creating AI is that it will come up with design ideas and techniques humans would not expect. “But not all surprises are good ones. If you harness AI to create something you yourself don’t understand, it is much harder to control it and to predict where the problems will crop up.”
The ventures we have told you about are the decisive first steps toward creating absolute artificial intelligence. But are we ready for the next ones? Not so much technically as civilizationally? We will keep looking for the answer together in the next Digital Stories…
