“The Game is Over.” That is what researcher Nando de Freitas declared on Twitter in May 2022, announcing big news in the field of artificial intelligence, news that stayed off to the side of all the noise. Scientists were still arguing over whether artificial general intelligence could exist at all. Behind that term lies an algorithm capable of solving a great many different tasks, much like the human brain.
And while they argued, it seemed to turn out that… the argument had already been settled. That terse tweet announced the arrival of Gato AI, the first contender for the title of “artificial general intelligence.” A step that was expected to be at least a decade away, and one that many considered outright impossible.
The news, however, didn’t attract much attention. Unlike ChatGPT, which appeared half a year later and has been monopolizing the spotlight ever since. Well, it managed to turn heads by showing, for the first time, an enormous number of users just how advanced the technology really is. And how we are getting ever closer to the next giant step. One that seems at once logical and… so impossible. Artificial general intelligence!
A General Change
Yes, we all hear its name far too often; by now even the straw in our beach cocktail has artificial intelligence. And yet there’s no denying that this field of knowledge is extraordinarily dynamic, advancing at breakneck speed in many of its applications.
And so we talk more and more often about the next giant step. An algorithm that isn’t merely ready to perform remarkably well at the task it has been given, but is able to solve problems of every possible kind.
Gato came along to settle (perhaps) this long-running debate, to change the game entirely. To show that there can be a universal algorithm for different tasks… one similar to humans. Yes, neural networks, the high art of machine learning, strive to imitate the way our brain works. But while we are capable of solving all sorts of tasks, until this step algorithms were confined to a single one. By all appearances, we are done with that.
Jack of All Trades
When they are given exactly the right task, algorithms are capable of wonders. They can analyze enormous volumes of information and then accomplish their goal. And that goal can be of only two kinds. Either they receive the next “piece” of information and determine what it is, or they try on their own to predict what comes next in a given sequence. That’s all; practically every task we set them today boils down to this.
The same goes for the much-hyped large language models—GPT-3 and 4—and, in full measure, for their beloved sibling ChatGPT. It, too, can impress with a wide range of applications, from machine translation through chatbots to generating computer code. And still: in essence, outside the bounds of that specific task, it also belongs on the list of competitors with a single event. Because what it does is process information and, in response, give us its closest possible guess about what will follow, based on the patterns it has found. It, too, works in one specific area: natural language processing. Artificial general intelligence is the next step.
DeepMind is the Google project that once made Musk worry about the future, because in his view it was being developed without sufficient oversight and had access to the vast troves of knowledge of the universal search engine.
Well, perhaps that warning is worth keeping in mind as we turn to the subject at hand.
King and General
On May 14, 2022, Nando de Freitas posted the landmark tweet saying the game was over. Of course, he added an important caveat: “It’s all about scale now!” In other words, we now have an algorithm capable of solving all kinds of tasks (which is why it’s called general), not just the one specific task it was built for. All that remains is “only” to teach it to work at larger scales than the ones it handles so far.
The news mattered so much not only because many specialists didn’t believe it was possible. But because it marks a decisive step on the road to an artificial intelligence that could be indistinguishable from a human one.
Nobody is saying this will happen tomorrow, but the arguments are piling up that one day the technology will be able to pass the famous Turing test, conclusively convincing its human interlocutor that they are talking to another member of their own species. Even de Freitas admits that we are not as close to that step as we’d like. What matters more, in his view, is that we now know it is entirely achievable.
Alongside the landmark tweet, the team published a scientific paper and a blog post presenting the new algorithm in detail.
Gato is named after the Spanish word for cat. And it is definitely as clever as a tomcat (like our friend Rumen, for example).
Multicooker
This little gif shows the many different tasks the clever cat can handle:
The picture didn’t impress you? Just a bunch of worn-out games from an ’80s computer? Fair enough, a little more explanation is needed.
As mentioned, none of the AI projects existing up to that point had been able to solve tasks of different kinds. And games like these turned out to be the perfect field in which to look for general AI, simply because they require different interactions. Gato turned out to be the first “electronic player” ready to tackle challenges that were child’s play for the teenagers of that era.
On top of that, the cat can write poetry and stack blocks by controlling a robotic arm. One and the same neural network can chat, recognize images, and who knows what else.
The Cat’s the Boss
“From here on, the goal is to make models of this type bigger, safer, optimized, and in line with computing capabilities. We have a lot of work ahead, but solving these problems will lead us to general AI,” Nando de Freitas is convinced today.
Of course, there is also the small matter of figuring out whether, in achieving this, we might be creating a program that will wipe us out as a species.
“Safety is of paramount importance in this kind of research,” Gato’s creator admits. “We must always be able to stop an algorithm like this when it starts making decisions that are too important. And that may be the biggest challenge before us.”
Google has reportedly already developed a special system, a big red button of sorts, so that such problems can always be prevented. So that the AI can be “switched off” if we decide it threatens us in some way.
Here, Kitty, Kitty
Gato can not only perform more than one task at the same time, it also does so reasonably well. In some of its “professions” it rivals algorithms specialized in them, but, alas, that is far from true for every task.
The team tested its contender on 604 different tasks, and the cat completed 450 of them with a success rate above 50 percent. High, but still a long way from what is being sought.
Because of the way it works, Gato is considerably “weaker” than many of its rivals. The basic unit by which algorithms of this kind are “measured” is the number of parameters in the neural network. While GPT-3 has 175 billion, Gato “learned” with just 1.18 billion.
Puss in Boots
Naturally, its creators’ claims were followed by quite a few conflicting opinions. According to some researchers, Gato falls far short of the requirements for us to accept it as a forerunner of general AI. It is, rather, a combination of straightforward models pretrained on data. Yes, it can perform 600 tasks at once, but that makes it more like a console with 600 games than a game that can be played in 600 different ways.
The approach to building it is interesting: the cat was pretrained on all sorts of varied data—texts, images, maps—converted into tokens. This runs against the usual approach, in which a neural network’s “homework” contains only objects of a single type, images, for example. And the fact that it still produces good results opens up yet another line of thought for scientists. It gives us additional insight into the way deep neural networks work.
Tom and Geoffrey
Talk of artificial general intelligence grew even louder with the explosive popularity of ChatGPT. Some of the pioneers and biggest names in the field, such as Yann LeCun, are skeptical that it is near. Until recently, Geoffrey Hinton was also on that list, but… the dramatic statement with which he announced his retirement made it clear that this is no longer the case.
Of course, sometimes it comes down to definitions; there are quite a few of what exactly “general” means, and at times they contradict each other. According to some, it is the ability of an algorithm to perform every mental task a human is capable of. And here we are still light-years away, when it comes to creativity, free will, irony, and abstract thinking, for instance.
Another definition sets five specific requirements for the title of “general AI.” It must show creativity and understand natural languages. It must perceive sensory information, have good motor skills, and move successfully through space. With the exception of creativity, the rest sound fairly achievable along the path the clever cat has taken.
A third definition of the term simply requires an algorithm that can perform complex and varied tasks. Here Gato is definitely already a good candidate that meets the requirements.
And does this step truly bring us seriously closer to the bigger goal—an algorithm approaching human abilities? We can still speculate more than we can be certain of anything. The only certainty is that we live in interesting times, and we hope the world of technology will bring big news, but above all good news… So that we never arrive at the other meaning hidden somewhere in the words of Gato’s “father.” “The Game is Over.”
Chat by the Hour
It is worth noting that OpenAI, the creators of ChatGPT and all the other GPTs, openly admit that their long-term goal is precisely artificial general intelligence. With a noble rather than a commercial aim, of course. As the company’s manifesto puts it, “our mission is to ensure that artificial general intelligence—AI systems that are generally smarter than humans—benefits all of humanity.”
It then goes on with even more “specifics”: “AGI could happen soon or far in the future; the takeoff speed from the initial AGI to more powerful successor systems could be slow or fast.”
Recently, scientists from Microsoft announced that in their view GPT-4 shows “sparks” of artificial general intelligence. According to the team, the new generation of large language models displays more general intelligence than previous models. “Beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting,” they write.
Well, their opinion was met with a fair amount of skepticism, and one can hardly help accusing them of being less than impartial, given the close ties between Microsoft and OpenAI in recent years.
The Parrot in the Pan
And so we have changed the definition quite a bit once again… But the big question remains: is it possible to have a next generation of systems that are not just “expensive parrots”? Caught up in the two extremes—gushing over the enormous progress of this kind of technology or dismissing it—it is usually the nuances that get sacrificed.
That is why, before we part, I’d like to repeat the fascinating view of a leading researcher on the subject. When I asked him about artificial general intelligence specifically, Prof. Preslav Nakov replied: “We’re not ‘there’ yet, but I’d like to believe we can be. The systems are certainly becoming more intelligent, we can see that.
The question is rather what we want to achieve. There are two main schools in artificial intelligence: strong and weak. The weak one says: ‘We want to build systems capable of solving tasks that are considered to require human intelligence.’ The strong one says: ‘We want to build systems that solve tasks the way a human would solve them.’
And here there is a big divergence. The strong school focuses on whether we are really achieving what a human does: do we have deep understanding, abstraction? The weak one exploits the fact that we have no need to imitate the human. If we want to multiply 17-digit numbers, we can do it much more easily and quickly.
The weak school says there are tasks that call for human intelligence, such as recognizing speech or handwritten text, generating or summarizing text. And we solve the task, regardless of whether it is done the way a human does it.
We have enormous success in this weak direction; the question is whether that also gives us progress in the strong one. I believe we will start making progress there too. But it will be slow. These large language models haven’t built some higher form of AI, but on some level they are moving in the right direction. Because there we are talking about solving a task posed in human language, and that is a qualitatively different pursuit.
What we need right now is better understanding. And people realize it. There are limits to computing power, to memory, to the data we can obtain.
We are at a moment of transition from systems that are very good at a specific task to ones that are more universal and to which I can explain in human language what I want. I want a machine that is universal, because the example is the human, and a human can solve many different tasks, and that is part of intelligence. Including the ability to carry experience over from one task to another.
At this stage, the toolkit we have isn’t ready, but I think we are asking the right questions and setting the right goal. So in the long run, I’m an optimist.”
