“AI Is Handling More and More Tasks. The Question Is What We Want Next”

– How did you end up choosing natural language processing as your research direction?

– I've always been interested in different languages.

I speak 8 languages fluently and have studied 4 more,

but it happened by chance. I was at University College London on a European program when my professors advised me to find some area to focus on and do something nobody else had done. I was still a student when I came back to Bulgaria, looked for specialists in the field and found Galya Angelova from the Bulgarian Academy of Sciences (BAS), who later became the supervisor of my diploma thesis.

People in computational linguistics came from two directions—either linguists who gradually moved into computational methods, or the reverse, because the field sits on the border between the two. But today the linguistic aspect has shrunk somewhat because of the development of deep neural networks, where the morphology of words and the structure of sentences don't matter that much. Everything can be modeled with complex models that learn about all levels of language. The results are very good.

– Downright impressive, and even that isn't a strong enough word. So good that we don't even realize it. I ran an experiment with creating, editing and retelling text, and became convinced that many professions in this area are already practically unnecessary. Even though we still don't grasp it.

– In recent years models have appeared that started a real revolution. 2019 was the year of BERT, and for several months all people did was run it on new and new tasks and almost every time get an improvement over the best results so far. By the time this paper was officially presented, over 90 percent of scientific publications were already using and citing it, since it had been released freely on arXiv at the end of 2018. There's a Bulgarian trace there: one of the authors is Kristina Tutanova. BERT comes from Google; RoBERTa is a similar model and Facebook's answer, and it also has a Bulgarian co-author—Veselin Stoyanov.

At first OpenAI didn't want to release GPT-2, because it is a perfect fake news generator. Then came GPT-3, which is used for all sorts of things—from generating essays through machine translation to writing computer code. Lately there has also been serious progress in image generation. There are problems there with how to adapt the text to the local culture—for example, the idea of what a boat, a coffee or a tent is changes quite a lot across cultures.

– I ran an experiment with DALL·E 2, MidJourney and others, and it's very interesting how systems that generate images from a description head off in radically different directions. That's probably less a result of the models themselves than of the data they were trained on, isn't it?

– In principle, yes, and it's a serious problem. When I was experimenting with GPT-2, I asked it about the coronavirus and it started telling me about SARS and other diseases, because it was trained on older texts. If I want to generate a story about Sofia, it won't manage, because its knowledge is limited.

There are also problems with various stereotypes connected with the way the model sees particular nationalities or countries, because that's the point of view in the texts it was trained on. People say these algorithms are biased. Sometimes that's true, but in most cases the problem comes from the data. Take, for example, a system that decides whether to grant a loan. The historical data may be such that people from a certain ethnic group received more rejections.

What can we do? If we train the algorithm on historical data, it will reproduce the statistics because of past practice. And there are different solutions here. We can adapt the algorithm so that it takes measures against these “tilts.” We can balance the data in some way, or take new data that has the characteristics we think we need.

– It's curious what data these algorithms are trained on. With GPT-3, 90 percent is texts randomly gathered from the internet plus a little Wikipedia. We don't teach it from dictionaries, textbooks and academic works, as would have been the case some time ago, but with the most ordinary, broad and varied information. The other big surprise is that GPT-4 will have fewer parameters than GPT-3.

– The race for more and more parameters had to stop sooner or later anyway. Moore's law says that hardware capabilities double every 2 years. Whereas if we look at the way models are growing, their requirements double every 3–4 months. Until a few years ago hardware kept pace with the growing requirements, but lately that hasn't been the case.

It's true that with more parameters the algorithm “knows” more. GPT-3 can be used for many different tasks with the help of additional tuning. For example, if our goal is to write a CV automatically and we have a few hundred examples, we can feed them to it and thus adapt the model to the task, which takes a few steps of additional training. Smaller models couldn't work like that.

And when we have an enormous number of parameters, we can use the model for many things directly, without additional changes—for example, to generate blog posts.

Today, very often,

the most important requirement turns out to be giving the algorithm a good description of the task.

– And that's a very interesting way for our intelligence and the artificial kind to interact…

– It's the natural development of processes in this field. It begins in the 1980s with expert systems. Back then people spent years talking to specialists in a given field, then hand-coding the rules learned from them. Everything works wonderfully until the number of rules grows: then, if we want to add one more rule, it's not clear how it will interact with the existing 5,000.

Then comes machine learning. Here we don't ask the expert how they solve the problem; instead we give the computer examples of solutions. Here, this sentence is translated like this; I'm giving you a million sentences with a hundred million words, and you won't need to know how each word is translated or what the rules are—you'll learn it from the examples. The problem here is hand-coding the features we want to observe and that matter for solving the task. For example, if we want to classify text into categories like politics/business/sports, we usually focus on words that carry their own meaning (nouns, adjectives, verbs, participles) and exclude punctuation and stop words like “and,” “of,” “for,” “him,” which only add noise. But if we want to recognize who the author of a given text is, we have to observe exactly those features, because they express the author's style well.

The third wave is deep neural networks, which work directly with the text and need no rules or features: the model learns by itself which features are important. There's still no free lunch, though: here the problem becomes how to choose the values of the hyperparameters. We can think of these hyperparameters as different “knobs” with which we can fine-tune the models, and if we do it well, they work wonderfully.

But if we look at things from a little higher up—people don't learn only from examples. If we want to solve a given task, examples are useful, but we also need to know what the task is. If we're given a list of restaurant reviews annotated with two classes, 0 and 1, it's not clear what is expected of us: whether to recognize in the two values the positive and negative reviews, or whether the review is about pizza or a cooked dish.

This is the very

powerful idea that came with these large models—that we can explain the task to the computer with a text description in human language.

So instead of teaching every system every new task, we have one universal artificial intelligence to which we explain what we want, give a few examples, and it returns an answer. For me this is a qualitatively new way of thinking, and it is the fourth wave.

– Each of these things still works poorly in Bulgarian, including automatic translation. How can this problem be solved?

– Our language is privileged when it comes to machine translation, because Bulgaria is an EU member and huge volumes of documents are translated into Bulgarian. But that's a very limited part of the language.

One answer is that things will improve as more training data accumulates. The other answer is that it depends on how interesting the Bulgarian language is to the big software companies. Let's recall the situation with localization. Some time ago the big software products—Windows, Office—came out in English and a few other languages, and we wondered when a Bulgarian localization would come out. The answer was clear, and that's exactly what happened: when there are enough users buying licenses for it to make economic sense. Things are similar here:

if it makes economic sense, someone will do it.

– Will we one day arrive at general AI? Because GPT-3 looks a lot like it…

– There's been a lot of talk about that lately—there was news both from Google and from OpenAI. Personally, at this stage I'm rather skeptical. We still haven't achieved general AI.

We have very powerful models, but they have no understanding of language.

At some high level these are very intelligent search engines. They can learn patterns, work out some connections and find different pieces that need to be put together to reach the answer. But they have no real understanding of language, of the semantics of particular words, still less of pragmatics.

Until a few years ago neural networks were considered suboptimal and were neglected. Then new neural architectures were invented and things changed.

We began to get closer to solving tasks much better. Specialists started wondering whether computers had become better than humans at a wide range of linguistic tasks. The GLUE benchmark of about 10 natural language tasks was devised, and in less than 2 years algorithms could already solve them better than humans. Then the Super GLUE benchmark was created, supposedly much harder, and the models “beat” it after just a year. Some scientists began to ask whether we really weren't approaching something like a superhuman.

In reality, although many of these models handle their tasks remarkably well, their accuracy drops significantly when we put them in a slightly different domain. They learn to solve a particular task very well, but if we step a little to the side, things aren't like that.

It's the same with self-driving cars. We want the model, if a cow appears on the road, to recognize it and go around it. And it learns, it recognizes the cow on a dry road, in a meadow… and at some point it turns out it can't tell it apart if there's a red bus behind it. For us humans the background doesn't matter, but the algorithms somehow have no understanding of the whole situation.

They work incredibly well, and at the same time elementary things can confuse them.

A picture in which a few pixels have been manipulated fools the model into thinking it's no longer a stop sign but a different sign. A similar situation can arise if there are stickers on the sign, or if someone has turned it slightly sideways… To our way of thinking, this is inexplicable.

– So in some way the algorithms lack a more holistic understanding of the situation.

– And of semantics, too. There are different definitions of “artificial intelligence”; today every intelligent system is called that, whereas it wasn't so in the old days. There were also periods when “artificial intelligence” was a dirty word; now it's the opposite,

we are in a phase of excessive expectations, which are largely justified, because the new models work very well for a wide range of tasks.

But for what we have at the moment, “artificial intelligence” isn't an accurate term; it's more correct to speak of “machine learning.” We have models that learn from examples and on that basis make decisions, generate text, do classification and so on.

To speak of real intelligence, there's the Turing test, and others too; their job is to give us the answer.

– If we go by Turing, haven't we achieved it long ago? If a person sits down to chat with GPT-3, there's no way to tell whether there's another person or a computer on the other side…

– I'm not sure. The Turing test officially still hasn't been passed; there's an annual competition, and a few years ago the press went wild that a system had passed it. That same day a number of well-known scientists from around the world began to dispute it. Nowadays it's considered that the Turing test officially still hasn't been passed.

Besides, we

have to be careful with tests and not treat them as absolutes,

the story with GLUE is telling. If we manage to beat humans at a given task, what does that mean? That we have a machine that is better than them, or simply that we need to make a better test that more adequately reflects the machine's actual “intellectual” abilities?

Since “artificial intelligence” as a term is already occupied by machine learning and by systems that act intelligently, a new direction is beginning to develop that asks precisely these fundamental questions—do we actually have real, fundamental understanding?

The new discipline is called “general AI,” and it asks exactly whether we're there.

– And can we be “there” in the foreseeable future?

– Yes, I'd like to believe we can. Systems are certainly becoming more intelligent—we can see it. The quality of both machine translation and text generation is improving.

The question is rather what we want to achieve. There are two main directions in artificial intelligence: strong and weak. The weak one says: “We want to build systems capable of solving tasks that are thought to require human intelligence.” The strong one says: “We want to build systems that solve tasks the way a human would solve them.”

And there's a big divergence here: the strong direction focuses on whether we're really achieving what a human does, whether we have deep understanding, abstraction. The weak one exploits the fact that we don't need to imitate humans. If we want to multiply 17-digit numbers, we can do it much more easily and quickly.

The weak direction says there are tasks that require human intelligence, for example recognizing speech or handwritten text, generating or summarizing text. And we solve the task, regardless of whether it's done the way a human does it.

We have enormous success in this weak direction; the question is whether that also gives us progress in the strong one.

– What's your personal answer?

– I think we'll start making progress there too.

But it will be slow. One reason is that we don't know how to do it.

We know how to solve a specific task that requires intelligence: first we collect lots of data, train a model, observe where it makes mistakes, focus on whether something in the model needs to change, whether we need to give it more and different data, and so on.

And that's a working solution for getting a better system. But does it bring us closer to a system that is truly intelligent and can solve a wide range of tasks? I'm not so sure.

GPT-3 is a very good direction of development—why not explain to the system in human language what we want from it? Let it learn not only from an enormous number of examples but also from a description of the task.

These large language models haven't built some higher form of AI, but on some level they're moving in the right direction. Because there we're talking about solving a task posed in human language, and that's a qualitatively different pursuit.

What we need right now is better understanding.

And people realize it. There are limits to computing power, to memory, to the data we can get.

We're at a moment of transition from systems that are very good at a specific task to ones that are more universal and to which I can explain in human language what I want. I want a machine that is universal, because the model is the human being, who can solve many different tasks, and that is part of intelligence. Including the ability to transfer experience from one task to another.

At this stage the toolkit we have isn't ready, but I think we're asking the right questions and setting the right task. So in the long term I'm an optimist.

– What do the next steps look like if we head down this path?

– Every science is empirical at first: we accumulate experience, knowledge, observations. We simply watch what happens, then try to formulate a theory and reach some higher understanding. After that we use the theory to solve various tasks better.

Right now we're at the stage of accumulating experimental results. Most of these models are very new, a few years old, and we're still exploring their limitations. Then we'll start trying to fix them through engineering, and in parallel build theories.

I think we need to go back

and take a little of the expert knowledge from expert systems, combine it with the enormous amount of data, and add the ability of the algorithm to learn from different types of data—to bring the four waves together.

We humans learn from all these things. The opinion of experts helps us, as does accumulating lots of experience, looking for connections, and finally abstract thinking and looking for connections between very different fields.

– You say you're an optimist, but for many people development in this direction is frightening. Are there directions that frighten you too?

– Given the current state of the technology, I see no reason for concern. Yes, a moment may come when it becomes frightening. Musk is among the people who worry, but others, like Google's Eric Schmidt, disagree.

For me, at this moment it's a theoretical question. We don't have computers with their own consciousness, but rather systems that solve tasks.

This will develop in the longer term. I don't know whether it will ever get that far—maybe. But

in my view this will be a lower-priority direction of development,

on the one hand because it's harder. To get there, we have to solve all the fundamental questions of artificial intelligence, and progress in the true sense of the term is gradual.

On the other hand: who would pay for it? Where is the economic interest? When it comes to machine translation or self-driving cars, there are companies and users ready to pay, but

who is ready to pay for a machine that has its own consciousness?

– Who indeed? And will we someday regret it, or be glad, that nobody invested in this direction? I'd be happy to meet again in a few years and talk about these topics…

Together with his colleagues in Abu Dhabi, Prof. Nakov is building a kind of oasis for artificial intelligence development. The university is already ranked 24th in the world in its field, a leader in machine learning, computer vision and natural language processing, and prominent at the world's conferences on AI (for example, with 27 papers at NeurIPS’2022).

The team is recruiting master's and doctoral students to learn from leading world specialists, and the doors are wide open to Bulgarians.

If your superpowers lie in this direction, apply or write to [email protected].