“Arti­ficial Intel­ligence Will Simply Make Our Work More Pleasant”

– In recent months, since ChatGPT appeared, people have split into two extremes. Let's call them “it will take the bread out of all our mouths” and “it's good for nothing.” You're in the first camp: you say this kind of technology will affect every profession in some way.

– Yes, but with a note of optimism,

I don't think it will take our bread, it will simply make work more pleasant and fulfilling.

This kind of technology will be a tool, an assistant. It sits next to you, it's at your disposal, and at the same time it is infinitely patient. It has an infinite memory for facts; it remembers countless things. It won't take the job of an experienced programmer, but it will make the tedious tasks disappear, or at least shrink drastically.

– We're seeing an incredible boom in this field. But so far every technology, after such enormous growth, has gone through a serious slump. Could that happen to the much-hyped artificial intelligence as well?

– It has already gone through moments like this several times. In the 1960s, when this kind of technology was in its infancy, the expectations were sky-high. I think the same thing will happen again now with large language models.

Because right now AI is not being presented objectively.

Many people take a superficial look, brush against the topic, draw some conclusions and write them up. That inflates the bubble, and society builds up unrealistic expectations.

At the same time, artificial intelligence is very close to the moment when it will deliver real value and won't just be a fad. We're very close!

– I'd say that, purely technologically, we're already there. And the time has come to think about what exactly we can use it for, and how far we can go.

– We're there in some areas. In generating and processing text, for example, we're very close; it's a matter of time before people start using these capabilities and turn them into something useful.

In many other directions we're further away. These models have a very shallow understanding of the world as a whole, and there's a lot of work left before we get meaningful, practical results. In some areas we're there, in others we're not. But everywhere we're getting closer…

– When it comes to text generation specifically, will we reach the point where a human no longer needs to be involved in the process? To check and evaluate what's been “written”…

– Perhaps

the human will be there to give the text meaning: what you're aiming for, what information you want to convey.

In generating, formatting and styling text we're already very far ahead, at least in English. Bulgarian still needs a few more steps, but it's progressing fast.

– Will we really have these technologies in Bulgarian soon? I think for smaller languages we simply don't have enough data for this kind of technology to work effectively.

– I think it will happen as a side effect of everything else. Yes, there may not be enough data for training of the same quality as in English. But first, the models are getting better and can learn from less data, and second, machine translation keeps improving. So it becomes entirely possible

for the text in any technology to be translated into English, processed, and translated back into the original language.

When you tell ChatGPT to write poems and songs in Bulgarian, for example, I think that's exactly what it does: it creates something in English, then translates it into Bulgarian, and the rhyme and rhythm get lost.

– You can also tell it lacks the data, for example, when we ask it about Bulgarian politics.

– In principle, you shouldn't rely much on ChatGPT for facts, Bulgarian politics included.

It's easiest to laugh at it when you try to ascribe to it qualities and uses it has never claimed to have.

ChatGPT, for instance, has never claimed to be an “online search engine” that will return facts. It claims to work well with text, if you give it something to summarize or expand, say.

– Yes, but GPT is now also part of the Bing search engine, and there its claims are different.

– With what they added to Bing, you can now expect it to give you facts too. And it will do better. With exactly these kinds of examples it will be better, because it first runs an online search and then analyzes the results.

– Do you think we'll get to the point of having personally generated news?

– Definitely, simply because it will become easy. If the company that delivers you the news in one form or another already has information about you and has built a model of how you prefer things to be presented, it will be all too easy for them to shape how they reach you. Just as they now pick which news you see, naturally

the next step is for it to be delivered in a way that pleases you as much as possible, that wins you over the most.

– But won't that make problems like the filter bubble even worse?

– That effect will certainly be there, but I don't know if it will be stronger. Of course, it's not good that what news reaches you, and how, is filtered. But that problem existed long before technology developed. Purely psychologically,

people tend to believe things they already wanted to believe.

It has been going on for so long, ever since we started exchanging stories. Every person has believed some more than others. Technology amplified it by filtering what reaches you at all. In my view, the fact that the way it reaches you is shaped will still have a smaller effect.

– What's your personal recipe for staying informed in a balanced way?

– I can't say I've achieved it perfectly. But it certainly helps if you make a conscious effort to look for another point of view.

If you're interested in different points of view, the platforms themselves start giving you that kind of content, because they decide you're interested.

If you don't read only the things you look at and say, “Hey, I like this, this is my thing, I agree with it before I've even read it,” if you glance at the other texts too, you can help the algorithms themselves balance out. From there on, everyone builds trust in particular platforms, institutions, podcasts.

There's no silver bullet that will solve the problem once and for all. You just have to know there's a problem, keep it in your head, and consciously take steps to seek out the differing opinion.

– Many specialists say ChatGPT isn't all that revolutionary, that plenty of successful language models are appearing. But OpenAI gained a big advantage by making their project so widely accessible.

– I think it's good to distinguish between the underlying language model, which is GPT-3.5, and the product, ChatGPT. OpenAI pulled far ahead of everyone else precisely with the product. While their model is most likely comparable to what the other big players, Meta and Google, have, they won by showing their model through a product anyone can get their hands on.

An interesting development happened recently. Facebook released its LLaMA model for academic use, and the data leaked onto a torrent. Building on it, scientists at Stanford made Alpaca, an algorithm that fine-tunes the model so it starts performing like ChatGPT. Having a language model doesn't yet let you talk to it, hold a meaningful dialogue. With ChatGPT the problem was solved by pouring a huge investment into people talking to it and telling it when it responds well and when it doesn't, and it learns from that.

At Stanford they used only 150 dialogues conducted by real people, then had ChatGPT generate more examples, made 50,000 of them and used them to further train the model. The effect OpenAI achieved with the millions of dollars it had invested in the final fine-tuning of ChatGPT was replicated for just $600, using ready-made generated data.

I find this experiment very telling, because

suddenly these technologies, which until now were locked away for the big corporations alone, are something anyone can download and run on their own computer.

– Isn't that dangerous?

– As dangerous as you like. It's both good and bad. It's good that anyone can experiment, and that will lead to very rapid progress: interesting applications will appear, and understanding of how these technologies work will grow.

On the other hand, all restrictions on what you use them for fall away. But that was inevitable. No one was under the illusion that these technologies wouldn't end up in people's hands, or that soon anyone wouldn't be able to run them and do whatever they please.

– One of the dangers that gets talked about a lot is cybersecurity, that this will make all kinds of attacks far more accessible. What are the other dangers we're not thinking of?

– Fake news, I think, is a big topic. It will appear in new ways, not just as it does now, where you release a story and hope it takes off. Comments, for example, can be generated very quickly. You can achieve what troll farms do now, but without the huge budgets.

I think we underestimate the danger

not just of planting fake news, but of using it to finally destroy trust in the real news.

Of flooding the information field with so much noise that the signal can no longer be told apart. When a genuine scientific paper on some problem comes out, people who have already seen 50 fake generated ones will tell themselves it's one of those too. Undermining trust in sources of information is very dangerous. It's hard to get people to stop believing fake news, but it may turn out to be a problem that they stop believing the real news.

– And I think that's already happening in full force, mostly because of the way we neglect journalism. The media can't find their place in the online reality, can't adapt to the new times. What do you think their future looks like? How should they transform to stay relevant to the world we live in?

– Personalized content is definitely one component, and so is content that's always available. It's old news, but linear television, where you have to be in front of the TV at 8 p.m. to see the news, is no longer relevant.

I think some of the more uniform information will be automated completely.

Sports journalism, for example, where you simply have to report something that happened and is more or less indisputable. The score of the match is what it is, it's clear who scored the goals.

But more serious journalism, investigative journalism for instance, will be very hard to automate. For a long time yet there will be room for real, serious journalists to shine.

– But I think AI technologies could be very useful here too, and they still aren't widely used. About 10 years ago it was thought that data analysis and processing would enter the work of the media, but it never really happened.

– Perhaps there weren't enough interdisciplinary people with a view of both sides: journalism and big data processing. Of course, that doesn't mean it won't happen.

But there's another aspect. The audience of the media that would do this is the kind that can draw its own conclusions. What I mean is, we were thinking about whether we could build an application that analyzes data and provides additional information. On MPs' statements, for example: judging whether there's a lie in a statement, building trust indexes. And then it occurred to us that the people who would use such software are the same people who are more critical anyway and don't believe the lies. Maybe data journalism has the same effect.

This will become easier and easier, and indispensable. Once such tools are in the hands of end users, journalists won't even need to do the analysis. Anyone will be able to; the question is who will put in the work.

– Are we close to the limits of what large language models can do? So far everything suggests we're not. GPT-4, for example, turned out to be far more impressive than the previous versions, and Midjourney 5 is capable of new wonders, giant leaps compared to its predecessors.

– It's very hard to say. In some directions we're close to the limit, for example how big such a model can get, at least with today's computing hardware. I don't expect anything much bigger than GPT-4, even though there's no public information on how big it is in terms of number of parameters. As for the limits of what the algorithms can do, I wouldn't dare say we're close.

There's a very interesting phenomenon there: emergent abilities. If we take something specific the model wasn't trained for, at a certain point it suddenly and surprisingly starts handling it. Take the task in which the algorithm tries to guess a movie from emoji. It's part of a large set of benchmarks used to evaluate models, known as BIG-bench. They tested language models that had never been trained for this task at all, and up to a point they did very badly. Yes, they were getting better at their core role, understanding language and completing text. With the emoji, though, success stayed at zero, until… at some point it shot up.

Unpredictable phenomena like this happen often. There's no way of knowing when a model will be capable of some side task until it suddenly turns out it's already doing it brilliantly.

Things like this keep appearing, which is why it's very hard to say we're close to the limit,

we literally don't know what new, unexpected results will improve with the next step up in a model's complexity.

– Officially, the Turing test hasn't been passed, but I don't think that makes much sense. In idea and in spirit, ChatGPT passes it: you're facing something that in a huge percentage of cases you can't tell apart from a human conversation partner. So why does the test remain open?

– I think that for some time now the Turing test hasn't been seen as a very serious criterion in AI circles. It's not something people sit around waiting breathlessly to be officially achieved. It will be more of a marketing move when someone officially announces the test has been passed.

Meanwhile, far more serious tests and lists of benchmarks have been developed to tell how advanced the technologies are. The meaningful tests are the ones aimed in specific directions. There won't be one general test after which we can say, “There, we've achieved human-level artificial intelligence.” It will be more like “human level in text generation,” “in solving math problems.” There will be specific tests for each field.

– I can't wait for an algorithm that works well at turning an audio file into Bulgarian text. For most people it's nothing special, but it would save me a lot of time. What are the most interesting things you personally expect to be achieved soon?

– I'd be very happy to have a personal assistant that works well. However much people praise Siri and Google Assistant, they're very limited: you find four or five uses for them and stick with those.

I'm eagerly watching the technologies that help create software. Things are changing very fast. In two years it will be very different, a completely different experience.

Another area is games. I came across a discussion about why the core mechanic there is usually walking around and fighting. It was simply the easiest thing to build. Until now we couldn't create a game where the core mechanic is having a full conversation with people. In general, the entertainment industry has the potential for very serious changes. I wonder what will happen from here on?