“Truly Intel­ligent AI? Only If We Take the Next Big Step”

– When you started out eight years ago with the idea of a platform that would analyze and synthesize enormous volumes of scientific publications, you knew there was no technology yet that could do it, but you expected it to appear. And that's how it turned out… Is that really so?

– That's right, we started working on Iris with the idea

of creating a question-and-answer machine to support scientific work.

That was a long-term goal. We had laid out a ten-year plan for how it would happen, and it was clear that with the technology of the time, there was no way it could work. Quite a few fields had to develop. We believed they would, and as it turned out, we guessed pretty well.

First we tackled the problem of too much information, caused by the sheer amount of content being created. Around 2010 a boom began in the publication of scientific papers, research and patents. The digital era made all of this easier, and content in the scientific sphere started growing exponentially.

When we took on Iris in 2015, publicly available open-access digital scientific content came to about 2 million documents, and digital content as a whole to about 30 million items—patents and scientific papers. Today there are around 60 million freely accessible, and the total volume is around 300 million!

– So suddenly scientists become far more productive and start churning out enormous volumes of content that is hard to keep track of…

– Yes, we noticed that it had suddenly become impossible to follow in detail what was going on in the field you were researching. Between 6,000 and 10,000 papers are published every day; for a given field it might be 100 or 200. Nobody could keep up with that volume.

We built the first search applications—at the time, they were semantic search engines. Before that, people used huge lists of keywords, derivatives of everything they needed, but those were clumsy and you could always miss something. We were able to solve that problem right away with a semantic search engine, where you describe in free text what you want the machine to find. We made something like an “ancestor” of today's search systems built on vector databases.

Then we moved on to the fact that people are sometimes interested in specific questions—is there a material with certain properties, how is it produced, is there something similar… Here neither classical nor semantic search engines worked, because they generalize, and that means finding a more specific fact is hard.

– And after that?

– We developed systems that extract facts from documents and structure them into knowledge graphs. When they're small, these structures are very useful and convenient to work with, because they let a person easily explore the connections between the elements. It gets difficult, though, once they grow large, and in some particular fields they started reaching millions of elements.

Then we built an application for fact extraction, and somewhere around there the need to combine the two began to emerge. To be able to find things semantically in order to narrow down the context or the number of papers, and to summarize. It became clear that the moment had come to build the system we had started with. Somewhere around then came the breakthroughs in transformer models, with the ability to ask shorter questions and get direct answers.

We brought in large language models that can answer questions directly. With the structured approach—the ability to extract the facts—and the unstructured one—the language model working directly with what it finds in the papers—today we combine the two results so as to reduce the models' hallucinations and make the answers as accurate as possible.

– As the company's description makes clear, the algorithm is already more successful than a human doing the same analysis of scientific publications… Do you think the growth of generated content creates a problem? We recently talked here about a study showing that certain words typical of large language models are suddenly turning up exponentially more often.

– Yes, over the last two years there has been growth in generated content. But that doesn't necessarily mean the quality of research is going down in any way.

I think it's rather that scientists spend more time doing experiments than writing

the content. Every paper has two aspects—the scientific work itself, and how to present the results in the most suitable way. And yes, that last part is now being handed over to the models.

– And so in your work today you face the same challenges as the creators of the big models—biases, hallucinations. Everyone is trying to solve them, starting with OpenAI and the giants, but somehow there doesn't seem to be much progress…

– First we should note that hallucinations are as much a bug as they are a feature, to put it in “IT speak.” They are an error, but also a desired effect. In the models they come from so-called generalization.

Over the years, the main goal was precisely to create a language model that speaks convincingly, that sounds like a human. And how does a human speak? We can choose words, paraphrase something that was said another way. For the model to do that, it has to be able to generalize, to recognize constructions of the same type and use them interchangeably. Except that

when we teach a model to generalize, that inherently means we doom it to be not very good with facts.

Yes, it knows it has to put a phone number, an address or a year as the next element in the string of generated text, but there's no way for it to know exactly which number to pick. Because when it comes to someone being born on a certain date, in the text it was trained on it will have seen many such examples, and the number will be different every time.

It has no way of judging which one is correct, because it doesn't make a cause-and-effect connection to the specific person.

This is exactly where it needs help. This is where we have to think about how to change things so that, besides generalizing, the model can also focus on specific cause-and-effect relationships.

– Forgive the naive question, but couldn't the solution be to pass the data through another model that checks the information?

– It's not that simple. There are different schools of thought, but in my view it won't work with the technology we have right now alone.

The mathematical machinery behind it has to change somewhat.

Right now it's optimized to generalize, because that was a problem that went unsolved for decades. The first language models date from the 1980s, and ever since we have been battling the generalization problem, on the assumption that the focusing one was supposedly easier, because you can have structured data. It turned out that's not quite the case once the volume gets too large.

Now this mathematical machinery has to be invented. The big tech companies are working on it quite actively, but it isn't easy. I don't think it will happen by training a new model with new optimizations; new things will have to be worked out at a lower level.

– Either way, the models handle facts impressively too. I ran this experiment myself—ChatGPT went up against undisputed quiz champions and beat most of them, and that was version 3.5, and in Bulgarian at that… Yes, if we're after facts, we should have learned by now that ChatGPT isn't the ideal solution. But we can use it and its siblings for so many other things. Could it be that our expectations are simply unrealistic?

– We can split a model's main capabilities into two components. It can generalize and understand linguistic structures—that is, how to construct sentences, organize them, prepare an answer and generate meaningful text. That's purely structural knowledge about how to formulate a thought. If we compare it to people, anyone who has learned a language can do that. Yes, the models are now capable of it too.

The other part, though, is knowledge. Being able to state a position on a subject doesn't mean you have the knowledge to do so. That, I think, is what people miss.

The models are trained on huge amounts of data, and they use much of it to learn the structure, the logic of the text, the way to answer. Yes, they also accumulate some knowledge, but it isn't at all that straightforward; sometimes you need an approach to get it out.

Quite often the problem isn't so much the model as the fact that we have failed to extract the knowledge we're after. We haven't asked it in a way it understands.

“I asked it this, why won't it answer the way I want?” When you talk to someone, the fact that you understand the question one way doesn't mean the person across from you understands it the same way. Prompt engineering is the way to explain something so that the “person” on the other side understands us. When you meet a stranger and ask them a question, they might not get it right away. You have to explain better what exactly you want, give them an example, a chance to picture it, and then they'll answer.

– The models themselves are gradually building up context, too. We often go looking for an assistant for everything, when this is a technology that can be useful to us in many areas—but we have to know it and know how to work with it.

So, as far as I understand, you think we're close to the limit of large language models in their current form, is that right? I read an article of yours saying that on their own they can't bring us to artificial general intelligence.

– They're constantly evolving; experimental models with different kinds of structure and larger context are being released. Things are moving. But in my view, yes, there has to be some next breakthrough that solves the focus problem.

How big a model is determines how many different structures it can learn. Some people, Sam Altman for example,

believe we need even bigger models and they will learn to make these cause-and-effect connections on their own.

Right now there are two main camps. Altman's “unstructured” one, which believes this will happen if we simply keep piling up in the same way. That a lot of effort has to go into bigger and bigger models. But there's another constraint there—data. We're already approaching the limit; we have no more data, and people are thinking about how to synthesize new data. But that in itself creates another serious problem—the models have biases even now, so imagine what happens if we start synthesizing data. It isn't clear how that problem can be solved.

The other camp is the “structured approach,” whose chief advocate is Yann LeCun, Meta's chief AI scientist. He believes the models must first learn an understanding of the world, some kind of cause-and-effect relationship, and then that should be complemented by their current abilities.

Personally, I think the truth is somewhere in the middle. We definitely haven't fully hit the limit, but I don't think general AI will look like the current models. The focus problem has to be solved through new mathematical constructs that teach the model when to focus and at what scale. That, to me, is the other problem—models can generalize successfully in general language, and with some additional training they cope with specific tasks. But a more general model struggles at a specific level, and vice versa.

– As a guest on Ratio, you said we've reached the point where we've made a breakthrough and it's time to find more practical applications, which in turn will attract funding for the next stage of AI development. A year and a half after ChatGPT, there still don't seem to be as many mass everyday applications as expected. Do you see it that way too?

– We sell products in this space. Before the end of 2022, people would come to us and simply not believe a machine could do something like that. We even heard things like: “Hey, how nice it'll be when this thing starts working, in five or six years.” “No, it works, we have it right now!” But people just didn't believe it.

And then you wake up a month later and suddenly everyone is coming in with inflated expectations. “It works great. We want to use it. To solve every problem we have.” Which is absurd; there's no way.

You didn't believe something could do any of your work, and suddenly you're saying it will do all of it.

We swing from one extreme to the other, and it's at neither end—it's somewhere in the middle. It can do useful work, but far from everything. After every such moment of inflated expectations, there's a bit of a cooling off.

– Most of the guests I ask, though, say there can't be a winter in the world of artificial intelligence, because the progress is too great and touches too many fields…

– I think a cooling effect has been there since the middle of last year. At first people imagined that suddenly everyone would start using it. No, it isn't yet making its way into business with the impact people expected, and that's normal. But there are fields where it is, technical support centers, for example.

The easiest field for this kind of technology to enter is, without a doubt… technology itself. We can already see what Copilot does, and more platforms are on the way. You can see the change in the job market, too—juniors have a harder time finding work, because one senior with ChatGPT can do the work of one senior and five juniors. The barrier to entry there is already much higher.

Maybe they should learn to use ChatGPT? It isn't that easy. Prompt engineering requires a certain way of thinking and knowledge of how the models work, so you can structure prompts with ease. It's good to take some courses; one of the pioneers is Andrew Ng, who has excellent courses both on understanding large language models and on prompt engineering. It takes a certain kind of thinking, and that has to be learned.

– But that creates a problem—if young specialists don't get into the system, in a few years there will be nowhere for experienced ones to come from… That's just how our profession works.

– I don't think the profession will disappear. There was certainly some bloat, and that has to be rebalanced now. And I think later there will also be a change in the way people are trained. Either way, technical support and IT are the two fields where the adoption is most directly visible.

There are others where serious changes will happen, personalized medicine, for example. At the first conference we attended with Iris, in 2016, that was an imaginary notion. Now, with the arrival of ChatGPT, it's entirely possible. Under the current healthcare system, anywhere in the world, there's no way a doctor can have the full context on a given patient and recall it at the moment they need to make a decision. Now, with the capabilities of large language models, a patient can provide much more context, from which the doctor can extract information and make a decision.

Very interesting things have already happened in medicine—the invention of CRISPR and the decoding of protein structures open up enormous possibilities. In these fields we'll be able to do many things we haven't been capable of until now. There will be a revolution in medicine. But it won't happen fast; I think at least one generation has to go through the new way of doing things, and it will take at least ten years or so.

– Another guest of mine, a specialist in medical technology, recently shared very similar conclusions. In Bulgaria, too, some fascinating projects related to health are being developed. But what is the path to having more startups, more product companies?

– My view, going back to the years when I was still in Sweden, is that

IT people in Bulgaria were in a kind of lethargy for a long time, not realizing that outsourcing companies can't be a long-term solution.

It's something that evolves along with the market; sooner or later someone will turn up who can do the thing more cheaply, and that work disappears from the previous market.

For quite a few years Bulgaria was in a very good position for outsourcing, because our culture is closer to the Western one and we had relatively good education. But that momentum of funds, capital and opportunities should have been used to build product companies and stimulate the startup environment.

We're quite late. I've been watching Poland—there are outsourcing companies there now, of course, but also an awful lot of product ones. They managed to notice that it mattered and quickly began moving toward startup innovation.

One example is a very interesting data storage project that uses plant DNA. We can now use biological elements in their real environment for practical purposes. A single tree can hold gigabytes of information—something extraordinary! This is being developed in Poland; the founder was at a conference in the US and people were amazed by the breakthrough. In other words, they managed to use the capital and opportunities the moment gave them to move on to startup companies, while countries like ours, unfortunately, lagged behind.

– How can we catch up?

– I don't have a clear-cut answer. The environment itself is very important, but the first thing I think people need to understand is that

the era of outsourcing companies is over. Its effect will become visible far more radically from here on.

Yes, there are a few big players in Bulgaria, but their influence will gradually diminish; we're seeing the layoffs. The only solution is for people to figure out how to build product companies.

Except that those are built with a different mentality. As Steve Jobs said, the goal of a company is to survive until the product is ready. That means you set out with an idea that has to become a product, but that road can take two, three, five years. For deep-tech companies like ours it sometimes takes more than a decade—as with storing data in plants, for instance.

But people set out to do it anyway. For it to work, at the moment they set out there have to be people around them with a positive mindset who will believe in them and invest for the long term.

– But I suppose that also comes down to national character…

– The surroundings have to be like that. If we want to be the country that builds strong startups, we have to support such people, give them strength, tell them they can do it, push them forward however we can. Whereas the mentality is still: “Why did you take that on, it's too hard,” “Ten years? You'd be better off working at an outsourcing company,” “That idea will never work.”

It won't happen that way. When everyone is pulling you down, a startup struggles to grow. I say this as someone looking in from outside; in Sweden the mentality is different, they've had a developed startup culture for nearly 70 or 80 years.

In the world of artificial intelligence, I think even the European Union is behind, because startups in Silicon Valley work with one huge, very simple difference. People there are extremely open about sharing their ideas. And once they've shared them and decided, “Okay, we're doing this,” nobody will take the idea and start competing. You go, you meet the other person and you do it together. You find ten more like-minded people and you end up with one big company. You all believe in the idea, you're ready to invest years of your lives, you come together and make something unique.

– Why did you come back?

– A combination of circumstances. Honestly, I'm still not convinced I feel very good here. There are definitely cultural differences; some good things I saw in Sweden I really miss here. And the other way around, of course, but when I add it all up, things are in the balance. Lots of people say: I loved it when I came back.

I'd say I rather like it in Scandinavia, and I even go back from time to time. But on the other hand, I'm trying to change what I can here…

If our conversation with Viktor Botev has intrigued you, you can see its continuation live on Sunday! Together with Anna Angelova, a journalist at BNT 1, the flagship channel of Bulgarian National Television, Viktor will be a guest at an event hosted by Digital Stories.

On May 12 (Sunday) at 7 p.m. it's time for the meeting “The Truth and Nothing but the Truth.” As part of the Sofia Science Festival, at Sofia Tech Park we'll talk about the future of the way we receive information. For more information and tickets—click here.