“We’re Already Gen­erat­ing a Tremen­dous Amount of Chaff”

– These days you research “cognitive shields,” an approach meant to help us preserve human knowledge against the backdrop of extraordinarily advanced artificial intelligence. What led you in this direction?

– Today far too many things are getting easier, including in science—doing a literature review, for example. When you don’t know where to start, you fire up ChatGPT, Claude, Gemini. There are so many models now that you hardly know which to pick. Any of them will summarize an article for you, so you don’t have to bother reading and understanding it. And this already works very well, because it gives you starting points, and fairly recent ones at that. But how far the summary has captured all the ideas in the article is anyone’s guess. When you read an article several times, new ideas pop up each time, ones you missed before. It depends on the state you’re in when you read it, on what’s on your mind at the moment. A living brain interprets; it doesn’t just summarize.

Another problem is that nobody reads scientific papers from 20 years ago anymore, and that is actually a huge problem, because some papers from 20, 30, 50 years ago contain ideas that were never exploited, whose time simply hadn’t come, and which we could use now. But now there’s no one to discover them, because nobody reads; everyone uses artificial intelligence instead.

– Or perhaps the growing problem is that, according to more and more studies, AI is now itself responsible for a larger share of scientific publications?

– Exactly. The bad part is that when doing a literature review is so easy, people churn out new reviews on a conveyor belt. If the journal has a serious reviewer, they’ll say: “All right, where’s the critique?” It’s not just a matter of gathering 20 papers and roughly categorizing them, but of taking a critical approach and saying what the new trends are, what’s missing. That’s something AI doesn’t do very well.

But people cobble papers together, send them to journals, and if they land two careless reviewers, the paper gets accepted, and so the conveyor-belt reviews multiply and “pollute the environment.”

A tremendous amount of chaff is being generated. Often, though, with an attractive title

and when someone searches for a given topic, ChatGPT finds it, the author cites it, and the wheel turns.

And many students don’t read the papers at all; at most they read the abstract. And that leaves out a lot. Every summary is simply an abstraction. There are many interesting details that a human brain, reading them, catches through some association, through the mood of the day, through having a stomachache. AI can’t catch them, because it works entirely at a formal, linguistic level.

A lot gets lost, a lot sinks to the bottom, and only what floats on the surface will be skimmed off.

– Sadly, it’s not just in science; I think the same is happening everywhere: in the media, in the whole information space. But how can we stop this cognitive decline? Today it almost looks irreversible.

– That’s right. That’s exactly why I proposed a research topic, which was approved and funded for the next five years. We’ll try to build “cognitive shields.” I’m not a specialist in developing algorithms, and I think that kind of work is now very hard to do at universities. It has moved to the big companies, because they have the resources and gather the smartest people. They have a lot of money, and now you need computing power. We universities are quite far behind in this field.

– Still, the big researchers still come from universities.

– True, but that’s because when they made their breakthroughs, they didn’t need so much computing power. The big Canadian names in AI, such as Hinton, LeCun, Bengio, created their algorithms and tested them 15 to 20 years ago under different conditions, when the gap between university computing systems and commercial ones wasn’t so glaring…

In any case, my idea is to work on building a shield at the interface level—the dialogue or connection between the human and the computer. I’ve been working for more than 10 years in the field of persuasive technologies, which came up in our previous conversation.

These technologies are very dangerous, because they really do have a powerful influence.

People are extremely easy to manipulate. And I’m thinking—what if we could “manipulate” them into caring, paying attention and worrying that AI is lying to them and manipulating them?!

And into looking deeper, putting those few kilobytes into their brains, investing them and approaching what AI gives them with care.

– How would that happen? My question is mostly about the media, the channels, because I’ve been trying to do exactly this for five years now and I don’t reach people on a mass scale, because social networks use persuasive technologies so effectively.

– I think things will move in that direction. I think

people already have a great deal of distrust toward social networks, and especially Facebook.

I think the trend in social networks is to hybridize with AI. New AI technologies such as agentic AI and OpenClaw will introduce new artificial personas into the networks, and people won’t know whether they’re talking to real people or AI agents. On the one hand, this will be very convenient: anyone can “hire” an AI agent to represent them online, keep track of what’s going on and inform them of the most important things, answer trivial matters, send greetings for name days—the feast days of one’s namesake saint, celebrated in Bulgaria like a second birthday—and birthdays, and so on. But when this becomes a mass phenomenon, everything becomes fake. The main reason to be “on the network,” to be in touch with acquaintances and friends, falls away.

The AI agents now arriving offer a lot of conveniences. People tend to “humanize” things: to find consciousness in their pets, even in inanimate objects. There are studies showing that people perceive chatbots as conscious, even as witty and empathetic other people. The bad thing is that people turn more and more to AI as partners and stop communicating with real people. Because AI is much more persuasive; it simply agrees with whatever you say, smiles and says: “Sure.” Which is pleasant, flattering, soothing. Most people actually don’t like to argue.

– Though you wouldn’t guess it online… All the more so since social networks now know us better than we know ourselves. Because we leave gigabytes of information interacting with them, and now with AI too. So how can we build effective shields when everything is in the hands of the “enemy”?

– That is a very big problem, and it isn’t new. Even before we got to social networks, when the internet appeared, research in information security began. There are also laws that force big companies to give users a choice—the annoying cookie banners. People who don’t want to be manipulated and led by the nose put up barriers. They check their security settings and so on.

So in my view, change will start with people who worry about such things. Who have some concern for their autonomy and don’t want to be manipulated.

How will we bring it to the masses? I hope it will happen through education and schools.

I don’t know how it is in Bulgaria, but in Canada, in several provinces, and very strongly in Scandinavia, phones are banned in schools.

I think there’s a big shift in attitudes. More and more is being written about how harmful social networks are for young people’s development, about how children fall into depression when they reach their teens, because they started on tablets as babies and simply don’t know normal human interaction.

I think this wave of phones and social networks will soon turn. I’ve lived quite a while now, and it seems to me that everything moves like a pendulum or a sine wave. We’ll come to the opposite trend.

– The topic of children and technology is indeed huge; I tried to start a conversation about it here too, with a long series, but… we come to the self-awareness that could drive some kind of change, and I don’t see it.

And have we really seen this opposite trend so strongly so far? Yes, there have always been people like Tristan Harris, Kim Dotcom, Assange, who fight against the way we’re manipulated through technology.

– They’re not isolated cases.

– But all along they’ve been a visible minority.

– That’s true, but look, it’s like fashion. Fashion is the same kind of social phenomenon. It moves along; it always starts with a few people who set it off, these influencers, innovators the rest follow. But not everyone at once. It simply starts at the top, then trickles down, and by the time it’s widely adopted, the peaks up above, those early examples, are already at the foot of another wave.

There’s always a lag in these waves and in how they spread. I hope it will be the same here.

– So do I, but I don’t believe it. My only hope is economic, and even that’s putting it strongly, because it would slow down the dangerous trends in technology but would do enormous damage to the industry. The idea that at some point the huge sums being poured into artificial intelligence simply won’t be recouped.

– That may happen too. But look at the dot-com bubble. It burst, but the technologies stayed. And afterward, little by little, there was simply a lag of four or five years and they came back. And it became profitable again. The “chaff” fell away; the serious companies stayed.

The technology won’t disappear. Why? Because it relies on human laziness.

It may sound cynical, but I believe human laziness has always been at the root of progress. People want to minimize their effort. We’re used to looking for ways to reduce the effort it takes to achieve anything at all. That’s why AI is spreading widely in universities too, which personally worries me a great deal.

I don’t know how it is in Bulgaria, but I have the feeling that around the world teachers have already accepted that students use AI to write their homework, and secretly during exams too. There’s simply no way to prevent it. Especially with the new OpenClaw technologies, in which AI models run locally, on the laptop, and in the near future on the phone. There’s no need to be connected to the internet and to a big commercial model. They’re also becoming ever cheaper and more accessible through platforms like Ollama.

In Bulgaria, at least, there are oral exams—at the Faculty of Mathematics and Informatics they still examine those huge cohorts of students orally, for three days. That’s unheard of in America. Of the people I know there, nobody remembers having sat an oral exam in 30 or 40 years, because it’s very stressful. Likewise, being called up “to the blackboard” in front of the class at school is unheard of. And for my generation it was an everyday thing at school.

But that’s the only salvation from AI right now.

Or, if it’s a written exam, on paper. That’s how I learned programming—I never touched a computer. When I was a student, we had big machines, and programs were punched onto cards, line by line. It took several days before a listing came out of the computing center with the message “syntax error in line 18.”

– Is that the future?

– Yes.

The future is back in the past.

– But how will we be able to convince each other of this? Because we really do like things easy; we’re lazy. We get used to delegating, to the idea that the next task doesn’t need doing ourselves, that we can skip the effort. The logic of all this is that diplomas will also have less and less real value.

– That’s exactly right. Let’s look at this topic specifically, at education.

My specialty is applied AI; my doctoral dissertation was already in that direction. But the AI we did back then was different. It wasn’t based on neural networks; it was symbolic AI.

Neural networks back then were developed by engineers, who used them to regulate the washing machine cycle.

The scientists developing artificial intelligence sought to transfer knowledge from the human to the machine. They tried to represent abstract constructs of the human mind in the computer’s memory and to develop algorithms that operate on these constructs through logical operations. That’s a much higher cognitive level than the level of neurons. We didn’t think at all that machine self-learning at the neural level could have the system build such abstract, conscious or conceptual knowledge automatically. And it turned out that this was the right path.

– Yet Yann LeCun still says that this isn’t the path to artificial general intelligence, and that we need to go back to that transfer of knowledge.

– Because that’s a way of controlling it; otherwise the machine learns whatever it pleases. It remains a stochastic model, and what it learns can be very harmful, very dangerous. This is the subject of responsible AI, a big topic that’s very strong in the West right now. Social scientists, philosophers, ethicists and lawyers are joining in. And it’s a hope for these humanities disciplines, which suddenly find an extremely important problem they can attack with their own methods.

– And how should we approach education? Should we ban AI altogether?

– No, that’s impossible.

We simply don’t need to control the final product, but the process.

In Canada and America it’s normal to give students five or six projects that they do at home and hand in a week later. They may be programming assignments, English essays, data analysis. They do them at home and hand in the final result for grading. Well, there’s no point in that anymore.

You don’t look at the final result at all, because it’s generated by AI. You make the student say which AI they used, what prompts they gave, how they analyzed it critically, what they think of the AI’s answer, whether they used another AI for critique. How they interpret the critique, what their own opinion is. In other words, to prove they approached the AI’s answer critically.

The dumbest thing, from which a student learns nothing, is to give the assignment to AI, get a ready-made solution and, without a single “twitch of the brain,” copy it and send it in.

I tell my students: “I know you’ll use AI, but before that, think for yourselves about what solution you can come up with for this problem. When you already have some idea and you get the AI’s solution, you’re in a much better position to critique and evaluate it than if you hadn’t thought about the question at all. That way you also remember it better.”

– Or perhaps there’s no point in putting ourselves through it, because it really is now better than us at each next thing. I admit the results of this experiment, in which people couldn’t tell my own writing apart, make me despair. We gave ground and gave ground: we were better at chess, but AI beat us at that, and at Tetris, at everything. We wrote better, more creatively, with better style—that’s over. We wrote code; AI couldn’t. Now it does that well too. Maybe there’s no point in putting ourselves through so much hardship, and we can simply rest?

– Yes, but who will feed us?

– We’re advanced enough in agriculture; it isn’t hard to feed ourselves if we act wisely. And robotics is advancing in parallel—recently, for the first time, an android beat us at a marathon in China. Why should we toil?

– Because we’ll atrophy. It’s like exercise—when you don’t work out, when you don’t walk, you get sick.

That’s it—we’re biological beings and we have to strain ourselves. I personally believe that nothing learned easily is remembered.

We had a project like this with our third- and fourth-year programming students, and we gave them a problem to solve of the kind often given at job interviews.

We had two one-hour sessions in a lab, to make sure they weren’t using AI. We split them up: one group used an AI of their choice, the others had to manage on their own. Two weeks later we called them back, this time for everyone to try without AI.

It was a small group and we didn’t have statistically robust results, but even so you could clearly see that the majority of people in the first group were a complete failure. They had seen the AI’s solution, but they couldn’t reproduce it two weeks later.

– We’re all like that. And maybe they don’t even need it. I’m a programmer who has written almost no code for the last half year, and I can feel myself forgetting what I knew. And it seems I’m not losing anything for now, because there’s less and less need both to write this code and to read it. Just as I don’t know Assembly.

– But notice we say “writes code.” I think computer science education has to change.

Let’s not torment students with writing code, but teach them to solve problems.

What approach will they use to write the algorithm? Yes, AI can do that too. If you give it an algorithmic problem, it will solve it. I don’t know if it’s very good at that yet; INSAIT said that for solving math olympiad problems the accuracy is somewhere around 60 percent.

– But, as with everything else, in a few weeks it may be 80 percent, in a few more, 90 percent…

– That’s true. And how do we find meaning? How do we preserve some autonomy and some goal to strive for? Because this way, every effort becomes pointless.

– That is exactly the hardest question, one we never imagined we’d face, or at least not so soon. Could it, as one of my guests said, change society so that the classes of the future are the minority with the concentration to read a whole book, and everyone else?

– Yes. I think that’s exactly what will happen. Universities will once again become elite institutions, as they were in the Middle Ages. When education isn’t for the masses, it will be terribly expensive, and it will be the children of very rich people who can afford it—to send their children off to pursue it, as with art. Who can afford to make art?

– Those who don’t have to earn a living… Are there things artificial intelligence will never be able to do? I thought it would never be able to write like me, and look, that didn’t pan out. What will be the thing that remains purely human?

– I think there are still quite a few tricky things where robots won’t cope for a long time—for example, the work of a plumber or an electrician, where the tasks are physical, complex and require a rich variety of physical activities. That’s manual work, which is still hard for machines. Enormous sums are being poured into robotics right now, but robots are strong when they’re in a relatively constrained environment.

For example, those home robot vacuums work well, but not always; they get confused.

– And see, that’s not what we’re missing!

– Exactly. I hate cleaning. If a robot could someday come and dust for me, wash the windows, the bathroom, I’d be only too glad.

– I imagine that caring for, raising and teaching children is another thing we shouldn’t hand over. And not because the technology won’t be able to do it better than us.

– Exactly! We mustn’t! But we can see how even today people very happily hand their children over to tablets to entertain them.

When electronic babysitters get even better, it will be terrible. These will be completely different children…

I really don’t know what’s happening with these children. I have young colleagues who say that even they can’t find a connection with the students, these generations are so different.

– And technology did this—or rather we did, with its help.

– Yes. Because it’s very convenient to hand a child some game, a video clip that entertains them with pictures and little songs. And the child settles down, doesn’t move at all, so you don’t have to worry that they’ll run off somewhere or get into mischief. They’re absolutely nailed in place. Their muscles degenerate; they don’t learn how to interact with other children. And when they meet other children for the first time, they go into shock and don’t know what to do.

Students are much meeker now; in the old days there were many more troublemakers. Now the problem is that they panic. In Canada we have students who faint at exams from terror, from fear.

Here in Bulgaria, I asked colleagues whether they have this problem. No, they don’t.

– And probably the reason lies in something most of us dislike about the system—the exams. Here, to survive school, you have to get through the national exams in fourth and seventh grade. A very unpleasant experience to put a 14-year-old through, but apparently it pays off later in this way.

– Exactly. I remember school being extremely hard. I wasn’t particularly studious as a child, especially in the things that mattered at school age. Writing neatly. Horror! Math too; I loathed math. I loved reading, and that was a great pleasure for me. But everything else was hard. I only came to like math in fifth grade, when we started studying geometry and proving theorems; that suddenly became interesting to me.

– It became interesting to me at 35.

– And I wanted to become a writer. Already in fourth grade my classmates thought so. We had a very good literature teacher who gave us a lot of freedom, and I wrote crime stories to entertain the class.

– I believe this is also a big problem for us, that we impose these stereotypes and specializations—we divide ourselves into humanities types and tech types, we specialize in specific narrow fields. And yet Henry Kissinger, for example, writes that the age of polymaths will return—people who combined knowledge from all sorts of fields, because they can provide the original ideas that AI will never “give birth” to.

– That’s true, but people are very different. Most of my colleagues are so focused. Indeed, when you work in a very hard field, such as algorithms, you have to be focused. On the other hand, that way you’ll never come up with anything truly original.

If you’re lucky and this narrow, unbranching path you’ve set out on leads somewhere, you’ll get there in the end. But devoting your life to it… I personally can’t do that; I get terribly bored. Throughout my career I’ve been tossed between different fields, because what interests me is the problem, not the techniques I’ve mastered.

So what if I’ve learned something and have mastered it; the interesting thing is to learn something new, to enter a field where I’m a novice, a student again.

– I think all of us should develop this way of thinking in these dynamic times. I wish with all my heart that you turn out to be right—that the pendulum swings back and cognitive shields save knowledge. I’ll be glad to talk about them again at our next meeting. Thank you for the fascinating conversation!