“Within Five Years, We Will Merge the Real and the Virtual”

– In recent years, especially before all the noise around artificial intelligence, ideas about a breakthrough in creating virtual reality were very much in the air. Now that people talk about these topics far less often after Zuckerberg’s ambitious plan for the metaverse, does that mean we’ve hit some kind of barrier?

– Virtual and augmented reality are parts of the telecommunications process. With the metaverse, the talk was that you wouldn’t have to go anywhere, you’d be able to communicate with everyone from home, to attend events. Behind all of this lies an enormous, very heavy and complex telecommunications process, which we call “holographic communication.” The goal is to use these technologies—virtual and augmented reality—to achieve full-fledged communication between people in different places, exchanging every kind of information in real time, spoken and visual.

The original idea was for holographic communication to replace the messengers we use every day. Today we are close to that—Microsoft already offers a similar option in Teams, as long as you have the right technology. Apple drew attention with its new Apple Vision Pro headset and added a new term, “spatial computing,” as if there weren’t already so much terminology—virtual, augmented, mixed, even extended reality. It’s easy to get confused, but all of them share a single task:

to reproduce the real world as realistically as possible, with no delay at all, to merge the real and the virtual.

That includes not only static objects but the person too. And not just as a figure, but as gestures, facial expressions, even their emotional state. For us to speak of communication and a communication process, you need a sense of yourself, of presence and of connection with the person you’re talking to. That means glances, gestures, handshakes, engaging many senses.

– What steps in that direction are you developing at the Technical University?

– We have a large team of students who have been working on this very enthusiastically for several years. For the fourth year now, we are also running hands-on sessions with students from engineering high schools. We show young people in the 10th and 11th grades that new technologies are being developed in Bulgaria. We have high-level research publications in very well-known journals around the world. We partner with American universities such as Princeton and Purdue, more than 10 universities in England, and with Japanese, Thai, Indian and Pakistani colleagues.

The problems we face in developing virtual and augmented reality are fairly familiar. The equipment still isn’t at a price the average person can afford—you need 7,000 to 8,000 leva (roughly $3,800 to $4,400) for an augmented or virtual reality headset, so

most people haven’t experienced anything like it firsthand, but the progress is remarkable.

Virtual reality is a bit more accessible, but in most cases it’s used for computer games. And that is one of our country’s strengths within the EU: we have very strong companies here, and combined with specialized education, this turns us into a sort of hub for game development.

Yes, right now everyone has thrown themselves at artificial intelligence, and with good reason. But the outlook for people who want to study and work in creative 3D technologies is decidedly very optimistic. Gaming, augmented reality, virtual reality—these are fields with a huge hunger for talent.

– Tell me about some interesting projects with which your students have impressed you.

– Many of them build applications they actually need, for example creating three-dimensional models of real objects. We have

an entire internal combustion engine, with all its components.

Projects like that are very useful, because one of the target tasks of the Technical University is developing electronic distance-learning materials. One graduating student recreated an entire lab exercise on signal measurement in virtual reality, and now every student can use it.

Another student built a virtual fitness trainer. An avatar that coaches you, follows in real time how you perform the exercises and then gives you recommendations. A “virtual coach”—you might not need to go to the gym at all. We also hold events about space; one of our students graduated in France, and with her project you can take a walk on an asteroid.

Students from abroad come here for internships, and in 15 days they manage to build their own augmented reality applications without any specific skills or knowledge. Anyone who decides to, without even a programmer’s experience, can now make something basic. The only limit for virtual and augmented reality is imagination.

– And yet we still seem far from the perfect metaverse…

– The fact that the metaverse didn’t take off with such force doesn’t stop the development of the technologies themselves, which is truly astonishingly fast. It’s no accident that the white papers on 5G and 6G identify virtual and augmented reality as one of the fundamental things these new communication technologies are built around. Right now there is active work on having enough computing and communication resources for us to fully experience virtual or augmented reality.

The movement is constant; the topic isn’t losing its relevance. With this drive to engage all the senses, haptic suits and gloves have already appeared at affordable prices. People can shake hands, pat each other on the shoulder, hug.

There is work underway, and in two or three, five years at most, it will be possible to transmit taste—

to see the little cake in the metaverse and taste it.

Smell turns out to be the hardest to reproduce, but we may manage that too. At least I believe we’re moving forward quickly.

– The ideas about virtual reality go back to the days of Jaron Lanier; we’ve been trying to create it for 40 years now. And it seems the main goal is still realistic visual recreation. Why is it so hard?

– In my experience, at least, one of our challenges is the human face. Nothing else is as problematic as the micro-expressions, the naturalness of the gaze, the movements that are very hard to capture and reproduce in real time. If it’s more static, it isn’t so complicated, but a person never stops moving, sending signals that we don’t consciously register but still perceive. That’s one of the things holding us back; it’s no accident people laughed about why everyone in the metaverse exists only from the waist up, with no legs.

The challenges are many, but let’s not forget there’s also apprehension. We easily imagine the scenario in which people forget how to communicate in reality, because everyone will be logged into the virtual world, as in science fiction.

You put on the headset and the suit, and you don’t live in the real world but in the virtual one.

That’s why this kind of technology is adopted a little more slowly, with more difficulty. We humans are peculiar: on the one hand we want and seek the best, and once we get it, we become suspicious. Since even believing that the Earth is flat is coming back into fashion, evolution clearly doesn’t always lead us only forward.

But this is what I tell my colleagues too: as engineers, our task is to improve the world, to make it better than it was when we were born into it.

– Can generative models help us in the search for this realism of faces?

– Definitely. Colleagues who were at Carnegie Mellon worked on generating photorealism, with synthetic data. That’s one of the trends,

thanks to the models, being able to generate objects,

so that everything becomes faster and the whole virtual world can be built.

But something important I want to point out is that very few people actually understand generative models. The foundation underneath is terribly complex, and for a task like this it’s important that they not be mere black boxes—that we understand well how they work. With these models, even their creators can’t always understand why the result they get is exactly what it is.

That’s something that worries people—that you have no control. That’s why I’m glad I work with specialists who dig in and try to grasp the meaning in depth.

– Do you think it’s possible to achieve indistinguishable virtual reality in the foreseeable future?

– Oh, yes! I’d even say much sooner than five years. Quite a few companies are working on it, including the very biggest.

In my view, we are very close to an absolute replica of our surroundings in the virtual world.

It seems to me that for now it isn’t more widespread for those two reasons—people are apprehensive, and the cost is still high. But I think both will soon fall away.

– What are the most important questions we need to answer before then?

– There’s a lot of discussion about how we humans behave, for example, in the metaverse. That is already a matter of ethics. Technologically you can impose restrictions, but that isn’t enough. Everything connected with technology raises ethical questions. Because we are human beings, and it isn’t always clear how we’ll react.

But I see that today’s young people think differently, their values are different, and that matters too.

When we were young, the world wasn’t in our hands; I had to go to the library, search, read, write. Now I just click and all the information is in front of me, which automatically changes one’s entire worldview.

For me, that’s the limiting factor—the ethical questions, which have to be resolved very carefully so that they don’t restrict the rights of the majority of people while at the same time protecting them.

We’ve seen bad examples of things happening in the metaverse to women or minority groups. These are problems that emerge only afterward, which there’s no way to plan for. For instance, we didn’t expect it would be such a problem for people to invade your personal space while you’re in there. It turned out there needs to be some kind of protective circle—but then if you want to shake hands with someone, what do you do?

The technological questions, we will solve. Sure, there are certain limitations that can’t be overcome right now, but options are being sought with new technologies. How will you overcome human nature? That is the hard question, which many people will be solving, and few of them will be engineers…