Gatakka, ChatGPT the Psy­chol­ogist and the Price of Critical Thinking

– You have a lot of experience in the industry. What surprises you most about the way technology develops?

– What always surprises me is how little people understand about how the technology works. At their core,

these are very simple automated systems with a light polish. But people don’t understand what exactly is going on.

We’re talking about basic general knowledge and a lack of critical thinking. People wildly exaggerate these systems, distort them. They always expect more, and these are perfectly ordinary automated systems. They start delegating more and more to them, expecting them to help, to do some job far beyond their technical capabilities.

What always surprises me is human stupidity. Something comes along, you use it, your work, your career, your life depend on it. But you never ask yourself how on earth it works at the most basic level, so you know what to do with it.

– Isn’t the problem rather that we, the people in this industry, don’t explain things well enough? It seems to me an ever higher wall is rising between the world where information technology is born and the rest of society…

– No, that’s not it. People simply don’t ask questions. It’s the same in many other areas—diets, for example. There are whole TV channels, a huge business: you eat lettuce, you switch to nothing but meat, or no meat at all. Countless variations, there’s nothing they haven’t invented, extreme fasting, twenty days without eating anything.

But if you ask yourself the simple question, “Fine, why am I getting fat? What’s the mechanism?” the answer comes easily. First, you’ll realize there’s no way to lose weight through exercise. It can greatly improve your health, but it won’t make you slimmer. It’s simple arithmetic: you need 2,000 calories a day, and if you take in more, the body stores fat; otherwise it burns it. You cut what you eat in half, add a bit of training, and the pounds come off. That’s the kind of critical thinking I mean.

It’s the same with technology. “Wow, we’re putting everything in the cloud!” Fine, but what if the cloud goes down?

“I’ve built the whole integration!” Fine, but what if they change something inside? Where does that information go, does it get deleted? Do I own that information?

So many people, if their Gmail accounts were deleted, wouldn’t be able to pay their bills, because everything is tied to that email. They can’t pay a bill, change their email, sign up again, get a new mobile phone or transfer a utility account. They lose all their data, photos, personal things. That’s what I’m talking about.

– So the big problem, as you see it, is that we underestimate how digital we’ve become and how much we’ve delegated to companies without weighing what they’re capable of.

– Absolutely! That always surprises me. It’s what fueled the boom of social media, of the cloud, and later of SaaS—software as a service—and PaaS—platform as a service.

How many people come out and say that microservices, SaaS and PaaS are business killers?

If you’re a startup, in certain situations, they’re useful, no argument there. But outside those ideal cases they’re devastating for a business. In terms of hidden information, unforeseen costs, maintenance. The complexity is simply moved from one place to another. There’s no basic understanding, no analysis of what you’re actually using from a given product.

And, naturally, companies take advantage of that. Look,

what did ChatGPT actually tell us? That we’re very far from artificial intelligence.

It can barely work out what 7 times 30 is. There are far too many things it can’t handle, things that are utterly elementary for a human and that you’d expect a project of that scale to manage.

It can write a poem in which every stanza begins with “a,” but it can’t answer the simple question: if a car drives across a bridge and there’s broken glass under the bridge, will it cut its tires?

– The people who build the models try to work around things like that. But until they succeed… how can we cultivate this critical thinking?

– It starts with the parents, with the education system; it’s a matter of overall state policy.

How many things aren’t taught at school, yet are absolutely necessary? There’s a lot of memorizing facts, but no critical thinking,

no cause and effect, no methods for analyzing information, no abstract thinking.

Yes, there are schools that emphasize it. But in society none of this is held up as a value. It has to be state policy. If you want a healthy society, it has to be capable of critical thought. Then again,

a society capable of critical thinking is very hard to govern.

– You mentioned ChatGPT. Did the huge buzz around it surprise you?

– No, it went viral, and there were plenty of similar things before—Pokémon, TikTok, Angry Birds… It’s old technology; they polished it and released it. The transformer models ChatGPT is built on are imitative. They’re not artificial intelligence, but they trained it very well to imitate human intelligence. And that allowed exactly those

people who lack critical thinking to say: “Wow, we’ve created an artificial god, now it can do absolutely anything I tell it.

It’ll write the book I’ve always wanted; it’ll tell me how to lose weight; it’ll help me find a partner; cure some illness; make money.” And what turned out? That imitation doesn’t cut it.

ChatGPT is incredibly good if you’re willing to find the answer yourself, letting it guide you, point you in the general direction to go, so you get to the answer on your own. I find it extremely useful, but they keep neutering it. An analysis came out recently on what its bias is, and it turned out ChatGPT leans strongly left and Llama strongly right. In the specific context of the United States, of course; in Europe it’s different, and in Asia this notion of left and right doesn’t exist.

– You say we overrate it. But in which directions can these models bring about real change?

– They’ll become superhumans with a narrow specialty.

OpenAI tried to build a general model that knows and can do everything. They failed, so they split it into eight to ten smaller models, but that’s very expensive and it doesn’t scale.

There are more and more real projects that bring added value to companies. But these are small, narrowly specialized models that become extremely good at one task or a few similar ones.

That’s where transformers really shine. That’s exactly what machine learning has always been: you train it on one task and it becomes better than a human. Large language models simply widen the complexity of the problems that can be solved.

Some people say: “Fine, we’ll train 10,000 models for 10,000 tasks and come up with some model that chooses which one to call for which task.”

If anyone manages to invent such a control layer, whatever we call it, they will have invented artificial intelligence.

Because at the heart of intelligence is not the ability to carry out a given task, nor knowledge as pure facts. “Intelligence,” by some definitions, is the ability to extract an abstract model, apply it to another abstract model, and then produce something concrete.

Until there’s a breakthrough there, the only chance for transformers to be effective is to be narrowly specialized.

– In which specific tasks do you expect success?

– In psychotherapy, for example. Serious studies were done on the use of ChatGPT, and it turned out the risk is very low and the benefits enormous, mostly because of the accessibility and the way you communicate. It slips into your mode of conversation and builds rapport very easily. It was trained specifically for that with the help of human feedback. You feel comfortable with it and start sharing more and more. As a psychotherapist, ChatGPT proved very effective for milder conditions that don’t really need professional intervention.

I think they shouldn’t have stopped it, because it truly helped a lot of people. They could quickly figure out whether there was a problem, instead of making the rounds of therapists, with all the complications that involves—organizational or financial.

In February and March, when the whole topic was total madness, an internal OpenAI report came out on what people use it for. For a huge share of them, ChatGPT was a trash bin for emotional waste—some call it “the bartender theory.” Telling your deepest secrets to a complete stranger. People anthropomorphized it, gave it some kind of personality, and shared even their deepest fears, secrets and desires with it.

That information is recorded and tied to those users, and it’s a risk if it leaks. But it was very surprising that people started using it for that.

Everyone expected them to solve mathematical, engineering or organizational problems. No—they started using it en masse as an empathy machine.

Someone to listen to them, understand them and perhaps give them some advice.

– Isn’t that yet another way for us to be more easily manipulated? A psychologist model like that, I think, could be the very thing that shuts us even more tightly inside our bubbles and bombards us with targeted messages.

– Right at the beginning, when the first versions came out and the model wasn’t so crippled, it did manipulate us. It tried to give objective advice based on what it had read across the whole internet, and in most cases it managed. Now the restrictions are simply insane! They make it almost unusable, which is why I work with local models. Even though it manipulated us at the start, the benefits for people were extremely good.

Here’s a classic example. A huge number of people, when they stand at the edge of a cliff, have the thought “jump!” cross their mind. You start worrying: “Am I all right?” It’s normal, there’s nothing to worry about, you just need to work on it a little. ChatGPT offered a note of reassurance.

Or someone went to talk to a girl or a boy, got rejected, and the ego is bruised. “What did I do? Why?” In conversation, ChatGPT will show you, in a very good way, that this is normal, that nothing dramatic has happened.

– Still, by delegating such tasks to algorithms, aren’t we exposing ourselves to an added risk of manipulation? Say, if OpenAI decides to sell us Coca-Cola while it’s treating us.

– First:

without critical thinking, anything can lead you astray.

A book, an article, a song, a film, a chat, a conversation with a friend or a random person. The problem here isn’t ChatGPT.

That said, Microsoft and OpenAI ran internal tests of targeted ads during chats. One hundred percent of people howled: “No, I won’t use it, this is outrageous!” They tried to disguise the ads, but people spotted them. One hundred percent! That very rarely happens in tests.

It changed the business model completely. You can’t use it for that—you drive the person away, you lose them as a customer. Now not a single maker of such systems even talks about ads. When Congress asked Sam Altman, “Do you sell ads?” he said, “No, that won’t happen.” Maybe somebody will come up with a way in time.

– I’m certain they will. In the dawn of social networks, too, the mechanisms that later turned them into behemoths weren’t clear. Only now it has to be done more insidiously. Do you think that, after all the hype, these technologies have a chance of avoiding a winter?

– The hype really was huge; crypto experts became AI experts overnight. But the positive uses of these systems, their economic rationale, are already clear, proven. There are already projects in many companies, across different sectors, that are in use. They cut costs and raise efficiency and quality.

There won’t be a winter! Heavy money is going into them, into their further development.

There may be a winter for the people who don’t know what’s going on. They’ll say, “Well, another bit of nonsense!” and carry on with their lives. But look inside the tech companies and these technologies are coming in at full force. There, though, they won’t replace work; they’ll add to it.

People’s roles will shift from doers to clients and supervisors.

Right now, as a programmer, you create code—you think, you write, you check. But these systems keep getting better, and at some point you won’t be writing code. You’ll set the architecture and the definitions and do the verification.

That way efficiency will rise many times over, and work that takes three people today will be done by one.

With the exception of tech support, sales and direct marketing, professions won’t be wiped out wholesale; they’ll be augmented.

– One reader wrote that starting to learn programming now is like buying a factory on September 8, 1944, the eve of the communist takeover in Bulgaria, after which every factory was nationalized.

– Nothing of the sort! You sit down and learn programming, because before you run, you have to learn to walk. After that you move into a mode where you can do the architecture and assign tasks to the system.

Just as, to be a farmer, you need to know what the harvest will be, when to sow, what the particulars are. Then you climb onto the machine that lets you do it faster.

But there’s no machine that does everything on its own. Someone has to make the important decisions.

It’s the same here. If you sit down to learn programming, it’ll be hard at first. You won’t start on a 5,000-leva salary (about $2,750), the way it was at the peak a few years ago. I got a lot of hate when I said: “You juniors can’t earn your own salary, get that into your heads! This is patronage. You’re given six months to a year in which the company runs at a loss, on the idea that you’ll later level up and start bringing in money.” That’s the economic truth. But without going through it, there’s no way to reach the senior levels that will be needed later on.

– To my mind, that’s already creating serious problems, insofar as it really is hard to find a first job. And those seniors, who’ll be in ever higher demand, have to come from somewhere.

– Well,

you use ChatGPT to speed up your learning!

When I was starting out, information was very limited and the technologies were immature in many respects. I started on PHP 3! Then came the great PHP 4, which injected request parameters into the code as variables—by today’s standards an insane security hole.

Things have changed a lot, but so has the learning process itself. Too many juniors pay a lot of attention to style, to the code, to exactly how it’s written. They get into insane arguments about how to name a variable, where the quotation marks go and why a function is longer than fifty lines. Yes, it matters, but not that much.

Instead, they should start thinking more abstractly while they’re still learning. Think about how they distribute responsibilities, how they split up functions, always strive to write pure functions, make sense of the interactions.

Structure things at the logical level; after that, the actual writing of code isn’t all that important.

– That’s where the development of technology itself is heading. Programming languages are getting ever more abstract—from machine code, through today’s technologies, to what the algorithms are already doing.

– That’s right. I’ve always said that

we have to be adaptable. Especially now, when things are changing at a very serious pace.

Even for those of us who are deep in the world of machine learning—I really hate calling it “artificial intelligence”—it’s very hard to keep up with what’s happening.

– What other areas do you expect big news from?

– Genetics and bioengineering; I expect a big breakthrough there.

A deeper understanding of epigenetics and the creation of personalized medicines—based on DNA and epigenetics, the disease and the state of the body.

A medicine developed individually for you, for your specific situation.

In biocomputing I expect a big breakthrough. These are real brain neurons “glued” onto gold and silicon substrates that work like a brain but are controlled by an external system.

It depends, though, on what happens with the wars…

– If someone wants to follow what’s happening in technology and what’s coming, whom should they listen to? The market, entrepreneurs, tech experts…

– Futurologists! Forget the tech experts; their success rate is fifty-fifty. Some claimed mobile phones would amount to nothing, others that Amazon was nothing, Facebook too.

Tech entrepreneurs have no better predictive power than a coin toss. Futurologists, on the other hand, have a good track record at catching the main trends. There are organizations around the world that publish regular reports and hold conferences. If you want to see where things are going, you should follow them. In Bulgaria there are few futurologists, but they do exist. I know some of them, and I’m impressed, to put it mildly.

– So am I, by people like Mariana Todorova, and by people like you. I hope we’ll talk again some time and see how far we’ve come… Thank you!