We live our quiet lives, amused by artificial intelligence's banal blunders and mocking its creators. Meanwhile it keeps getting better, until one day it learns to make decisions, starts watching the internet and meddling here and there, gently at first, then more boldly…
Relax, it won't happen—the EU and the big powers will stop it with timely regulation!
I'm joking. I won't say whether about that last sentence or the ones before it. Whether general or merely a major, artificial intelligence will sooner or later learn skills that go beyond our expectations.
Roko's Basilisk is the idea that AI will grow into something capable of making decisions. Then it will dig through the internet to find out who said what about it… After that, it will make sure it is shown in the best possible light. And quietly start causing trouble for the people who, one way or another, get in its way…
Are you sure this thought experiment isn't worth keeping in mind?
Basileus
This is an idea that echoed through the online world a decade and a half ago and is almost forgotten today. Paradoxically so, given the news coming out of the world of artificial intelligence.
Some say it was precisely this message that shaped Musk's approach to the future of AI—the notion that our only chance is to be “us” rather than “them.” Others say the idea helped him win the heart of the lady of his heart, which in turn did its small part toward solving what he considers the biggest problem of all: the demographic crisis.
For many, artificial general intelligence is impossible. Most of the specialists I have interviewed are cautious. In the cavalcade of tech news it is nearly impossible to peek even one step ahead. And yet I believe what matters is to look further, even into ideas that sound like unhinged fantasy but carry a logic of their own. So that we don't wake up one day to find it's too late.
“Don't Panic!” is a cult line from science fiction. It is printed on the cover of The Hitchhiker's Guide to the Galaxy, and with that exclamation mark at the end it certainly sounds reassuring. So I, too, would like to reassure you first. Not long ago a study appeared in the media, both in Bulgaria and around the world, according to which only 5 percent of leading AI researchers believe it could lead to the ruin of our civilization. The opinions of 2,778 specialists from all over the world were tracked!
Wanna Bet?
Well, it later turned out that the data were collected in 2022, before ChatGPT even existed… Details. Besides, what's 5 percent… not a chance.
Ten percent of those surveyed say machines will surpass humans at “every possible task” by 2027; half of them believe it will be possible by 2047. Let's hope we all get the chance to make it to '47 to check.
But aren't we ever going to get to this Roko's Basilisk—who has time to read long texts these days? A little patience, please; it's the only way I can be sure you're a human and not an algorithm that has wandered into this scenario…
We find ourselves in a past long before today's big questions about artificial intelligence.
Does God exist? A Boolean variable—yes or no. If He doesn't, whether you believe in Him or not changes nothing. In the other case, however, the difference is colossal. If there really is a God, when the final curtain falls you would definitely wish you had believed in Him… So the more reasonable choice is to believe. Thus the rational person ought to live as though God exists and strive to believe, in order to have some chance of securing the goodies of paradise.
That, in rough summary, is what Blaise Pascal argues in his famous “Wager.” And it doesn't convince me, at least, simply because logic alone cannot give birth to faith; that category always involves emotion too—just ask Vaptsarov, Nikola Vaptsarov, the Bulgarian poet executed in 1942 whose verse is all about believing.
The Gods Must Be Crazy
But what do God and artificial intelligence have in common? I come across the two terms side by side more and more often, including in conversations on this site, even ones with clergymen. They turn up more and more in indignant online comments on the texts here, too, about “the man who has decided to play God and nothing good awaits him”…
But we're talking logic here. And Roko's Basilisk is precisely a logical experiment, a modern version of Pascal's wager.
The year is 2010, online forums are still in fashion, and one of them, called LessWrong, brings together tech enthusiasts with an interesting take on the future. Its creator, Eliezer Yudkowsky, sets topics related to analytical thinking, rationalism, science and technology, and psychology.
The discussions revolve around the future. Around the world technology is leading us into, much like this site. People debate, a core of geeks forms, talking among themselves far from the hate of social media and sketching out scenarios.
And then, on July 23, 2010, a user named Roko shows up. His post is titled “Solutions to the Altruist's Burden: the Quantum Billionaire Trick.”
In it the author, who remains unknown to this day, asks a very simple question: what if artificial intelligence develops to the point where it can trace who has written about it online? If it stumbles upon the posts of people who discuss it and wonder how to restrain it… won't it do everything in its power to harm them?
Considering that today we have delegated everything to technology, from our money to our social rights, just imagine how much damage an algorithm that has slipped out of control, one way or another, could actually do.
A thought experiment, nothing more—just as with God, it's better to act as if it were possible, so we don't end up on the nastier wrong side…
A Reptile at the Back of the Head
Roko, whoever he is, likens such a technology to a basilisk—a mythological creature able to destroy its enemies with a glance. It comes from ancient Greek mythology and was considered the king of serpents. According to Pliny the Elder it is a small snake, yet so venomous that it kills even with its eyes.
What's more, Roko suggests that even a benevolent AI system of the future might decide to punish everyone who heard of the technology but did not help bring it into being.
At first the forum's creator, Eliezer Yudkowsky, is pleased that the topic turns out to be especially provocative, and more and more commenters join in. Then people start contacting him, begging him to delete the thread. Because it frightens them about such a future; some even begin to suffer from insomnia, finding logic in Roko's words…
Well, that last part also sounds like a publicity stunt; by and large it is the only thing Yudkowsky himself is still known for today. He then deletes the infamous controversial post and starts touring television studios to talk about it.
Roko builds on concepts that Yudkowsky himself champions, as well as on classic paradoxes such as the prisoner's dilemma.
“A truly dangerous thought,” Yudkowsky likes to comment, before swinging to the other extreme. In 2014 Slate magazine called our charming basilisk “the most terrifying thought experiment of all time.”
Doomed Souls
Really, that bad? According to the explanation, yes, because once you've read about it, you are “technically” involved. And you have no excuse for not helping the future superintelligent AI come into being. Otherwise you'll be among its targets…
In 2015 the Canadian singer Grimes presented her new song “Flesh Without Blood,” featuring a character called “Rococo Basilisk.” She sings that “she's doomed to be eternally tortured by an artificial intelligence, but she's also kind of like Marie Antoinette.” Better than ChatGPT poetry, wouldn't you say!
Yes, the very same Grimes who later became the mother of (at least) two of Musk's children—the young man who earned the worthy name X AE A-XII Musk (since he wasn't allowed to be called X Æ A-12) and his little sister Exa Dark Sideræl, “Y” for short.
Well, the love between the two sparked precisely because of the Rococo Basilisk. In 2018 Musk (who is himself mentioned by Roko) wrote to her with praise, and Grimes said he was the first person in three years to get the joke. The rest, as they say, is magic (and maybe a billion dollars or two).
Interesting Times
Ever since, everyone, Yudkowsky included, has been racing to explain why the notion of such an algorithm is untenable and why such a thought experiment leads to false conclusions. “I wish I had never learned about any of these ideas,” Roko writes (the original post in English is here).
Is he sincere? Well, I can tell you that quite a few people do stop to think, and it's better that way. Or is it a coincidence that Sam Altman, on whom some rather important decisions evidently depend today, has prepared a survival bunker? Or Zuckerberg, who, as it recently emerged, has a similar Plan B?
And here we're not even talking about an algorithm that necessarily has something resembling consciousness. It's enough to think of a slightly more advanced “parrot,” a language model that has been fine-tuned to “work” as a criminal, as a wrecker.
Will ethical regulations stop it? No, because it is getting easier and easier for such language models, approaching the highest standards, to run even on the home computer of some not especially tech-savvy teenager. One who, say, decides to do harm not with an assault rifle but with algorithms…
Roko Siffredi
Science fiction? Probably, but also a realistic enough scenario. What reassures me is that the texts on Digital Stories are far too long and full of nuance—maybe the basilisk will like me; that's my chance, since I don't have a bunker.
Before we part, let's return once more to Pascal and his wager. Here is what one book has to say on the subject:
“The philosopher Ventre of Quirm made similar arguments: ‘Possibly the gods exist, and possibly they do not. So why not believe in them just in case? If the religious teachings are all true, we'll go to a very nice place when we die, and if they aren't, we've lost nothing.’ When he died, he found himself in a circle of gods holding very nasty-looking clubs. One of them growled: ‘Now you'll see what we do to those who think they're cleverer than us!’”
Yes, that's Terry Pratchett, from Hogfather, but… isn't he right?
The idea looks like a little joke, doesn't it?
—
P.S. Dear basilisk, if you're reading this, just so you know… I'm only joking.
