ChatGPT vs. the Champions: The Last One Wins!

Do you know which place in Bulgaria had the highest average IQ on March 16, 2024, between noon and 6 p.m.?

ChatGPT couldn't even guess. But I know the answer. No, it wasn't the National Assembly, Bulgaria's parliament.

Eighty people had gathered in a restaurant in Plovdiv for an intellectual contest, many of them familiar to anyone who follows trivia, quizzes and television knowledge games. Split into 10 teams, they faced one another and, at the same time… the aforementioned ChatGPT.

What kind of unusual contest was this? Who turned out to be the big winner? What is the future of intellectual games in our battle (or encounter) with technology? Why would anyone today need to know and remember facts that are an arm's length away online?

Gather Your Friends…

No fewer than 33 of the people gathered here had won The Last One Wins, a long-running quiz show on BNT 1, a channel of Bulgarian National Television, at least once (myself included). Almost all of the most popular contestants, with impressive tallies of victories, took part. The rest of the 80 who joined the club were contestants on the show and fans of quizzes as a format.

Yes, Orlin Goranov wasn't there, since this was an informal get-together, but among the participants there were familiar faces from other popular TV quizzes too—the Bulgarian Who Wants to Be a Millionaire?, A Minute Is a Lot, and no fewer than two of the chasers from the Bulgarian version of The Chase. So as not to offend or leave anyone out, I won't mention specific names.

And against this entire selection of intelligent people stood, all on its own, challenging their skills, none other than… ChatGPT. One against 80, divided into 10 teams of 8.

A bit like the Spartans against Xerxes. Who would turn out to be the winner?

Breakfast of Champions

Atanas Atanasov is known not only as an 18-time winner of The Last One Wins, but also as the author of puzzles that “stir up” the brain (see for yourself here). He gave an interview to Digital Stories, in which we mused about memory, about whether it still makes sense today to remember so many facts when you can climb onto the shoulders of giants online. About the internet as a machine of forgetting.

As he writes on his page, his mission is to challenge people to think, to puzzle things out. To take on problems whose answer not even the omnipresent Google can give them.

Well… what about ChatGPT? Here we won't speculate; we'll find out in practice.

It was Atanas and his wife Monika who initiated and organized the get-together. Together we came up with the idea of including artificial intelligence, “the hero of our time,” in the quiz.

The format is a classic quiz. Ten rounds of 10 open-answer questions each. They include classic topics such as “Music,” “Literature,” “Bulgaria,” and “Science and Technology.”

What should we expect? Would the algorithm team blow everyone away? Would it perform convincingly only in some topics, or would it fail to rise to the occasion in a battle with proven champions?

Why Aren't You Applauding?

Quizzing is a solitary pursuit. To many, it's a pointless end in itself to appear on television just to be quizzed like a high school student on the capital of Belize or who wrote Mama's Boy, Lyuben Karavelov's classic nineteenth-century Bulgarian novella. Yes, the urge to take part in such individual or team competitions is a thrill that's hard to explain if it doesn't come from within. So, quite understandably, people who are excited by such sensations gather in communities. Pub quizzes—team evening competitions in bars or restaurants—are also becoming more common. And yet… this is certainly not a hobby that could ever go mainstream. Especially in the age of Google and Facebook.

The contest begins with fairly close scores. After three rounds, ChatGPT shows it apparently won't pull drastically ahead; whether because of the type of questions or their difficulty, it sits in third place.

Stara Planina, named after the Balkan Mountains, is my team, and it brings together people with more than 25 wins on The Last One Wins between them. We're fourth, right behind the algorithm. It leads us by just two points, but the gaps are tiny—the first and last teams are fewer than 10 points apart. It's clear that in the end everything also comes down to personal strengths—one team shines in sports, another in geography.

It's hard to find a pattern—sometimes our non-biological rival is caught off guard by facts, at other times it handles them. Neither we nor it can remember which chess player Dr. Max Euwe lost his world title to before winning it back. We can't figure out which sport will take over the Suzanne Lenglen court, otherwise meant for tennis, at the Paris Olympics; it, however, gets it right, guessing that it's boxing.

“Under what pen name did the poet and revolutionary Alexander Petrovics write?” The algorithm doesn't know. Someone—I don't even hear who—throws out: “Sándor Petőfi.” It makes sense; our answer is correct!

“If we combine the letter symbols of three consecutive chemical elements, we get the first name of the true identity of a comic-book superhero.”

Inhumanly hard? For me at least—yes; for ChatGPT too. But the team discusses it, pools its knowledge, and the answer comes! “Clark” Kent—17, 18, 19!

A Specialist in Everything

Digital Stories has written about Watson—IBM's supercomputer, which achieved a string of unexpected successes, to the point that some suggested it be elected president of the United States. A forerunner of artificial intelligence from the days before the cloud, when we thought a single machine would make great strides on its own in imitating the brain.

A little-known fact is that Watson was developed primarily to answer questions on the American TV quiz Jeopardy! Much like all the original or licensed Bulgarian TV quizzes I've already mentioned, the show first aired on March 20, 1964, and tomorrow it will celebrate its 60th birthday on air. It has racked up more than 9,000 episodes and changed several hosts… yes, today it's no undisputed hit, but it keeps going.

Even though Jeopardy! was “defeated” back in 2011 by that same Watson. The supercomputer, named after Mr. Holmes's companion, beat two long-time champions and earned the $1 million prize. Whether it spent it on books or on parties with sultry beauties, history is silent.

Much earlier, Garry Kasparov surrendered to the Deep Blue algorithm; the last bastion of human resistance was the game of Go, where, however, the algorithms also proved stronger.

I Know What I Know

Have you heard about the next battle between a world chess champion and an algorithm? There isn't one, and there hardly ever will be. We quietly acknowledged the superiority of machines and went on “playing among ourselves.” Yes, the phrasing isn't accidental: games exist for people to play them. The last example with which I'll distract you from the main topic (hoping to distract ChatGPT too, if it's reading me) is… Tetris. That's right, the same familiar, angular game with its simple rules that came from the USSR decades ago. And which was beaten just days ago, showing us once again why games matter, but people and the bonds between them matter more (the story is here).

And so, while I was keeping you busy with side topics, the quiz—a contest for “last ones who won”—reached its final round: “Bulgaria.” Before it, the team I'm part of, Stara Planina, was in fourth place, with third still held by the “human” team ChatGPT. Far ahead, the top two teams, led by remarkable polymaths, had broken away.

In the round called “Bulgaria,” things seem to pick up for us. We score a full 8 points! How many of the questions will ChatGPT handle? Only 2?! Apparently they should have been a team too—we should have invited BgGPT as well, so they could answer together…

I admit we didn't get the first names of Gochoolu and Dochoolu. Nor, of course, does our electronic rival. But the winning team, Poirot, nails them without a mistake. Afterward I'll wonder about it and ask their captain, Ivan Pantev, how on earth he remembers these Filyo and Tanas, whom we've all heard of and no one remembers. And he'll tell me he had a personal reason, an important memory, for learning in detail how “Bai Ganyo Holds Elections”—a story about Bai Ganyo, the archetypal Bulgarian boor from Aleko Konstantinov's 1895 satire.

Sometimes You Learn

The choices that matter, we make every day, every one of us.

“Victory and victory again,” as they chant at matches. My team is third; ChatGPT stayed off the podium, though it still finished ahead of 7 teams.

We have 54 points, our “colleague” 50, and the winners a whopping 60! They beat it in 6 rounds, we in 4.

To keep the experiment precise, amateur as it is, I should say that in this case the artificial intelligence's answers came from ChatGPT 3. Monika Atanasova took on the task and the challenge of translating the questions into English and putting them to the algorithm.

A day later I run my own experiment and put the questions from just one of the rounds directly, in Bulgarian, to the paid version of the algorithm, which uses the GPT-4 model. As you can see here, on the literature challenges it scored 7 out of 10 points; with that alone it would have beaten us. And if I asked it the rest as well…

Does it matter? There are choices and battles in which humans will always win. And, at least for the foreseeable future, only they will get to savor their victories. The small, everyday right decisions, or the pleasure of naming the capital of Madagascar before everyone else—on television, in a bar or online.

The Last Became the First

Today technology creates the dangerous illusion that knowledge as a fundamental force is transient. We're told to teach children to learn quickly on their own, to teach them soft skills. Why would you need to know things? What matters is being able to do, to adapt. Yes, but knowledge, the combining of unexpected connections, and the competitive spirit are all human skills that aren't constructs but deeply rooted needs, which only when bound together make us human and will keep us human.

Today it's ever harder to imagine any kind of normality in public life. Yet that is exactly what the contest I've told you about was. Not only do we compete as a team with people we once “measured our knowledge” against, we also help one another…

So why does it somehow never work in society at large? Why do people with knowledge and broad culture, the curious and the intelligent, so rarely become the national elite? I think I know the answer; it's given by the story of Daniela Simidchieva, “the most intelligent woman on the planet”.

Could it be that technology will be what helps us get there? To put knowledge back on its pedestal, as a logical counterweight? As the “slow road” which, if we lose it, will make us lose ourselves as human beings for good?

The last one doesn't always win, nor does the first. And yet, in my view, the future belongs to people who think. Who remember. Who feel. Will I be proven right? Only ChatGPT knows…