This article is republished from Toest, the Bulgarian online weekly. It is part of the rubric A Portion of Language and was prepared together with Pavlina Varbanova.
While we debate whether artificial intelligence will ever write indistinguishably from a human (and it already can!), everything quietly changed. Whether we realize it or not, we now read algorithm-generated texts every day. The latest undeniable proof surfaced on social media a month ago: some of the most respected Bulgarian media outlets uncritically published texts riddled with the telltale sentences that unmistakably show they were written by ChatGPT or its kin. And that were not even read by their “author” before going online.
So can't we invite AI to save us… as an editor? Not long ago we tested how some of the most advanced AI models perform as proofreaders, and now it is time to raise the bar.
In our next major experiment, we put AI's skills as an editor to the test.
The Newsroom
In Bulgarian, the place where media content is created is called the redaktsia, the editorial office—and not by accident. Alas, as the information environment and the way we learn the news have changed, editors and proofreaders have landed on the endangered species list at most outlets. Editing a media text is a skill with countless facets: you are expected not only to know spelling and the rules of the language, but also to keep in mind the context, the subject, the content, the style and your audience.
Since so many outlets have no need for such people… let's see whether AI can replace them.
We choose to give it a short text translated from English. We take the original from the National Geographic website, and for the translation we use the popular platform DeepL—maybe it will turn out flawless and put our new colleague out of a job?
No, it does not turn out flawless. Both of us, as editors, spot quite a few errors that can be fixed, rough patches that need the editor's coarse file or at least some fine sandpaper. The only liberty we take is to shorten the already translated text, so that the edits the models propose will stand out more. The places where we would make changes are marked in bold.
Stop for a moment and look into the eyes of your dog. Do you feel overtaken by how cute they are? Or maybe you feel an irresistible desire to hug them? Now think about your habits. Do you send them to kindergarten, dress them in clothes and do you take them on vacation? Do you talk to them like to babies?
If so, you are not alone—after all, studies show that our brain reacts in the same way to domestic dogs, as well as to human children. Alison LaCos, a mother of three children, says that the moment she gave birth to her children, she felt an overwhelming desire to love them and to protect them. A similar phenomenon happened when she adopted Shio, a one-year-old dog of the Great Pyrenees breed, and Babka, a standard little poodle.
LaCos's behavior is not an anomaly—and a brain picture study conducted in 2014 gives some important clues to the reason. Researchers from Harvard University picked a small group of mothers, who went into machines for magnetic resonance imaging and looked at various pictures of dogs and children—some of them their own, others not. The researchers found significant overlap between the emotional experience of the mother-child bond and the mother-dog bond. The amygdala, an area of the brain that stimulates the formation of bonds and rewards, was activated when the women looked at pictures of their child and their dog. The same effect was observed in the hippocampus, the thalamus and the fusifor gyrus, which are parts of the brain connected with memory, social cognition and visual and facial processing.
“The areas of the brain connected with connectedness, love and connection were stimulated in a similar way,” says Niwako Ogata, associate professor of animal behavior at the College of Veterinary Medicine of Purdue University. The women also reported similar levels of pleasure and excitement when they looked at pictures of their children and dogs.
Man's Best Friend
We try three different assignments (prompts): 1) a short one, in Bulgarian (SBg); 2) a detailed one, in Bulgarian (LBg), in which we explain the task to the algorithm in detail—we hand it the editor's entire job description; and 3) a detailed one, in English (LEn), to see whether that might drastically improve its skills. We try nine of the most talked-about models, the latest cry from Silicone Valley, and… some of them fail spectacularly. (Incidentally, a seasoned editor would note here that the valley is Silicon, not Silicone.)
We keep six “contestants” in the experiment. Here are the superstar GPT-5; its predecessor o-3 from OpenAI, hastily pushed off the pedestal; Google's newest and most refined flagship, Gemini 2.5 Pro; and Anthropic's Claude 4 Opus. For the sake of pluralism we add the Bulgarian BgGPT and the free version of ChatGPT (which also runs on GPT-5, but with certain limits).
And here come the results. It should be easy, as it was with the proofread texts—we mark the errors and count who caught and fixed how many. Suddenly, though, we face a very serious and unexpected problem.
No People in Sight
The first question both of us ask, independently of each other, is
Why did we have to take this on?
The texts edited by the different models with the different prompts contain such a variety of solutions that they resist any systematic and comprehensive presentation, especially within a single article. It is also almost impossible to build an objective ranking in which to compare them.
That is why we decide to comment on the more important interventions (marked in bold), and on their absence, in one of the texts we consider best edited—the one by GPT-5 (LEn). Along the way we will slip in observations on the results of the other models.
Stop for a moment and look your dog in the eyes. Are you overcome with tenderness at how cute it is? Or do you feel an irresistible urge to hug it? Now think about your habits: do you send it to a doggy kindergarten, dress it and take it on vacation? Do you talk to it as you would to a baby?
If so, you are not alone—a number of studies show that our brain reacts in a similar way to pet dogs and to human children. Alison LaCos, a mother of three children, says that the moment she gave birth, she felt an irresistible urge to love and protect them. The same urge comes over her when she adopts Shio—a one-year-old Pyrenean Mountain Dog—and Babka, a standard poodle.
LaCos's behavior is not an isolated case. A brain imaging study conducted in 2014 by researchers from Harvard University involved a small group of mothers who underwent scans in a magnetic resonance imaging machine while looking at pictures of children and dogs—both their own and unfamiliar ones.
The scientists found significant overlap in the emotional experiences of the mother–child and mother–dog bonds. The amygdala—a structure involved in emotional response, attachment and the sense of reward—was activated when the women looked at pictures of their child and their dog. Similar activity was recorded in the hippocampus, the thalamus and the fusiform (spindel-shaped) gyrus—areas connected with memory, social cognition, visual processing and face recognition.
“The areas of the brain connected with connectedness, love and connection were stimulated in a similar way,” says Niwako Ogata, associate professor of animal behavior at the College of Veterinary Medicine at Purdue University. The participants also report similar levels of pleasure and excitement when looking at pictures of their children and dogs.
A Correct Corrector
Already in the first paragraph, most models have corrected the literally translated pronouns and plural verb forms—it is not the dog's eyes that are cute but the dog itself; the owner feels the urge to hug it (not them), and so on. They have also noticed the unusual use of kindergarten in this context. GPT-5, along with other models, has added the clarifying modifier doggy or for dogs, while some have settled for simply putting kindergarten in quotation marks.
Look your dog in the eyes sounds far more natural than look into the eyes of your dog—and here there is no other good option, unlike the source text's Do you feel overtaken by how cute it is? GPT-5's choice slightly shifts the meaning (Are you overcome with tenderness at how cute it is?), because what grips you may not be exactly tenderness when a strong feeling overtakes you, yet in this context the edit is fitting.
The AI recognized that A similar phenomenon happened is alien to natural speech and again replaced it with something suited to the context—The same urge comes over her. Here are two more apt substitutions: Surprisingly, the same feelings overwhelmed her (Claude LEn) and She experienced a similar feeling (Gemini LBg).
Of course, not every model handled these expressions relatively well, and when it comes to sensitivity, every last one of them failed the Purdue University test. The standard Bulgarian transcription, Пърдю (Pardyu), carries the unmistakable root of a vulgar Bulgarian verb for breaking wind, so the name is usually rendered as Пардю, with a different vowel. Translators from English, and from other foreign languages too, have experience with names like this and take care to avoid unwanted associations. One example from our socialist past: the general secretary of the Communist Party of the USA, Gus Hall, was called Гюс Хол (Gyus Hol). Today the name Gus is also rendered as Гас (Gas), while the version with the vowel ъ is not recommended.
An Unrepeatable Repetition
The source text contains repetitions that call for an editor's intervention. Two of them are in the second paragraph: children—children and to—to. GPT-5 dealt with the problem in the most effective way—by cutting. So far so good, but we note that these are repetitions of words, and apparently AI has no trouble flagging and editing them. At the end of the text there is a quotation with a different type of repetition—words sharing a root: connected—connectedness—connection. Add connected in the preceding sentence, and the situation becomes dire and clearly needs relief. GPT-5 did not register the problem, nor, for that matter, did several of its “colleagues.” Of course, some models attempted edits, but they were not particularly successful, and it is precisely in cases like this that the irreplaceability of the human (at least for now) becomes evident. Here is one way out of the situation, with no claim to being the best: The areas of the brain responsible for attachment, love and relationships with others were stimulated in a similar way.
PterEDITactyl
The text we tested AI with is popular science and contains quite a few terms. That means they have to be checked carefully. Although we have our reservations, many of the models handled this task well. GPT-5, as it wrote in its explanations, followed “the established Bulgarian terminology for breeds” and replaced a dog of the Great Pyrenees breed with the correct Pyrenean Mountain Dog, and the “illogical” little poodle with poodle. And indeed, standard poodles, also called royal poodles, are the largest in size, so using the diminutive is wrong.
Using brain imaging instead of brain picture is also a good call. GPT-5 noticed the missing m in fusifor (gyrus) and corrected it to fusiform (gyrus), but then, when trying to translate the word, it made a spelling error—spindel-shaped.
We also give a positive assessment to the edit of visual and facial processing. Our AI replaced it with visual processing and face recognition. Other models offered a more precise wording: the processing of visual information, including face recognition (BgGPT SBg, Gemini LEn).
Final Edits
As we take our leave, it is time to briefly share our personal impressions and conclusions from the experiment.
Georgi: Something that surprised me: the models performed almost equally well with quite different assignments—in length and in language. Clearly they are also getting better at understanding what we want from them, and the idea that prompting is the skill of the future is not so indisputable. Some of the proposed edits impressed me with their ideas and the errors they caught, others with the absurd additional errors they introduced into the “edited” text. But that is hardly a surprise.
Traditionally, the role of “devil's advocate” falls to me (in literature, AI has long been associated with the horned one as well). But whether it is a devil (still) depends only on us. Of course, it remains as unacceptable as ever to use these incredible new skills uncritically. To take things on trust, to copy, to publish—and done. The salvation of those who write is in the hands of the writers themselves. And yet, at a time when there is no way I can have an editor for my site Digital Stories, the models are invaluable to me. We saw it in this experiment too—especially when the work cannot be done by a human, a large part of a good editor's job can be taken over by algorithms.
Even the sharpest and most experienced editorial eye can make a mistake. The virtual one can too. My rule is to ask AI to suggest edits for every text I write, and then to choose which of its suggestions are apt. Very often they are, and you can see for yourselves—you will find all the proposed edits, together with the models' explanations, here.
Pavlina: The main conclusion of this experiment for me is that one should not be prejudiced about AI's capabilities. I did not expect, for example, that it would handle the terms at such a level. You will still need to check them, because it will not notice everything that is wrong, but you can spare yourself a good deal of digging through specialized websites, because it will give you valuable hints.
I also did not expect AI to change the source text so substantially, although in the long prompts we did give it free rein. GPT-5, whose work we discussed, stuck closer to the original, but other models got rather carried away—adding information and concluding sentences, for instance, or reformatting parts of the text (with bullet points). So you really have to be careful when formulating the tasks in your prompt.
AI can also be useful with ideas for varying your phrasing, and it makes some good hits when rewording expressions—again, it can save you time, especially if you are stuck and wondering, “Now how do I say this more clearly?”
Pavlina and Georgi: In our analysis we focused more on AI's achievements in editing, because what matters is what it is good at and how it can help us. We went easy on it in our criticism, but that must not dull our vigilance; we have to keep it on a short leash, because it may well mutilate, if not the whole text, then at least part of it. The final say still rests with us, the humans, and we must not let it slip from our hands, at least for the foreseeable future.
P.S. This text was written and edited by human beings only. Word of honor!
