How close are we to models that generate video indistinguishable from human work? Let's find out together with one important experiment.
Generative models are evolving at astonishing speed. Two years ago I ran the same test with images: I fed Midjourney classic Bulgarian poems to see how well it would cope. Would it catch the message? Would it create worthy images? Gradually the algorithms grew better and better at the task, so much so that today we can no longer tell the created images from the generated ones.
But what about video? What kind of clips might artificial intelligence create from a selection of classic poems? I chose two platforms that take quite different approaches to generation. And the results, to me at least, are definitely impressive…
A Story in Pictures
First, a few important clarifications. Algorithms are already quite good at “animating” an illustration they are given, even at adding sound. They don't do all that badly even in Bulgarian:
Here, though, we'll be testing something entirely different: how well they do when we leave them to create on their own, with no extra instructions, no expectations, no human intervention. Grab the popcorn—it's time for some unexpected cinema!
1. My Spring…
Assignment:
Spring
My spring, my white spring,
not yet lived, not yet celebrated,
only dreamed in luminous dreams,
how you pass low above the poplars
but do not halt your flight here.
My spring, my white spring—
I know you'll come with rain and hurricanes,
terribly stormy, fiery and rebellious,
to bring back thousands of hopes
and wash away the bloody wounds.
How the birds will sing among the wheat!
Joyfully they'll swim through the open sky…
People will rejoice in their labor
and love each other like brothers.
My spring, my white spring…
Let me see your first flight,
that gave life to the dead squares,
let me see only your sun
and—die on your barricades!
Nikola Vaptsarov
(Click on the title to read the poem.)
Video (generated by InVideo):
It may be a scorching summer, but, as we can see, generative technologies are already living through their spring, which is why we begin with this classic by Nikola Vaptsarov, the poet executed in 1942. On the recommendation of people with experience in the field, I chose the InVideo platform for the experiment. I tried Bulgarian first… the result was far from good, so I resigned myself to feeding it English translations. I picked poems that, in my view, have good enough versions in that language; almost all of them were translated by Christopher Buxton, who once visited us with his memorable Digital Story.
I have some additional settings to play with: I can choose exactly how creative the AI director should be, which audience the clip is intended for…
Aaaand… the result is remarkably good! Yes, maybe there are no storms and hurricanes, but the spring is so expressive, and the narrator really puts his heart into the emotional lines.
This is the moment for an important clarification. Here I'm using a platform that casts artificial intelligence precisely in the role of director. It interprets the text it is given and, based on it, selects ready-made short clips and photos, edits them together and adds a voice-over.
And the result is remarkable; we'll see a few more interesting examples shortly.
But how far have the other algorithms come, the ones that can create video without relying on ready-made footage? The heirs of the image generators…
2. …Beautiful Eyes
Assignment:
Two Beautiful Eyes
Two beautiful eyes. A child's soul
in two beautiful eyes;—music—rays
They neither ask nor promise…
My soul is praying,
child,
my soul is praying!
Passions and misfortunes
will cast on them tomorrow
the veil of shame and sin.
The veil of shame and sin—
they shall not cast it on them,
passions and misfortunes.
My soul is praying,
child,
my soul is praying…
They neither ask nor promise!—
Two beautiful eyes. Music, rays
in two beautiful eyes. A child's soul…
Peyo Yavorov
Video (generated by KlingAI):
Are there eyes? There are. Are they a child's? They are.
Of course, you have every right to be disappointed—a measly 5 seconds, and such a literal interpretation at that. Yet they show that we are on the verge of taking a huge step.
In mid-February this year OpenAI once again caught the eye of the whole world with yet another stunning model, called Sora. It showed off remarkably convincing videos generated entirely by artificial intelligence. Everything suggested that the model already understood three-dimensionality, which really is a huge leap forward. Then it was announced that Sora would undergo further testing and would afterwards be made available to every user.
Well, half a year has passed and that still hasn't happened. Perhaps the model turned out not to be as good as the demo clips made it look, perhaps new ethical questions came up, or perhaps the company simply shifted its focus to other, more important projects, such as the long-awaited GPT-5… As so often happens with OpenAI, “open” in name only, the answer remains behind its wide-shut doors.
And still, perhaps video generation is closer than the pessimists imagine? Other models are gradually appearing that promise the same thing—fully generated video. That is exactly what you see here.
3. Beautiful, but Not Ready
Assignment:
Lord, How Beautiful You Are!
How beautiful you are!
Lord,
how beautiful you are!
How beautiful your hands are.
And your feet, how beautiful they are.
And your eyes, how beautiful they are.
And your hair, how beautiful it is
Don't torment yourself any longer—love me!
Don't spare yourself—love me!
Love me
with the true strength of your hands,
your feet, your eyes—with all
the grace of their movements.
Believe me forever—and never
will you be foolish—love me!
And if you're wicked—love me!
Love me!
In the streets, and then on the stairs,
on the stairs especially you are beautiful.
With clothes and without clothes, ceaselessly
you are beautiful… Most beautiful in the room.
In the dark, when you are with your comb.
And the comb sinks into your hair.
Your hair is full of electricity—
if I touch it, I'll light up in the dark.
You truly are beautiful—believe me.
And try to be beautiful to the end.
Not so much for me as for yourself,
for the trees, the windows and the people.
Don't destroy your beauty too soon
with jealous suspicions—forgive me
my sudden vanishings somewhere—
don't overdo it, please, with the cigarettes.
Never lose me—discover me,
fill me with childlike wonder.
So I may once again be sure of your hands,
your feet, your eyes… Love me!
How I want to keep you forever.
To love you always—
forever.
And how impossible it is for me… How
made of sand you are… And please, don't tell me
that you want to keep me forever,
to love me always,
forever.
How beautiful you are!
Lord,
how beautiful you are!
How beautiful your hands are.
And your feet, how beautiful they are.
And your eyes, how beautiful they are.
And your hair, how beautiful it is.
How beautiful you are!
Lord,
how real you are.
Hristo Fotev
Video (generated by KlingAI):
Yes, I see it too; let me make the skeptics happy. The model chose the very thing that still gives image generators the most trouble to this day—hands. Yes, the video is undoubtedly unconvincing; you can tell at once that it's generated… But I remember how it was exactly the same with image creation. We had seven or eight fingers, no feel for anatomical detail… Then the models got better at solving that problem with astonishing speed. Not that they don't make mistakes now, but it happens rarely and not so crudely.
And so, here the platform tried to capture the tenderness of Hristo Fotev's verse but stumbled over the aesthetics.
By the way, for this part of the experiment I'm using the KlingAI model. As its name echoes, it comes from China. And that is another important topic that gets little attention. Amid all the progress of artificial intelligence in the Western world, we hear rather little about the enormous achievements of the East in the same field…
How will the “Chinese director” handle the next assignments?
4. The AI Sleeps in the Silent Dark
Assignment:
The City Sleeps
The city sleeps in the silent dark.
The faithful son of the faithless night,
I, homeless and alone—
and the rain drizzles, drizzles, drizzles…
Trembling by the black walls,
my footsteps echo in measured beat
and behind me, unseen, there follow
laments for the days gone by.
The image of the dear child
who once lit up my threshold
rises again in memory—pure and dear—
and the sorrow grows, grows, grows…
She came—a child—with awakened ardor,
with a fiery smile upon her lips,
and I, who thirsted for eternal beauty,
rejected her mortal gift.
The past—ah, it remained
a dark land, frozen with sorrows,
and from there she sends to me
her grieving cry: why, why?
The city sleeps in the silent dark.
The faithful son of the faithless night,
I wander homeless and alone—
and the rain drizzles, drizzles, drizzles…
Dimcho Debelyanov
Video (generated by KlingAI):
Pretty good, in my opinion at least. Apart from the slightly crooked streetlamp posts here and there, I, for one, can't find any other weakness. It even seems to carry Debelyanov's melancholy… or am I just imagining it? What do you think?
It's no accident that video generation is such an important next step in the development of AI and of technology as a whole. When you create an image, the task is simply to draw it, pixel by pixel. But to generate logical, consecutive pictures that flow smoothly into one another and make sense, the models need a better understanding of three-dimensionality, of how the physical world is built. Once that is achieved, it will be an extremely important step forward in many other fields from which we expect big news—from self-driving cars all the way to robotics…
But enough unhealthy optimism; it's time for a little healthy nihilism with good old Petko Slaveykov, the poet and journalist of the Bulgarian National Revival.
5. Cultural Revolution
Assignment:
We Are Not a Nation
We're not a nation, not a nation, but carrion,
people who want to do nothing at all.
Everything's too heavy, everything's too hard for us!
“I don't know how! I can't!” is the common cry.
And we don't know, we can't, we won't
work for ourselves while there is time.
All we know, all we can, all we will
is to gnaw at one another in spite…
Among ourselves we're greedy, wild, uncouth,
before others we're quiet, meek, obedient…
Whoever comes along tramples on us,
because we're dawdlers, because we're inept…
Everyone cries, “Our lot is wretched!”—
yet everyone's intent is to ruin things…
We're not a nation! Not a nation, but carrion,
I'll say it again, and with that I'll be done.
Petko R. Slaveykov
Video (generated by KlingAI):
Hahahaha. I wonder if China had its own equivalent of Grandpa Petko back in the days of the Cultural Revolution? Here our colleague completely missed the context; only the rebelliousness made it through.
Algorithms of the world, unite!
6. People, Houses, Splendor
Assignment:
Proud Balkan Mountains
Proud Balkan Mountains,
beside them the Danube shines blue,
the sun lights up Thrace,
above Pirin it blazes.
Dear Motherland,
you are a paradise on earth!
Your beauty, your charm,
ah, they have no end!
Tsvetan Radoslavov
Video (generated by KlingAI):
Video (generated by InVideo):
And what will happen next, when the two platforms we're testing join forces? What if we turn them into AI agents—platforms like KlingAI generating short videos, and “colleagues” like inVideo arranging them, assembling them, adding the voice and making further sense of them?
The sky's the limit, as programmers so often like to say.
Well, yes, it's still a long way off, but I, at least, see enormous progress. The first platform managed to generate a typically Bulgarian kind of beauty, while the second took us on a whole virtual journey around Bulgaria, even though it didn't quote the actual words of the anthem I gave it—“Mila Rodino” (“Dear Motherland”), Bulgaria's national anthem… And although there were some clearly misguided, far-fetched choices, with shots from Asia, on the whole the result was a good deal better than quite a few ads for domestic tourism I've seen.
AI makes mistakes? Well, natural intelligence in our latitudes sometimes confuses the Rocky Mountains with Vitosha, the mountain above Sofia, too. Only it won't get any “smarter” with each new version…
7. The Things It Has Borne…
Assignment:
September
The night gives birth from a dead womb
to the age-old rage of the slave:
its purple wrath—
majestic.
Geo Milev
Video (generated by KlingAI):
Video (generated by InVideo):
I included these lines in the image experiment two years ago, and here I give them to both platforms. In my view, how well they do with them says a lot about how good the algorithms are at seeking out context and metaphor, at reading between the lines…
The image in the fully generated clip is strong, I think, and there's a metaphor in it too. The second platform, meanwhile, came up with some rather strange and unexpected associations. Yes, I'm starting to notice that the narrator reads most of the poems with a similar intonation (but flawlessly!), though that's a very small and easily solved next step…
Where on earth did it get those baseball associations, though? Yes, the “director's” American handwriting shows…
8. “I Want to Tell All Who Are Sleeping”
Assignment:
Evening Trumpet
Life spins us round under the blazing sun
and we wear out our feet on the scorching stone…
But as soon as evening descends from the sky,
I'll take my trumpet and sit on the threshold.
Enough have I wandered along these walls
like the ring of a cracked bell.
I must play, I must bring down
the silence—so only the cry remains.
I want the hot wind to thunder
and throw the doors wide open.
I want the earth to set off again
after the crusading march of the crickets.
I want the barbed wire in front of your house
to be torn by my song.
I want the neighbor who pretends
to be deaf to get his hearing back.
I want the thief to tie up his fingers,
the watchman to buy himself a heart.
I want to let a drop of my tears
fall into the eye that is rusting.
I want the fair to come back to us
and blow the dust away.
I want the one who is dying of boredom
to die of laughter and tickling instead.
I want us to stand over the dead
as if on guard until morning.
I want to tell all who are sleeping
that there will be time to sleep…
I must play in the deaf evening
until I hear coming toward me
the voice of a thousand distant trumpets.
Or of some invisible archangel.
Boris Hristov
Video (generated by InVideo):
Quite charming. We have a new narrator, with a rather fitting intonation.
Cool trumpet! AI can already score (almost) a 6, the top grade, on the literature matura, Bulgaria's school-leaving exam—I've checked. Here, on the “video matura,” it's still far from the truth, but it's visibly making progress and offers interesting food for thought…
9. Romance Today Lives in the Robots
Assignment:
Moan
To Lora
My soul is a moan. My soul is a call.
For I am a bird shot down:
mortally wounded is my soul,
mortally wounded by love…
My soul is a moan. My soul is a call.
Tell me, what do meeting and parting mean?
And here I tell you: there is hell and anguish—
and in the anguish, love!
The mirages are near—the road is far.
The wonder-smiling joy of life
of innocence and greedy youth,
of sultry flesh and a light phantom…
The mirages are near—the road is far:
for she stands in radiance before me,
stands, yet does not hear who calls and moans—
she, flesh and a light phantom!
Peyo Yavorov
Video (generated by InVideo):
I think it did best here, what do you say?
10. AI Visits the Devil
Assignment:
A Visit to the Devil. A Tale of Honor
I never had hoped in the whole of my life
for so gracious a compliment:
the Devil invited me—the old Devil himself—
to his home for a glass of absinthe.
The candle traced out his sharp profile
with a strip of golden rays
and, blowing smoke rings, Mephistopheles
looked at me with moist eyes.
An autumn sorrow was hidden in his eyes,
yet still he was proud and amused,
and he waved his hand: “In vino veritas!
I'll be candid with you!”
I'm weary by now of this yoke of
pretense and wicked intent—
let us drink to my unearthly sincerity
and the gray earthly flesh!
Centuries ago I stopped on the earth
and played myself a joke here:
I was wed to the sacred earthly Truth,
but she crowned me with horns.
I burned with jealousy, and in black hatred
for my trampled honor—
human honor I tirelessly trample,
yet I've gained no honor to this day.
I thought to shine through wondrous feats—
I died a hundred times in battle,
always a knight of an honorable cause,
yet still I was not crowned with honor.
Desperate, wretched, one day on the boulevard
I set out unearthly forlorn.
And I raised above me a glaring placard:
“A man without a drop of honor!”
But strange: not a drop of contempt!
Everywhere they greet me with interest,
from all sides they politely doff their hats:
“You have no honor? It does you honor!”
One gentleman kissed me: “Ah, brother,
you too!… Well, who would have known!”
Two lovely ladies said sweetly to me:
“Come round to ours tomorrow for tea!”
Marvelous! Unheard of! With rare honors
I was escorted all the way home.
Ministers, kings and court coquettes
wrote me charming letters.
And here I am: important, brilliant, elegant,
as rich as a veritable Croesus!
And I know: I'm a thief, a liar, a profiteer,
a man without honor; but… always with honor!…
And the Devil fell silent. He poured more absinthe,
clinked glasses with me warmly,
and, letting out smoke in bluish ribbons,
pierced me with his green gaze.
Hristo Smirnenski
Video (generated by InVideo):
The devil is in the details, I know…
And still, it's impressive how far video generation algorithms have come, and the next interesting news in this direction is surely very close.
What if we imagine a future in which algorithms already read our minds? They know from our mood what kind of film we feel like watching, who our favorite actors, scenes and directors are. And for every minute they prepare something made to please us, to satisfy us, something just for us?
Maybe there's no need to imagine it. Maybe that future will be here very soon. Unless, of course, it turns out to be impossible…
