Child’s drawing or AI? The experiment drew enormous interest.
Almost 4,000 people took part in the test of whether we can tell 15 drawings by children under 11 from 15 images created by artificial intelligence! Time to sum up the results.
Did we rise to the challenge? Are the real or the generated drawings easier to recognize? Did the task turn out to be hard, or did almost everyone manage to guess all 30? Were the children themselves more accurate in their judgment? Why do the conclusions matter?
TL;DR
The article is long and detailed. If you’re only interested in the main conclusions, you’ll find them
here.
- The experiment drew enormous interest: in a month and a half, no fewer than 3,933 people took part!
- The participants’ average success rate is a mere 52 percent. Pure guessing would give exactly 50 percent.
- In other words, we can say unequivocally that we are unable to tell children’s drawings from generated images.
- Though by a small margin, participants rated the generated images as more creative and better in technique.
- The differences between the scores of adults and of children who took the test are minimal.
- For 6 of the children’s drawings and 9 of the generated images, more people got the author wrong than right.
- Some images stand out as recognized by fewer than 1 in 5!
- Overall, we are slightly more inclined to take a generated image for a child’s drawing than the other way around.
- If you haven’t done the experiment yet, it’s waiting for you here.
Keep Your Eye on the Picture
No fewer than 3,933 people took up the challenge between mid-May and the end of June, when answer collection for the experiment closed! The interest hasn’t stopped since, and the number of people who have taken the test is now approaching 5,000, although their results no longer count toward the analysis.
If you’re among them… you’re hardly going to be surprised that the results are far from perfect. More than that—they’re close to sheer guesswork. We excluded people who answered fewer than two-thirds of the questions, and still… Almost half the participants have more wrong answers than right ones!
The average score is 52 percent; pure guessing would give 50 percent! So based on the results, we can boldly say that today we are no longer able to tell children’s drawings from ones created by AI! (And that matters—here’s why!)
And yet… there isn’t a single image that failed to mislead at least 1 in 10 participants. Nor is there one that fooled more than 90 percent.
I should note that no fewer than 12 people achieved the maximum score of 30 points during this period. I should note even more that, despite my best efforts, there were two ways to cheat—because of an oversight on my part, the generated images all had exactly the same proportions, and after submitting, a careful look at the result revealed which answers were correct. Still, evidently few participants discovered the loophole, if any did at all. And the results are so unambiguous that they leave no doubt about the most important conclusion. We turned out to be unable to tell the two groups apart.
Two Beautiful Eyes. The Soul of a Child
Children up to 14 could also take part in the experiment. Did they prove better at recognizing the work of their peers? I wouldn’t say so. Granted, only 128 children took the test, but their average score is not far from that of the adults.
Still, there are some pictures, such as the one with the snowman, where the young participants proved more observant.
A Fair Grade
For the first 10 images (5 generated and 5 drawn), participants could also give a rating—for technique and for imagination. The truth is that in both categories the average ratings were quite close, just over 3 out of 5.
And yet… the generated images received slightly higher average ratings, almost without exception! Participants gave AI a slightly better grade than the children!
The AI-generated drawings get slightly higher average ratings for both imagination (+0.15 points) and technique (+0.10 points), and the highest-rated of all is drawing 9 (AI).
Here are some of the images that stood out either way, whether or not they managed to fool the participants:
Judged by the Pictures
So here are the conclusions in brief:
- The experiment drew enormous interest: in a month and a half, no fewer than 3,933 people took part!
- The participants’ average success rate is a mere 52 percent. Pure guessing would give exactly 50 percent.
- In other words, we can say unequivocally that we are unable to tell children’s drawings from generated images.
- Though by a small margin, participants rated the generated images as more creative and better in technique.
- The differences between the scores of adults and of children who took the test are minimal.
- For 6 of the children’s drawings and 9 of the generated images, more people got the author wrong than right.
- Some images stand out as recognized by fewer than 1 in 5!
- Overall, we are slightly more inclined to take a generated image for a child’s drawing than the other way around.
- If you haven’t done the experiment yet, it’s waiting for you here.
It’s also time for the promised prizes! The four participants who scored the maximum 30 points and left their email for the prize draw are: p.vas***@ard**.bg, kalina***@abv.bg, lubo***@abv.bg and dimitrana****@yahoo.de . They will receive selected books and congratulations on their impressive result!
And once again, heartfelt thanks to everyone who took part and to everyone who helped this fascinating experiment reach so many people!
And can you tell a generated voice from a human one? Try it here!
If you have ideas for the next Digital Stories experiments, I’m waiting for you here!
If the topic interests you, it’s definitely worth reading the interview with psychologist Elitsa Staneva-Britton, whose idea inspired the experiment. You’ll find it here.
