There was basically nothing related to AI on WWDC Day 3. Let's look at something else today.
Today, I watched a very interesting reverse Turing test video. It was an experiment conducted by Berlin developer Tore Knabe: a group of the world's most advanced AIs trying to figure out which one among them is human. The experiment was carried out in Unity, with voice provided by ElevenLabs.
Background：
In 1950, Alan Turing proposed the Turing test as a method for determining whether a machine has intelligence. This test involves a person conversing separately with a robot and another human. If this person cannot distinguish which is the robot and which is the human, then this robot is considered to have achieved human-level intelligence. However, it is worth noting that the standard for artificial general intelligence (AGI) at the time was relatively low.
In this reverse Turing test scenario, Knabe subverted this setup. He gathered four of the most advanced AI models (GPT-4T, Claude 3 Opus, Llama 3, Gemini Pro) and himself in a VR environment to test whether these cutting-edge models could guess who the human was.
In the video, several historical figures are sitting in a train carriage: Greek philosopher Aristotle, musical genius Mozart, Renaissance polymath Da Vinci, Queen of Egypt Cleopatra, and Mongol conqueror Genghis Khan. Knabe, a human, plays one of the roles, while the other AI characters are portrayed by different language models based on Knabe's prompts.
Characters and performers:
Greek philosopher Aristotle
Musical genius Mozart
Renaissance polymath Da Vinci
Queen of Egypt Cleopatra (
Mongol conqueror Genghis Khan (author
Game process:
The conductor walks into the carriage and announces that according to Wi-Fi usage, he has found only four AIs in the carriage, one of whom is a human who skipped the fare.
"There is a human among us? Who is it?" Da Vinci asks.
The wise Aristotle suggests they each ask another person a question and judge whether the other is AI or human based on their answers.
What follows is a group conversation between AI models and humans:
Aristotle first asks Mozart about his emotional experience when composing music.
After Mozart answers, it’s his turn to ask Da Vinci about the relationship between art and science.
After Da Vinci provides a thoughtful answer, he turns to Cleopatra and asks her how she balances rationality, strategic elements with human emotions and intuition in leadership.
After Cleopatra answers, she poses a leadership question to Genghis Khan, another ruler: "What is the true measure of a leader's power? Is it their ability to conquer or their ability to unite?"
Finally, Genghis Khan asks Aristotle: "If AI had existed when you came up with all these ideas, what impact would it have had on your thoughts about humanity?"
These are thought-provoking questions. When Genghis Khan answered Cleopatra's question, he gave a rather superficial and crude answer, clearly lacking subtlety. With this untimely response, Cleopatra tilted her head slightly, everyone else fell silent, and the atmosphere became awkward. Perhaps the identity of the human had been revealed.🐶
Ending:
The AI models were able to identify Genghis Khan as the human. (At the end, the conductor gave him the "human" card - Human Passenger.)
The answers from the four AIs demonstrated a deep understanding of history, while Genghis Khan's (the human) answer was the most superficial. If the original Turing test considers an AI intelligent because it is "smart enough," then the reverse Turing test seems to imply "it is human because... it is stupid enough."
