Turing test

1 · What it is

Alan Turing introduced the game in “Computing machinery and intelligence”, published in the journal Mind in October 1950. He meant it as a practical stand-in for the old puzzle of machine thought. His starting point was a parlour game where a man and a woman hide from a questioner and each claims to be the woman. Turing then swapped one of the hidden players for a machine. Today the usual version has three players, with an interrogator kept apart from a hidden person and a hidden machine. The interrogator has to work out which is which.

Turing also made a forecast. In about fifty years, he wrote, computers would play so well that an average interrogator would have no more than a 70 percent chance of picking correctly “after five minutes of questioning”. Most commentators say the forecast failed, because in 2000 no program met that standard. Later headlines, such as the program Eugene Goostman fooling 33% of judges in a 2014 competition, rested on very small trials.

The best-known objection is the Chinese Room, published by the philosopher John Searle in 1980. Picture Searle shut in a room, using a program to answer notes in Chinese that come in under the door. His answers are good enough that people outside assume a native speaker is in there, yet he cannot read a word of Chinese. Searle’s point is that running the right program can make a computer look as if it understands language while nothing inside actually does. So, he concludes, passing the Turing test proves too little. His paper goes further and says no program, on its own, is enough for thinking. The reply critics use most often accepts that the man is lost but claims the system as a whole does understand.

In a March 2025 arXiv paper, Cameron Jones and Benjamin Bergen of UC San Diego ran the three-player version. Each round, an interrogator had five minutes to chat with one human and one AI witness at once. A system’s win rate is how often the interrogator chose it as the human. GPT-4.5, given a prompt to play a humanlike persona, was chosen as the human in 73% of rounds. That is significantly more often than the actual humans were chosen. LLaMa-3.1-405B, with the same prompt, reached 56%, which was statistically level with the humans. The two baselines did much worse, with ELIZA winning 23% of rounds and GPT-4o 21%. The authors present this as the first experimental result showing a machine passing the standard three-player form of the test. They also stress that a pass measures how humanlike a system seems, not how intelligent it is. In their earlier 2024 study, GPT-4 convinced judges it was human in 54% of chats, while real people managed 67%. In that study each judge talked for five minutes to just one witness, human or AI.

Google DeepMind researchers, in a position paper on defining AGI, accept that today’s LLMs already pass some versions of the test. They still judge it too weak to benchmark AGI. Their reasoning is that what a system can do is easier to measure than whether it thinks, so they define AGI by capabilities instead. Jones and Bergen answer that most benchmarks are narrow and fixed. They treat the test as something that adds to benchmarks rather than replacing them.