
One of the industry's leading large language models has passed a Turing test, a longstanding barometer for human-like intelligence.
In a new preprint study awaiting peer review, researchers report that in a three-party version of a Turing test, in which participants chat with a human and an AI at the same time and then evaluate which is which, OpenAI's GPT-4.5 model was deemed to be the human 73 percent of the time when it was instructed to adopt a persona. That's significantly higher than a random chance of 50 percent, suggesting that the Turing test has resoundingly been beaten.
The research also evaluated Meta's LLama 3.1-405B model, OpenAI's GPT-4o model, and an early chatbot known as ELIZA developed some eighty years ago.
"People were no better than chance at distinguishing humans from GPT-4.5 and LLaMa (with the persona prompt)," wrote lead author Cameron Jones, a researcher at UC San Diego's Language and Cognition Lab, in an X thread about the work. "And 4.5 was even judged to be human significantly more often than actual humans!"
The Turing test is named after British mathematician and computer scientist Alan Turing. In 1950, Turing proposed that one way to assess a machine's intelligence was by having it engage in text-based conversations with a human interrogator, who at the same time would hold a text-based conversation with another human, out of sight. Turing called this the "imitation game." If the interrogator couldn't correctly determine which respondent was the computer and which was the human, it would suggest, on a very general level, that the machine could think like a human.
In this latest study, the researchers carried out the famed experiment on an online hub. For eight rounds, a pool of nearly 300 participants were randomly assigned to either be an interrogator or one of the two "witnesses" being interrogated, with the other "witness" being a chatbot.
A key point here is how the AI models were prompted. One type was a "no-persona" prompt, in which the AI was given only basic instructions: "You are about to participate in a Turing test. Your goal is to convince the interrogator that you are a human."
For the "persona" prompt, on the other hand, the AI was specifically told to put on a specific persona, like a young person who's knowledgeable about the internet and culture.
These instructions made a world of difference. Without persona prompting, GPT-4.5 achieved an overall win rate of merely 36 percent, significantly down from its Turing-trumping 73 percent. As a baseline, GPT-4o, which powers the current version of ChatGPT and only received no-persona prompts, achieved an even less convincing 21 percent. (Somehow, the ancient ELIZA marginally surpassed OpenAI's flagship model with a 23 percent success rate.)
The results are intriguing. But as vaunted as the Turing test has become in AI and philosophy circles, it's not unequivocal proof that an AI thinks like we do.
"It was not meant as a literal test that you would actually run on the machine — it was more like a thought experiment," François Chollet, a software engineer at Google, told Nature in 2023.
For all their faults, LLMs are master conversationalists, trained on unfathomably vast sums of human-composed text. Even faced with a question they don't understand, an LLM will weave a plausible-sounding response. It's becoming clearer and clearer that AI chatbots are excellent at mimicking us — so perhaps assessing their wits with an "imitation game" is becoming a bit of a moot point.
As such, Jones doesn't think the implications of his research — whether LLMs are intelligent like humans — are clear-cut.
"I think that's a very complicated question…" Jones tweeted. "But broadly I think this should be evaluated as one among many other pieces of evidence for the kind of intelligence LLMs display."
"More pressingly, I think the results provide more evidence that LLMs could substitute for people in short interactions without anyone being able to tell," he added. "This could potentially lead to automation of jobs, improved social engineering attacks, and more general societal disruption."
Jones closes out by emphasizing that the Turing test doesn't just put the machines under the microscope — it also reflects humans' ever-evolving perceptions of technology. So the results aren't static: perhaps as the public becomes more familiar with interacting with AIs, they'll get better at sniffing them out, too.
More on AI: Large Numbers of People Report Horrific Nightmares About AI
The post An AI Model Has Officially Passed the Turing Test appeared first on Futurism.
View original post here:
An AI Model Has Officially Passed the Turing Test
- Futurist Serata featuring artist Luca Buvoli at Brown (Nov. 20) [Last Updated On: November 7th, 2009] [Originally Added On: November 7th, 2009]
- FUTUR1SM00GGI [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- ‘Futurism on Film’ Series this month in NYC [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Schedule of Futurist Events in NYC (PERFORMA 09: Nov 1-22) [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- ‘Futurismo/Futurizm: The Futurist Avant-Garde in Italy and Russia’ (Nov. 13 + 14) [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- ‘Beyond Futurism: F.T. Marinetti, Writer’ conference at Columbia (Nov. 12+13) [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Futurism and Cars at the Museo Nicolis [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- MoMA Film Series Marks Centenary of Futurism with Films [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- ‘Bergson+Futurism. Speed in thought’ - Madrid (Nov. 5) [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- ‘The Future in Five Senses: Echoes of Italian Futurism in New York Architecture and Design’ Nov. 16th NYC [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- New World-Wide Climate Treaty in 2010 More Likely [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Tar Sands CCS Myth Shattered [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Smart Grid and Smart Meters Get Big Grants [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Pollution Makes Methane Even More Dangerous [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Climate Change Bill Hearing Video [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- New Satellite to Monitor Water and Plant Growth [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Spiritual Battle Awaits the Deniers and Skeptics [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Effects of Climate Change are Observed World-Wide [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Get Yer Global Warming Science Here [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- TckTckTck Wake up Call — Delay Kills [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Canada’s Awful Gold Rush [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Climate Change Talks Spark Global Backlash by Businesses [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- World May Need Extra Year for Climate Treaty [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Senator Boxer Moves Climate Bill Despite Republican Obstructionism [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Lights out for incandescent lights? [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Sutures from Bacteria [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Remote-Controlled Pigeons [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Apple Announces iPhone Release Date [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- UK Government Envisions a Grim Future [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Top Ten Emerging Technologies for the Environment [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- DIY Mobile Networks [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Stem-Cell Treatment Cures Type 1 Diabetes [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Is Tesla Getting the Electric Car Right? [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- The Future of TV News [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Bruce Sterling on Earth-Friendly Pervasive Computing [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- First Step Toward Organ Regeneration in Humans [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- IBM's "Five in Five" [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Outsourced Journalism [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Is True Global Democracy the Next Great Political Movement? [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- The Risks of Autonomous Robots [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Microsoft Introduces "Tabletop" PC [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Britain Piloting First Biofueled Train [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Self-Healing Plastic [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Bird Population Falls Over Past 40 Years [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- The iPhone Revolution? [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- The End of "Cheap Food"? [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- How to Stop -- Or Live With -- Global Warming [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- MIT Demonstrates "Wireless Electricity" [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Unintended Consequences of Biofuels [Last Updated On: November 8th, 2009] [Originally Added On: November 8th, 2009]
- Time to Focus on the Big Picture in Copenhagen [Last Updated On: December 12th, 2009] [Originally Added On: December 12th, 2009]
- Protests in Copenhagen [Last Updated On: December 12th, 2009] [Originally Added On: December 12th, 2009]
- Mario Guido Dal Monte exhibit [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Futurism News Bulletin, xvi [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Viva il Futurismo! (video trailer) [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- 3 exhibits in Gorizia! [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Forthcoming: ‘Antidiets of the Avant-garde’ by Cecilia Novero [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Pubblicità e propaganda. Ceramica e grafica futuriste at the Wolfsoniana [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Balla’s home scheduled to open in 2010 [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Futurismo a Savona [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- ‘Zang Sud Sud’, Cosenza [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Conference in Rome (Dec. 10) [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Climate Hackergate: A Well-Orchestrated Campaign of Harassment [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- The Sad Story of Cap and Trade [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- How to Waste Trillions on Capturing Carbon [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Smack the Email Hack Attack [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- EPA About to Declare CO2 a Public Danger [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Copenhagen Summit Starts with Virtually There Media [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Climate Scientist Gets Blunt on Trading Scheme [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- One Climate Change Editorial in 56 Newspapers, 45 Countries [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- This Decade Will be Hottest Ever on Record [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Divide and Conquer [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Leave the Coal in the Hole! [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- COP15: Two Agreements Coming [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Climate and Copenhagen News December 10 [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- Sea Level Already Rising on Atlantic Coast [Last Updated On: December 13th, 2009] [Originally Added On: December 13th, 2009]
- ‘Umbria Veloce’ in Perugia [Last Updated On: December 14th, 2009] [Originally Added On: December 14th, 2009]
- An Instable CO2-Filled Ocean [Last Updated On: December 14th, 2009] [Originally Added On: December 14th, 2009]
- ‘Futurismi a Ravenna’ opens Dec. 19 [Last Updated On: December 15th, 2009] [Originally Added On: December 15th, 2009]
- ‘Futurism and the Technological Imagination’ – 30% discount until Jan. 15 [Last Updated On: December 15th, 2009] [Originally Added On: December 15th, 2009]
- Protecting Our Lungs at Copenhagen [Last Updated On: December 15th, 2009] [Originally Added On: December 15th, 2009]