“I tend to believe what it says”… Godfather of AI, Geoffrey Hinton on trusing AI chatbots

Godfather of AI

Geoffrey Hinton, often referred to as the “Godfather of AI,” has admitted that he trusts his go-to chatbot — OpenAI’s GPT-4 — a bit more than he should, even though it still makes basic reasoning mistakes,

In an interview with CBS, Hinton said “I tend to believe what it says, even though I should probably be suspicious,” Hinton said in a CBS interview aired on Saturday. The AI pioneer, who received the 2024 Nobel Prize in physics for his contributions to machine learning, revealed that GPT-4 incorrectly answered a simple logic riddle he posed.

“Sally has three brothers. Each of her brothers has two sisters. How many sisters does Sally have?” Hinton asked the model.

While the correct answer is one — Sally is one of the two sisters her brothers have — GPT-4 responded with “two,” assuming Sally had two other sisters besides herself.

“It surprises me. It surprises me it still screws up on that,” Hinton remarked, calling attention to the persistent limitations of today’s large language models. “It’s an expert at everything. It’s not a very good expert at everything,” he added.

Hinton, who said he uses GPT-4 for daily tasks, expressed optimism that future models would show better reasoning capabilities. When asked if GPT-5 might handle the riddle better, he responded, “Yeah, I suspect.”

The segment triggered widespread reaction online, with many users saying that newer models — including GPT-4o and GPT-4.1 — got the answer right when they tried the same riddle. OpenAI has not issued a response to the broadcast.

Launched in 2023, GPT-4 was OpenAI’s most advanced model at the time and quickly gained attention for its performance on academic and professional tests. In May 2024, OpenAI introduced GPT-4o, its new default for ChatGPT, claiming it maintained GPT-4’s intelligence but offered faster, more versatile performance across text, voice, and vision.

The current leaderboard on Chatbot Arena — a crowd-sourced model ranking site — puts Google’s Gemini 2.5-Pro in the top spot, followed closely by OpenAI’s GPT-4o and GPT-4.5.

Despite progress, a recent study by AI testing firm Giskard suggests the models remain vulnerable to factual errors. Researchers found that when prompted to be brief, leading chatbots like GPT-4o, Mistral, and Claude were more likely to “hallucinate” or invent information.

These limitations highlight the ongoing challenge of ensuring consistency and reliability in generative AI systems, even as their capabilities continue to grow.