Education & Career Trends: October 5, 2024
Curated by the Knowledge Team of ICS Career GPS
Article by Will Lockett, published on medium.com.
One of the most pressing issues with Large Language Models (LLMs) like OpenAI’s ChatGPT is their tendency to hallucinate—spouting false claims that can be outright alarming. Take, for instance, instances where Grok AI, developed by Elon Musk, has accused him of serious crimes, or where ChatGPT inaccurately stated that Musk had been dead for years. These blunders highlight a crucial flaw in AI reliability.
Recently, OpenAI stirred excitement in the tech community by announcing its latest model, o1 (also known as “Strawberry”), claiming it effectively addresses the hallucination problem by fact-checking itself and rigorously analysing all parts of a question. If this claim holds, it could mark a monumental advancement in AI technology. But how does o1 achieve this? Spoiler alert: it might not.
Understanding o1’s Approach: Chain-of-Thought
This prompting technique encourages the AI to break down a question into smaller, manageable steps, ostensibly allowing it to check each part for accuracy before concluding. By doing this, the AI is thought to have a better chance of correcting its hallucinations; if a previous step contains an error, it gets another opportunity to rectify it in the next phase.
At first glance, this seems like an elegant solution—an innovative tweak that could potentially eliminate one of the most frustrating aspects of AI chatbots.
Experts Weigh In: Is it really an improvement?
However, a closer examination of o1 by AI experts has revealed some concerning issues that challenge the model’s effectiveness. Instead of simply resolving hallucinations, o1 might inadvertently exacerbate the problem.
In traditional chatbots, hallucinations typically arise from generated content. With a chain-of-thought model like o1, hallucinations can now originate from three sources:
- The generated content: What the AI ultimately produces.
- The displayed chain-of-thought: The reasoning steps that are visible to the user.
- The hidden chain-of-thought: The internal reasoning steps that remain unseen.
Surprisingly, the visible chain-of-thought provided by o1 isn’t the actual thought process it uses to interpret prompts. Instead, it resembles a summarised version, allegedly to protect proprietary information critical to its functionality. This lack of transparency creates challenges in identifying and addressing hallucinations.
The Difficulty in Diagnosis
The consequences of this hidden layer of complexity are significant. If false facts or logical flaws are embedded in the hidden chain-of-thought, they can be nearly impossible to identify. This situation is worsened by recent studies indicating that as LLMs expand in size, their performance on basic tasks like arithmetic can diminish. Such oversights may be buried within the hidden reasoning processes, making them difficult to uncover and correct.
Consequently, rather than enhancing the model’s ability to fact-check itself, o1 could instead create an environment where hallucinations are concealed more effectively, leading to a potential increase in unchecked inaccuracies.
Conclusion: A leap of faith
In light of these findings, several AI experts advise caution when using o1, urging users to scrutinise the displayed chain-of-thought for any signs of hidden hallucinations. Some even characterise reliance on o1 as a “large leap of faith.”
So, while OpenAI may herald o1 as a breakthrough in AI technology, the reality could be less revolutionary than anticipated. As always, critical engagement and diligence remain essential in navigating the evolving landscape of artificial intelligence.
…
Have you checked out yesterday’s blog yet
AI Is Hitting Limitations It Simply Can’t Overcome
(Disclaimer: The opinions expressed in the article mentioned above are those of the author(s). They do not purport to reflect the opinions or views of ICS Career GPS or its staff.)
Like this post? For more such helpful articles, click on the button below and subscribe FREE to our blog.
Download our mobile app, ICS Career GPS, a one-stop career guidance platform.
One Reply to “OpenAI’s “Revolutionary” o1 Model Has A Fatal Flaw”