SAN FRANCISCO: A quietly growing belief in Silicon Valley could have immense implications: the breakthroughs from large AI models -– the ones expected to bring human-level artificial intelligence in the near future –- may be slowing down.
Since the frenzied launch of ChatGPT two years ago, AI believers have maintained that improvements in generative AI would accelerate exponentially as tech giants kept adding fuel to the fire in the form of data for training and computing muscle.
The reasoning was that delivering on the technology’s promise was simply a matter of resources –- pour in enough computing power and data, and artificial general intelligence (AGI) would emerge, capable of matching or exceeding human-level performance.
Progress was advancing at such a rapid pace that leading industry figures, including Elon Musk, called for a moratorium on AI research.
Yet the major tech companies, including Musk’s own, pressed forward, spending tens of billions of dollars to avoid falling behind.
OpenAI, ChatGPT’s Microsoft-backed creator, recently raised $6.6 billion to fund further advances.
xAI, Musk’s AI company, is in the process of raising $6 billion, according to CNBC, to buy 100,000 Nvidia chips, the cutting-edge electronic components that power the big models.
However, there appears to be problems on the road to AGI.
Industry insiders are beginning to acknowledge that large language models (LLMs) aren’t scaling endlessly higher at breakneck speed when pumped with more power and data.
Despite the massive investments, performance improvements are showing signs of plateauing.
“Sky-high valuations of companies like OpenAI and Microsoft are largely based on the notion that LLMs will, with continued scaling, become artificial general intelligence,” said AI expert and frequent critic Gary Marcus. “As I have always warned, that’s just a fantasy.”
Time to Think
OpenAI has delayed the release of the awaited successor to GPT-4, the model that powers ChatGPT, because its increase in capability is below expectations, according to sources quoted by The Information.
Now, the company is focusing on using its existing capabilities more efficiently.
This shift in strategy is reflected in their recent o1 model, designed to provide more accurate answers through improved reasoning rather than increased training data.
Stevenson said an OpenAI shift to teaching its model to “spend more time thinking rather than responding” has led to “radical improvements”.
He likened the AI advent to the discovery of fire. Rather than tossing on more fuel in the form of data and computer power, it is time to harness the breakthrough for specific tasks.
Stanford University professor Walter De Brouwer likens advanced LLMs to students transitioning from high school to university: “The AI baby was a chatbot which did a lot of improv'” and was prone to mistakes, he noted.
“The homo sapiens approach of thinking before leaping is coming,” he added. —AFP
