Ramblings of a Mad Man: The Plateau of AI

Warning: these are simply some impulsive thoughts I thought i’d share.

The ramblings of a Mad Man this time on the plateau of AI: I found the GPT-5 launch a really key moment for AI. Sam Altman focused on generating hype for launch (“PHD-Level Expertise on demand), because, well, why wouldn’t he it’s his business after all. But what we received was automated model routing, reduction in hallucinations, it also delivered a 256,000 token context window improving retention over longer conversations, and a bit more product rather than model fine-tuning. Now don’t get me wrong, I don’t wish to downplay the effort that goes into hallucination improvement but from what I have read in the market the hype hasn’t met the reality.

My view is, like all innovation, we reach a barrier to progress, a plateau – a bit like me on a Monday afternoon. The current plateau in perceived AI progress isn’t necessarily because the field is out of steam, but rather because:

1. LLMs are hitting diminishing returns on scale: Doubling parameters or training data no longer yields the same awesome leaps we saw from GPT-3 to GPT-4. The easy gains from “make it bigger, BIGGER, BIGGGERRRRRRR” have been mostly harvested.
2. The public’s expectations have shifted: GPT-3 amazed people by writing coherent paragraphs. GPT-4 amazed them with reasoning steps. Now, people expect human-level understanding, memory, and agency, which is orders of magnitude harder than better autocomplete. I mean don’t get me wrong my Mum’s mind is still blown it can do a recipe in the style of Elton John but others demand more.
3. Engineering challenges are now front-loaded: We’ve built incredible text-in/text-out systems, but AGI requires more: Persistent long-term memory, Autonomous multi-step reasoning, Embodied interaction (real-world context, not just token predictions), Reliability and truthfulness under adversarial prompts, Site-wide perception. This list will likely require more than is outside my realm of understanding, especially if we wish to get to the “Innovators” phase of AI evolution.
4. The bottleneck has moved from algorithms to architecture: LLMs are not “thinking” systems; they’re statistical sequence predictors. True AGI probably requires hybrid models combining symbolic reasoning, memory networks, world models, and self-supervision in dynamic environments – things that go well beyond transformers. I wish I could write more here and state the answer, but if I knew it then I’d be on a beach in Hawaii somewhere!
5. Lack of self-improvement mechanisms: Current LLMs don’t truly learn from experience. They rely on static training runs and retrieval, rather than updating their internal models through trial, error, and reflection. A pathway to AGI almost certainly requires models that can autonomously critique, adjust, and improve themselves over time not just wait for the next large training cycle.

If we have indeed reached a slowdown, I think the pivot won’t be abandoning LLMs outright, but integrating them as part of larger cognitive architectures agents with:

• – A reasoning core that can plan, hypothesise, and test
• – A knowledge base that isn’t bound by context windows
• – A sensory-motor loop for interacting with the physical or simulated world

Historically, fields hit a “local maximum” before a new paradigm shift. AI in the ’80s got stuck on expert systems, let’s not forget the AI winter too!!! Now we might be nearing the limits of text-only LLM scaling. The next leap may come from blending LLM fluency with explicit models of the world, perhaps something closer to how AlphaZero fused deep learning with tree search (ping me via LinkedIn for some articles on this if you’d like some).

I suspect we’ll see a few years of “sweating the asset” of the current models and how we apply them, refining LLMs, adding retrieval, multi-modal capabilities, and better guardrails, until someone finds that breakthrough architecture. But who knows, I’m sure somewhere out there, there is a group of geniuses building an open source model just as I type this that gets us to AGI 😉.

I see this possible lull as a positive; it gives us all breathing space to get the right adoption and responsible AI frameworks in place inside business, and bodies, learn what drives real value with AI in, and focus on applying beyond the hype. Then hopefully the advancements and innovation within models will start to accelerate again, but we will be in a much better position to reap the rewards!