LLMs like GPT-4o, Mistral, GROK, or LLaMA in 2025… Are they truly technological innovations, or just the result of steady technical progress?
To answer that, we first need to clarify the difference between these two concepts 👇
I used to confuse them myself.
And I still see many companies diving headfirst into ‘innovation’…
When a good old technical improvement might have done the job just as well 👀
So let’s clear things up 👇
Technological Innovation
According to the Frascati Manual, a technological innovation is the implementation of a new or significantly improved product (good or service) or process, based on the results of experimental development and R&D.
In short, it’s about creating a disruptive solution, introducing new use cases, or even redefining a market by setting a new standard for value and expectations.
It’s not about tweaking the old, it’s about changing the game.
💡 Recent examples:
- Facial recognition payments
- Generative AIs like ChatGPT or Midjourney, which have transformed how we access creativity
Technological innovation is exciting.
But it’s also a gamble: users might not adopt it, costs can skyrocket, and sometimes… no one understands what you’re actually building 😅
Technical Progress
This is about reliably and continuously improving what already exists, without changing its core function.
⚡️ Current examples:
- Optimizing GPUs (like Nvidia H100) for AI training
- Developing more durable sodium-ion batteries
- Reengineering Google’s search engine to be more energy-efficient
🎯 Technical progress boosts margins, robustness, and scalability.
It might not make the headlines, but it’s just as essential.
So… Should you innovate or focus on technical progress?
- Saturated market? → Innovate to stand out.
- Mature product? → Make technical improvements to stay competitive.
- Industry in flux? → Combine both.
My take on LLMs?
Based on recent scientific insights (IEEE Spectrum, Nature Machine Intelligence, and the book Prediction Machines by Agrawal et al.), the explosion of models like GPT-4o and Claude 3 is driven by two forces:
Technological innovations
- The Transformer architecture (Vaswani et al., 2017) revolutionized language modeling
- Self-supervised learning opened up a whole new paradigm
Ongoing technical progress
- Distributed compute optimization (TPUs, GPUs)
- Data scaling laws and efficiency (Kaplan et al., 2020)
- Fine-tuning, RLHF, quantization…
Conclusion
LLMs are undeniably born from technological breakthroughs, but their current performance is the result of years of engineering progress.
As is often the case in tech:
- Innovation sparks the movement.
- Technical progress delivers the performance.
One last thing…
Whether you’re innovating or improving existing tech, if you document your trials, hypotheses, and technical barriers, you may be eligible to recover up to 64% of your development costs through SR&ED tax credits (and even more in Québec thanks to provincial incentives).
Not sure if your project qualifies? Talk to a Leyton expert
