Subvolts take Understand LLM evolution with expert insights on reasoning, scaling, and future directions. Jan 29, 2026 - Developers
- Learn about new reasoning techniques like RLVR and GRPO that unlock latent capabilities in base models.
- Business
- Understand where AI moats lie now, why in-house training might increase, and how inference scaling drives progress.
This is a solid interview with a respected researcher providing a deep dive into current LLM trends and future outlook.
State of LLMs 2026: RLVR, GRPO, Inference Scaling — Sebastian Raschka
Audio streams from Matt Turck. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
Sebastian Raschka joins The MAD Podcast to review LLM advancements in 2025 and look ahead to 2026. The discussion covers transformer architectures, world models, and reasoning techniques like RLVR and GRPO. Raschka also addresses evaluation methods, inference scaling, tool use, data moats, in-house model training, and the challenges of continual learning.
This episode explains how AI language models are getting smarter, focusing on new techniques that help them reason better and more cheaply. It's important because these improvements drive the AI tools we use daily and shape future AI development.
This episode offers a masterclass on the current state and future trajectory of Large Language Models. It's for AI practitioners, researchers, and business leaders tracking LLM progress. Raschka's insights on reasoning, scaling, and evaluation directly inform how frontier labs like Google Gemini, OpenAI, and Anthropic are pushing model capabilities and efficiency. Understanding these shifts is crucial for staying competitive in the rapidly evolving AI landscape.
- HostMatt Turck
- GuestSebastian Raschka
- Guest: Sebastian Raschka
- Title: State of LLMs 2026: RLVR, GRPO, Inference Scaling
- Show: The MAD Podcast
- Length: 68 minutes
- Key Topics: RLVR, GRPO, inference scaling, transformer architectures, continual learning
Sebastian Raschka joins the MAD Podcast for a deep, educational tour of what actually changed in LLMs in 2025 — and what matters heading into 2026.
We start with the big architecture question: are transformers still the winning design, and what should we make of world models, small “recursive” reasoning models and text diffusion approaches? Then we get into the real story of the last 12 months: post-training and reasoning. Sebastian breaks down RLVR (reinforcement learning with verifiable rewards) and GRPO, why they pair so well, what makes them cheaper to scale than classic RLHF, and how they “unlock” reasoning already latent in base models.
We also cover why “benchmaxxing” is warping evaluation, why Sebastian increasingly trusts real usage over benchmark scores, and why inference-time scaling and tool use may be the underappreciated drivers of progress. Finally, we zoom out: where moats live now (hint: private data), why more large companies may train models in-house, and why continual learning is still so hard.
If you want the 2025–2026 LLM landscape explained like a masterclass — this is it.
Sources:
The State Of LLMs 2025: Progress, Problems, and Predictions - https://x.com/rasbt/status/2006015301717028989?s=20
The Big LLM Architecture Comparison - https://magazine.sebastianraschka.com/p/the-big-llm-architecture-comparison
Sebastian Raschka
Website - https://sebastianraschka.com
Blog - https://magazine.sebastianraschka.com
LinkedIn - https://www.linkedin.com/in/sebastianraschka/
X/Twitter - https://x.com/rasbt
FIRSTMARK
Website - https://firstmark.com
X/Twitter - https://twitter.com/FirstMarkCap
Matt Turck (Managing Director)
Blog - https://mattturck.com
LinkedIn - https://www.linkedin.com/in/turck/
X/Twitter - https://twitter.com/mattturck
(00:00) - Intro
(01:05) - Are the days of Transformers numbered?
(14:05) - World models: what they are and why people care
(06:01) - Small “recursive” reasoning models (ARC, iterative refinement)
(09:45) - What is a diffusion model (for text)?
(13:24) - Are we seeing real architecture breakthroughs — or just polishing?
(14:04) - MoE + “efficiency tweaks” that actually move the needle
(17:26) - “Pre-training isn’t dead… it’s just boring”
(18:03) - 2025’s headline shift: RLVR + GRPO (post-training for reasoning)
(20:58) - Why RLHF is expensive (reward model + value model)
(21:43) - Why GRPO makes RLVR cheaper and more scalable
(24:54) - Process Reward Models (PRMs): why grading the steps is hard
(28:20) - Can RLVR expand beyond math & coding?
(30:27) - Why RL feels “finicky” at scale
(32:34) - The practical “tips & tricks” that make GRPO more stable
(35:29) - The meta-lesson of 2025: progress = lots of small improvements
(38:41) - “Benchmaxxing”: why benchmarks are getting less trustworthy
(43:10) - The other big lever: inference-time scaling
(47:36) - Tool use: reducing hallucinations by calling external tools
(49:57) - The “private data edge” + in-house model training
(55:14) - Continual learning: why it’s hard (and why it’s not 2026)
(59:28) - How Sebastian works: reading, coding, learning “from scratch”
(01:04:55) - LLM burnout + how he uses models (without replacing himself)
- RLVR
- Sebastian Raschka
- Reinforcement learning with verifiable rewards, a new way to train AI.
- GRPO
- Sebastian Raschka
- A new reinforcement learning method for AI, paired with RLVR.
- RLHF
- Sebastian Raschka
- Reinforcement learning from human feedback, an older method for training AI.
- benchmaxxing
- Sebastian Raschka
- Focusing too much on improving AI performance on specific tests, potentially ignoring real-world use.
Same guests, other shows
1 h 19
#490 – State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI
WithNathan Lambert, Sebastian Raschka
Questions people ask
- What are RLVR and GRPO?
- RLVR (reinforcement learning with verifiable rewards) and GRPO are new techniques that make scaling reasoning in LLMs cheaper than older methods like RLHF.
- Why is real usage more important than benchmarks?
- Sebastian Raschka suggests that 'benchmaxxing' warps evaluation, and he trusts real-world usage data more than benchmark scores to gauge progress.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Matt Turck.