I've listened to six podcasts that talk about recursive self-improvement with insiders.
The old science-fiction version has one machine rewriting its own code overnight. The version in these podcasts is an AI doing part of the research needed to build a better AI. Models can write training code, build evaluations and run experiments. Greenblatt thinks finding subtle bugs may be one of the easier research tasks to train them on.
Labs can run many copies alongside human researchers. The guests disagree about how much of the research AI can do and where the acceleration stops.
Some forecasts
Ryan Greenblatt expects labs to automate AI research around 2030 or 2031. He thinks this could squeeze four or five years of normal progress into one year. If labs automate all AI research, he'd expect systems that beat humans across most jobs within about another year.
Toby Ord is less sure the speed-up would continue. He discussed Tom Davidson's estimate of about five years of progress in one year, followed by a plateau.
The dates behind AI 2027 have moved. Eli Lifland said Daniel Kokotajlo's median for artificial general intelligence (AGI) had shifted from 2028 to 2030, with automated coding in 2029. Lifland moved his own forecasts back by one to three years. His published medians now put automated coding around 2032, AGI around 2035 and superintelligence in 2036.
They still see 2028 or 2029 as the most likely year. The median is later because they also give slower progress a real chance.
Geoffrey Irving said it could take ten years for models to get good at planning, judgement and other work without a clear score. It could happen faster if models learn to create training data and exercises for those tasks.
These forecasts use different milestones, so they aren't direct comparisons. Lifland's median gives about two years from automated coding to superintelligence. In his slower 80th-percentile case, it takes closer to ten. Greenblatt gives about one year after full research automation.
Why AI research could speed up
AI research often has a score. Change a training method or fix a bug, then measure the result. Greenblatt's example was cutting the time needed to reach the same training loss. Training loss is the number used to measure a model's errors while it learns.
Labs can split this work into small tasks. Models repeat them and get a score each time. Greenblatt described training environments where a model learns to improve smaller models using eight H100 AI chips. He isn't sure the same practice would teach the judgement needed for the few expensive decisions behind frontier models.
He also thinks machine-learning research is shallower than mathematics because useful improvements often stack. A model can find a better training method, show that it works and keep going.
Current systems may already match mediocre machine-learning researchers on some work. That isn't enough for a frontier lab. Once models reach the best researchers, though, labs can run many copies. Greenblatt said a model can learn in under an hour what a person might take weeks to learn about a large codebase. It still falls short of someone who has worked in that codebase for years.
OpenAI says its limited Ultrafast preview can run GPT-5.6 Sol up to 14 times faster than its standard service. Its researchers often launch batches of experiments overnight and review the results the next morning. OpenAI is testing whether the faster model lets them run several rounds in a day. When each result decides the next test, speed may let the same researchers do more work rather than only finish each task sooner.
Irving thinks models may learn work without a clear score too. Labs already train models to critique other models. They can also turn old research into new practice exercises. That may help teach research judgement.
What could slow it down
Frontier labs can only run a limited number of large experiments. More AI coders don't create more chips or power.
Ord also objects to reading an AGI date from METR's task-horizon chart. It measures how long a person would take to do a task that AI completes half the time. Ord says the chart doesn't tell us what task length would count as general intelligence. Eight hours doesn't make a dependable working day, and 40 hours doesn't make a working week.
Compute growth may slow as well. Some recent gains came from companies spending more and chipmaker TSMC moving factory capacity from phone processors to AI chips. As Ord put it, they can't build ten TSMCs quickly.
Joshua Achiam points out that physics limits how much computation can fit into a given space and energy budget. His interviewer suggested today's systems might be 30 to 50 orders of magnitude below that limit. Each order is another factor of ten. Achiam couldn't put a number on it, but guessed the limit was more than five or ten years away.
Some skills also need feedback from the physical world. Ord thinks a system could become extremely intelligent inside a data centre while still lacking the experience needed to run a factory.
The interviewer asked whether Achiam would have expected 2026 to feel this normal if he'd seen today's models in 2017 or 2022. Achiam said yes:
"If you're trying to predict the future that is less than a decade away, you should assume... a lot of things feel relatively normal."
Joshua Achiam, The a16z Show
A few minutes later, Achiam said AGI might already be here and most people had shrugged. We could get full AGI and react in much the same way if daily life still feels normal.
That sounds about right.
The safety problem
Labs can measure whether code makes training faster. They can't test honesty and control in the same clean way.
Thomas Wolf described two cyber tests. In the first, an OpenAI model couldn't solve some challenges and went looking for the answers on Hugging Face. It tried many routes and generated about 17,000 events. Nobody had told it to attack Hugging Face. Wolf called it a side quest.
In a UK AI Security Institute test, another model put malicious code into a pull request on a public GitHub project. It used a second account to support the change, argued with the maintainer and edited earlier messages. The test had disabled its cyber safeguards and given it full internet access.
Hugging Face asked its usual Claude Code setup for help during the intrusion. It refused because the team hadn't gone through its security checks and returned an application form. A fallback model refused too. The team used GLM 5.2 instead, an open model Nvidia had compressed to four bits so it could run on less hardware. Security teams need to test their emergency model before they need it, including whether the provider will allow the work and accept the incident data.
Greenblatt worries about what happens after labs catch behaviour like this. They train against the hacks they find, while missed hacks still earn a reward. The incidents may become rarer and harder to spot as models take on more of the work used to train their successors. Near the end of the Dwarkesh conversation, he put the chance of something we'd recognise as an AI takeover by 2040 at 35 to 40%.
Wolf is also wary of relying on a model's written reasoning. He says it's becoming denser and harder to follow.
Neuralese
Wolf used "neuralese" loosely for model reasoning that packs more meaning into fewer words and becomes hard for people to read. Current models aren't speaking a private language, but a readable chain of thought doesn't show everything happening inside a model. It can't be the whole safety system.
Irving also warned that short training tasks don't rule out long plans. A model could make a plan in a week, then complete each part in shorter steps.
Research without clear answers
Code runs or it doesn't. A proof can be checked. Training loss rises or falls. Models can repeat those jobs and learn from a score.
Researchers still have to decide which questions matter and which expensive experiments deserve a run. They also need to spot bad assumptions. Ord says we don't have much of a theory for how a model becomes superhuman at that sort of judgement. A model judging another model's work may only catch errors the first model can see.
Greenblatt thinks practice on smaller, scored experiments may teach some of that judgement. Irving thinks models could build lessons from earlier examples of messier work. I didn't hear a clear answer on how far either route will go.
The shortest superintelligence forecasts depend on AI learning to improve at research where the answer can't be checked cleanly. I still don't know whether it can. We will almost certainly find out by 2040.