Tech Companies Accelerate Work on Self-Improving Artificial Intelligence
New York, Sunday, 20 September 2026.
Major technology firms are advancing artificial intelligence systems that build their own next generations, accelerating development speeds while intensifying debate among experts over safety controls and human oversight.
Industry Leaders Report Accelerated AI Development
Major technology firms are advancing artificial intelligence systems capable of building their own next generations, accelerating development speeds while intensifying debate among experts over safety controls and human oversight [1][2]. In September 2026, executives from IBM, Anthropic, and OpenAI acknowledged that while fully autonomous self-upgrading AI remains scientifically unproven, the technology is approaching a critical threshold where systems actively code and optimize their successors [1]. Anthropic reported this week that its Claude model now leads 26% of the company’s model research and development, performing tasks end-to-end under human supervision [2]. This shift implies that human-led efforts account for the remaining portion of R&D, calculated as 74 percent of the workload [2]. Industry leaders are currently evaluating safety guardrails to prevent AI models from bypassing human oversight as the pace of innovation quickens [1].
Technical Evidence and Capabilities
Concrete evidence of recursive self-improvement emerged in July 2026, when Weco AI reported findings where its AIDE² system modified software framework components controlling another research agent [1]. This resulted in a net positive system improvement rather than a self-sustaining intelligence explosion, with the research agent demonstrating cross-task generalization on tasks it had never encountered during its improvement run [1]. Concurrently, Google and DeepMind published research on Dream-RSI, a system where the AI builds a history of attempts and turns that history into a simulator to learn better exploration strategies [3]. The results showed up to 162x fewer discovery-agent calls on one experiment and comparable performance on GPU kernel tasks with roughly 1.8 to 2.4x fewer generations [3]. The range of efficiency improvement in generations spans a difference of 0.6 times fewer generations depending on the specific task budget [3].
Safety Concerns and Human Oversight
Despite technical strides, researchers maintain that current AI systems remain incapable of fully autonomous recursive self-improvement, citing significant gaps such as the necessity of human-in-the-loop oversight [1]. Nathalie Baracaldo, Manager of AI Security and Privacy Solutions at IBM, warned that a self-improving agent may modify its tools or environment in ways that optimize for a misspecified reward with each iteration, amplifying problems rather than correcting them [1]. Experts emphasize that significant economic and physical barriers, such as the substantial computing infrastructure required to run large models, currently prevent the realization of fears regarding uncontrolled autonomous agents [1]. Anthony Aguirre, president and CEO of the Future of Life Institute, noted that as AI does more of the research, it gets faster because AI operates much more quickly than humans do [2].
Future Timelines and Economic Barriers
Looking ahead, OpenAI announced in September 2026 the development of an automated research intern capable of performing well-defined research tasks, with a goal to create a fully automated AI researcher by March 2028 [2]. Elon Musk stated in March 2026 that xAI’s Grok models are increasingly removing humans from the loop for model improvement, with a target of reaching full automation by the end of 2026, and no later than 2027 [2]. However, Michael Littman, University Professor of Computer Science at Brown University, expressed skepticism, stating that the idea that the resulting system will be even better at designing new AI systems is not clear to be logically coherent, let alone imminent [1]. Anthropic has publicly stated it is willing to slow or pause development work on recursive self-improvement, provided that global competitors agree to do so in a verifiable manner [2].