I disagree that evaluation is always a coding task. Evaluation is scrutiny for the person who wants the thing. It’s subjective. So, unless you’re evaluating something purely objective, such as an algorithm, I don’t see how a self contained, self “improving “ agent accomplishes the subjectivity constraint - as by design you are leaving out the subject.

Sure. There will always be subjective tasks where the person who asks for something needs to give feedback. But even there we could come up with ways to make it easier / faster / better ux. (one example I saw my frontend colleagues do is use a fast model to create 9 versions of a component, in a grid. And they "at a glance" decide which one is "better", and use that going forwards).

OTOH, there's loads you can do for evaluation before a human even sees the artifact. Things like does the site load, does it behave the same, did anything major change on the happy path, etc etc. There's a recent-ish paper where instead of classic "LLM as a judge" they used LLMs to come up with rubrics, and other instances check original prompt + rubrics on a binary scale. Saw improvements in a lot of evaluations.

Then there's "evaluate by having an agent do it" for any documentation tracking. Say you have a project, you implement a feature, and document the changes. Then you can have an agent take that documentation and "try it out". Should give you much faster feedback loops.

Agree. It's code all the way down. The key is to give agents a substrate where they can code up new capabilities and then compose them meaningfully and safely.

Larger composition, though, starts to run into typical software design problems, like dependency graphs, shared state, how to upgrade, etc.

I've been working on this front for over two years now too: https://github.com/smartcomputer-ai/agent-os/

The whole theme of llm dev to date has been "theres more common than not" in llm applications

because submarine piloting is a going-under-water activity, improvements in holding one's breath can lead to faster submersibles.

I don't think generation/discrimination is fundamental. A more general framing is evolutionary epistemology (Donald T. Campbell, 1974, essay found in "The Philosophy of Karl Popper"), which holds that knowledge emerges through variation and selective retention. As Karl Popper put it, "We choose the theory which best holds its own in competition with other theories; the one which, by natural selection, proves itself the fittest to survive."

On this view, learning in general operates via selection under uncertainty. This is less visible in individual cognition, where we tend to over-attribute agency, but it is explicit in science: hypotheses are proposed, subjected to tests, and selectively retained, precisely because the future cannot be deduced from the present.

In that sense, generation/discrimination is a particular implementation of this broader principle (a way of instantiating variation and selection) not the primitive itself.

Hermes agent does this, if you're curious

https://github.com/NousResearch/hermes-agent

It's a feedback loop.

I've always felt that the most important part of engineering was feedback loops.

Maybe nature is the greatest engineer ever?

There's a paper at https://arxiv.org/abs/2603.19461

Abstract:

Self-improving AI systems aim to reduce reliance on human engineering by learning to improve their own learning and problem-solving processes. Existing approaches to self-improvement rely on fixed, handcrafted meta-level mechanisms, fundamentally limiting how fast such systems can improve. The Darwin Gödel Machine (DGM) demonstrates open-ended self-improvement in coding by repeatedly generating and evaluating self-modified variants. Because both evaluation and self-modification are coding tasks, gains in coding ability can translate into gains in self-improvement ability. However, this alignment does not generally hold beyond coding domains. We introduce \textbf{hyperagents}, self-referential agents that integrate a task agent (which solves the target task) and a meta agent (which modifies itself and the task agent) into a single editable program. Crucially, the meta-level modification procedure is itself editable, enabling metacognitive self-modification, improving not only the task-solving behavior, but also the mechanism that generates future improvements. We instantiate this framework by extending DGM to create DGM-Hyperagents (DGM-H), eliminating the assumption of domain-specific alignment between task performance and self-modification skill to potentially support self-accelerating progress on any computable task. Across diverse domains, the DGM-H improves performance over time and outperforms baselines without self-improvement or open-ended exploration, as well as prior self-improving systems. Furthermore, the DGM-H improves the process by which it generates new agents (e.g., persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs. DGM-Hyperagents offer a glimpse of open-ended AI systems that do not merely search for better solutions, but continually improve their search for how to improve.

What are the differences wrt Recursive Language Models

In science there are ways to surface subjectivity (cannot be counted) into observable quantized phenomena. Take opinion polls for instance: "approval" of a political figure can mean many things and is subjective, but experts in the field make "approval" into a number through scientific methods. These methods are just an approximation and have many IFs, they're not perfect (and for presidential campaign analysis in particular they've been failing for reasons I won't clarify here), but they're useful nonetheless.

Another thing that get quantized is video preferences to maximize engagement.

This 'self vs non-self' logic is very similar to how plants prevent self-pollination. They have a biological 'discrimination' system to recognize and reject their own genetic code.

Here is a breakdown - https://vectree.io/c/plant-self-incompatibility-logic

The most important part of engineering is problem-solving, which feedback loops don't necessarily do. The reason we are here as engineers is: 2.5 billion years ago, the earth made cyanobacteria, which flourished, then flooded the earth with toxic oxygen, killing almost all life on the planet. The initial feedback loop didn't solve a problem, it destroyed a use case. That's not a solution to a problem that an engineer would choose, even if those organisms that came after were pretty happy about it...

Completely unrelated. Recursive Language Models are just "what if we replaced putting all the long text into the context window with a REPL which lets you read parts of the context through tool calls and launch partitioned subagents", ie divide-and-conquer applied to attention space.