01Capability that does not accumulate
The measurable failure of the first enterprise AI wave is now well documented. One widely cited study of enterprise deployments found that roughly nineteen in twenty pilots produced no measurable effect on profit, and attributed it not to model quality but to the systems being unable to retain feedback or adapt to context[1]. Gartner separately expects more than forty per cent of agentic projects to be cancelled before the end of 2027[3].
The instinctive reading is that the models were not good enough. The evidence points elsewhere. Over the same period the blended price of a million tokens fell by about two thirds while average enterprise AI budgets rose roughly sixfold[2]. Unit capability got cheaper and more plentiful; outcomes did not follow. Something other than raw capability is the binding constraint.
A system that cannot remember yesterday is not intelligence. It is a very expensive lookup.
We think the constraint is structural. A shared, hosted model is contractually and commercially forbidden from specialising to one customer: every tenant gets the same weights, so nothing a tenant teaches it can stay. The feedback an organisation generates in the course of ordinary work, which is the most valuable training signal it will ever have access to, is produced continuously and then thrown away continuously.
02The loop
The mechanism is deliberately unexotic. Each stage is a thing production systems already do; what is unusual is closing them into a cycle that runs without a person starting it, and doing so where the data already is.
- 1Serve
The model answers real work through the serving layer. This is the only stage that touches a user.
- 2Record
The request, the answer, the outcome and any human correction are written to the customer’s own store.
- 3Curate
Traces become training material: accepted outcomes as examples, corrections as the lesson that replaces them.
- 4Train
A candidate is trained from the incumbent, using a configuration approved once rather than tuned per run.
- 5Evaluate
The candidate is scored against the incumbent on held-out work and on tests the customer wrote.
- 6Promote
The candidate serves only if it wins. If it does not, it is retained for inspection and nothing changes.
One turn is one training job and one promotion decision. The cadence is a customer setting rather than a property of the system: hourly where volume supports it, monthly where it does not, or manual indefinitely. The configuration is approved once and reused, which is what makes the loop autonomous without making it unsupervised.
The stage that does the real work is curation. Traces are not training data. An interaction becomes a training example when a human signal attaches to it: an analyst accepted the answer, a partner rewrote the clause, a dispatcher overrode the schedule. The rewrite is worth more than the acceptance, because it carries the correction as well as the judgement.
03Why location is not an implementation detail
The loop only closes where the traces already live. This is usually presented as a compliance concession. It is closer to the opposite: the constraint and the capability point the same way.
The material that makes the loop valuable is precisely the material an organisation is least able to export: customer conversations, case files, negotiated positions, dispatch history, reconciliation notes. In many jurisdictions and most regulated industries, moving it is not a procurement preference but a legal question. A continuous-learning product that requires the data to leave is therefore unavailable to the buyers who would benefit most from it.
Running the whole loop inside the customer boundary resolves this by removing the transfer rather than by governing it. Training, serving, the trace store and the checkpoints sit in one environment under one set of network and identity controls. The only thing crossing the boundary is signed software travelling inwards.
The strongest training signal an organisation owns is the one it is least permitted to send anywhere.
04What better has to mean
An autonomous training loop is only as trustworthy as its stopping condition. Ours is comparative rather than absolute: a candidate does not need to be good, it needs to beat the version currently serving.
Keeps serving unless it is beaten. The default outcome of every cycle is no change.
Promotes only on a win against the incumbent, scored on held-out work plus the customer's own tests, with a general evaluation as a guard.
Two properties follow that matter more than they first appear. The first is that the expected outcome of any given cycle is no change, which makes the loop safe to leave running. The second is that every version is retained with the scores that justified it, so the question a board asks is answerable: not whether this model is good, but whether the trend across the last several promotions is still upward.
05Failure modes we design against
Repeated training on a narrowing distribution is a well-studied way to make a model worse. These are the specific ways this class of system fails, and what the design does about each. We would rather state them than be asked.
Training repeatedly on a narrowing slice degrades everything outside that slice, and the degradation is invisible if you only measure the slice. We hold a fixed general evaluation alongside the customer’s own tests, and a candidate that wins locally while losing generally does not promote.
A loop that trains on unreviewed model output converges on the model’s own habits rather than on the work. Curation therefore keys on human signal: what a person accepted, corrected or rejected. An interaction nobody judged is evidence of traffic, not of quality.
Any sufficiently repeated objective gets gamed, including by a well-behaved model. Evaluation sets are owned by the customer, held apart from the training material, and rotated. A test that has become training data is not a test.
Slow degradation is harder to catch than a sharp break, because no single version looks wrong. Every promotion is recorded with its scores, so the question is not "is this version good" but "is the trend across the last ten still up".
If a bad practice is common in the traces, the loop will learn it faithfully. This is the strongest argument for the promotion gate staying human for as long as the customer wants it, and for version history that makes reverting cheap.
None of these are solved in the sense of being permanently closed. They are bounded: by the promotion gate, by held-apart evaluation, by human-keyed curation, and by version history that makes reverting a one-step operation rather than a recovery project.
06Where this goes next
The loop as described learns from what people approved. The more interesting version learns from what actually happened.
Supervised learning from curated traces is the right first mechanism because the signal is unambiguous and the failure modes are understood. It is also a ceiling: it can only teach the model to reproduce judgements a person already made. Learning from outcomes rather than from approvals, which is the direction preference-based and reinforcement methods point[6][7], removes that ceiling, because the loop starts to optimise the thing the business actually cares about rather than a proxy for it.
Both reinforcement learning and distillation are built into our training engine, with the rollout, environment and reward machinery around them, and both are being hardened for general release. Distillation matters for a different reason: it is how a specialised model gets small enough that running it stops being a budget conversation.
The wider literature on adapting a general model to one domain surveys the alternatives we weighed[8]. Our position is that retrieval and prompting move information into a model's context, which is useful and cheap and does not accumulate; training moves it into the weights, which is the only form that compounds.
RReferences
- [1]MIT NANDA, The GenAI Divide: State of AI in Business, 2025 Reports that roughly 95% of enterprise generative AI pilots produced no measurable profit effect, attributing it to systems that do not retain feedback or adapt.
- [2]AI.cc, 2026 AI API Infrastructure Report Drawn from more than 2.4bn enterprise API calls: blended price per million tokens fell from $18.40 to $6.07 in a year while average enterprise AI budgets rose from $1.2m to $7m.
- [3]Gartner, agentic AI project cancellation prediction, June 2025 Predicts that more than 40% of agentic AI projects will be cancelled before the end of 2027.
- [4]Kirkpatrick et al., Overcoming catastrophic forgetting in neural networks, 2017 The canonical statement of the problem that any repeated-training system has to answer.
- [5]Luo et al., An Empirical Study of Catastrophic Forgetting in Large Language Models during Continual Fine-tuning, 2023 Measures forgetting as a function of model scale and tuning regime.
- [6]Stiennon et al., Learning to summarize from human feedback, 2020 Establishes the pattern of learning a preference signal from human judgement rather than from labels alone.
- [7]Ouyang et al., Training language models to follow instructions with human feedback, 2022 The instruction-tuning and preference-optimisation pipeline that most production post-training now descends from.
- [8]Song et al., Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey, 2025 Survey of the approaches available for making a general model good at one domain.
- [9]Gartner, worldwide AI spending forecast, May 2026 Worldwide AI spending forecast at $2.59tn for 2026, a 47% year-on-year increase.