The BiOS training engine: single-device model code, declared parallelism

A platform assembled from components can promise features. It cannot promise behaviour, because behaviour is decided in layers it does not control.

The commitments an enterprise buyer actually asks for are unglamorous. That a run which started will finish. That a configuration which cannot work is refused before it costs anything. That a release behaves on Friday the way it did on Monday. Each is a statement about the whole path from the model definition to the scheduler, and a wrapper can only forward whatever the layer beneath it decided to say.

There is a second reason, and it is the one that matters commercially. A model trained in one system and served by another has no path from a live answer back into its own next version. Building that path across a vendor boundary is the project that quietly kills most enterprise AI programmes. Owning this layer, and the serving layer beside it, is what makes a continuous-learning loop possible at all.

One decision organises everything else: model code is written for a single device, and how it is split across many is declared separately.

Each model family gets its own implementation rather than sharing an abstraction with the others. That is a deliberate trade of reuse for readability, and it buys a property worth more than the duplication costs: adding a new family cannot break an existing one. The parallelism strategy is then expressed as placement on module boundaries instead of being woven by hand through the forward pass.

The practical consequence is that growing a job from one machine to a cluster is a change of numbers in the job description rather than a rewrite. Splitting by sequence is the one worth dwelling on, because it changes what is possible rather than how fast it is: most systems meet a very long document by truncating it, and cutting the sequence itself across machines means a long contract, a full case file or an entire codebase can be trained on whole.

Three ways to train, three ways to adapt, two kinds of model. The combinations matter more than the counts.

The engine covers pre-training from scratch, continued pre-training on an organisation's own corpus, and supervised fine-tuning on work its people already did. Each runs as a full fine-tune that rewrites every weight, or as an adapter touching a fraction of one per cent of them, with the adapter exportable on its own or merged into a single model. Language models and vision language models go through the same path.

The choice between those is usually presented as a cost question and is really a question about how much you want to move. The wider literature on adapting a general model to one domain surveys the alternatives[6]. Our position is that retrieval and prompting move information into a model's context, which is cheap, useful and does not accumulate; training moves it into the weights, which is the only form that compounds.

If there is one thing in this engine we would defend hardest, it is not a performance decision. It is a refusal.

Faster numeric formats are used only where the hardware genuinely supports them. Where it does not, the run says so and stops, rather than silently falling back to something slower and billing as though it had not. An impossible configuration is refused before it consumes anything, with the reason written as a sentence a person can act on. A setting a customer asked for is never dropped on the floor: if it cannot be honoured, that is an error, not a default.

The same discipline governs releases. A version is tested across every model family we support before any customer sees it, and it ships whole rather than being patched in the field. Internally we keep a status document that separates what we have actually run on our own hardware from what is merely supported in code, and that distinction is maintained honestly enough to be uncomfortable to read. An engineering culture that will not write “verified” without having verified it is the cheapest reliability mechanism available to anyone.

Supervised training on curated work is the right first mechanism. It is also a ceiling.

Learning from examples a person approved can only teach a model to reproduce judgements somebody already made. Learning from outcomes rather than approvals, which is the direction preference-based and reinforcement methods point[5], removes that ceiling. Reinforcement learning and distillation are both built into the engine, with the rollout, environment and reward machinery around them, and both are being hardened for general release.

Distillation matters for a separate reason. It is how a specialised model gets small enough that running it stops being a budget conversation, which is the difference between a model an organisation is proud of and one it can afford to put in front of every request.

Any engine that trains repeatedly on a narrowing distribution has to answer for forgetting[4]. How we bound that, and why the promotion gate rather than the trainer is the right place to do it, is the subject of the Conscious Loop paper.