How a family of models is built

Building a family of models does not have to begin with training an artificial intelligence from scratch. It can begin with existing models, a clearly defined function and a question: what does each member of this family need to do well?

Understanding this work starts with distinguishing concepts that often appear together: base model, merging, fine-tuning, adapter and reinforcement learning. Each has a role. None of them, on its own, defines a family.

Before the tools, the trunk and the branches

A base model is an already trained starting point. It carries capabilities and limitations from its training. It is not a blank page, and having broad reach does not mean knowing everything or being equally suited to every task.

A domain-focused model is prepared for a particular scope: a field of knowledge, a type of document or a need. Marketing and technical documentation are examples of possible domains; maintaining a style or extracting fields are examples of functions within them.

The image of a tree helps: a shared trunk can give rise to branches with different functions. It is one way to organize a family, not a rule requiring every project to have exactly the same origin. The family is the planned collection; an adaptation technique alone does not define it.

Each model's role

A family begins with defined functions. What task does each model receive? How should it deliver its answer? What errors make the result unusable? What falls outside its function?

Without those answers, it is easy to end up with several model files and no clear difference between what each offers. A specialized family needs roles that make sense to the people who will use it.

Choosing the base

The choice of a base model influences each member's capabilities, limitations and adaptation possibilities. That choice is part of designing the family.

It is also necessary to check what each license allows you to do with the models and their derivatives. The license for a tool used to work on the weights does not replace the models' own licenses.

A domain base can support several specializations

Between a general-purpose base and a model prepared for a specific task, there can be a domain base. It concentrates knowledge and patterns from a field while remaining a starting point for different applications within that scope.

In that sense, it is more neutral with respect to the final application: it is not yet committed to a single function, response format or service style. This neutrality is relative to the use that comes later; it does not mean an absence of biases or limitations.

The same domain base can support one model for documentation, another for information extraction and another for user interaction. They share a foundation, but each needs its own preparation and evaluation for the function it will take on.

Choosing an existing base and developing a domain base are distinct decisions. The latter can build on an already trained model and continue its preparation for that field. Creating this shared foundation is part of designing a family; it does not, by definition, require starting with random weights.

Combining can be one stage

MergeKit is a tool for combining the weights of trained models. That is different from gathering answers from several models during a conversation: the operation produces a new artifact from the selected weights.

There are different methods for this combination. They make choices about mixing the models' contributions and handling differences between them. The choice depends on the models involved and the outcome being investigated; compatibility needs to be checked.

Combining a model with good writing skills and another with good reasoning does not guarantee a result that brings together the best of both. A merge creates a candidate. Evaluation determines whether it preserved capabilities, gained something useful or got worse at what mattered.

The project offers CPU execution paths and GPU acceleration. That does not turn a specific amount of memory into a guarantee for every combination: requirements depend on the models and configuration.

The training objective and the way parameters are adjusted are different things

A simple way to organize the terms is to ask two questions: what do we want the model to learn, and which parameters will we adjust to achieve that?

During pretraining, a model learns patterns from large datasets. This learning can also continue from an existing model, using material chosen for a domain. In supervised fine-tuning, or SFT, examples guide the desired behavior: the model receives situations and answers used as training references.

Another objective is learning preferences between possible answers. This can involve human evaluations, automatic signals or combinations, depending on the method. Not all preference training is reinforcement learning: some approaches use those comparisons directly.

Where reinforcement learning comes in

In reinforcement learning, training uses a reward signal to guide the model's behavior. The reward represents a criterion chosen for the task; it is not automatic proof that the answer is correct.

RLHF stands for reinforcement learning from human feedback. In one common approach, people evaluate answers and those evaluations help build a reward model, which guides the adjustment of the model that responds. It is one way to turn preferences into a training signal.

This does not mean someone must manually approve every answer once the system is in use. Nor does it guarantee perfect alignment: the signal can poorly represent the intended goal and favor behaviors that look good in evaluation but fail in real situations.

Fine-tuning can use adapters — and adapter is not a synonym for LoRA

Fine-tuning means adjusting an already trained model. It can update all the base model's parameters or use strategies that adjust only a subset or additional parameters. That choice differs from the training objective: following examples or learning preferences, for instance.

An adapter is an adaptation mechanism associated with the model. Depending on the method, it can add trainable components or represent adjustments that work alongside the base while keeping its original weights frozen. The exact form depends on the method and implementation.

LoRA is a type of adapter in this broad sense: it represents updates through smaller trainable matrices. The term adapter does not exclusively mean LoRA. There are other forms of adaptation, and the name used in a project needs to identify which one is involved.

An adapter is not, on its own, the complete model either. It depends on a compatible base. That is why a collection of adapters should not automatically be confused with a family of complete models or with any single recipe for specialization.

Adapting to the scope is another possible stage

When the objective calls for adaptation with specific data, tools such as Axolotl help organize training. The configuration describes elements such as the model, data and adjustment strategy, making it possible to record the process and track its results.

This does not make adjustment mandatory after every merge. Nor does it make fine-tuning and a family of models the same thing. One is a way to adapt; the other is the organization of models with defined purposes. Each member's development path depends on the project.

Choosing suitable data remains a central part of the work. A training tool organizes execution; it does not decide on its own what counts as a good answer for the domain.

Evaluate before presenting it as a specialist

Each candidate needs to be compared with a reference: the starting model, another suitable alternative or both. Evaluation examples should reveal behavior in situations that were not simply repeated during preparation.

What is measured depends on the function. It might be the correctness of extracted fields, adherence to a format, fidelity to a reference or the amount of revision required. When there is variability, a single run is not enough to represent performance.

It is also important to record losses. A model can improve in one area and get worse in another. Specialization does not imply an automatic reduction in size, elimination of errors or guaranteed superiority.

From a collection to a family

The intended result is a collection with clear functions, identifiable versions and criteria for deciding when each model is ready for use. If those models are offered through an API, integrators need to understand each one's function and the limits of what it delivers.

That is the direction of Kuaapy's family of specialized models. In our proposal, phase 3 fine-tuning and phase 4 family development have distinct scopes. Adapting a model to a need and developing a family with its own functions are different deliverables.

To learn about this direction and discuss a specific need, visit the Kuaapy Services area.

Further reading

The starting point is reusing existing knowledge. What turns that reuse into a family with a purpose is defining functions, building candidates and checking what each actually delivers.