Model merging: combining fine-tunes without another training run

Two teams fine-tune the same base model for two different tasks. The standard assumption is that combining their work into one model requires a new training run over both datasets. A line of weight-space research shows that assumption is often wrong — and shows exactly where it becomes wrong.

A fine-tune is not a new model. It is a direction away from a shared base — and directions can be added.

Task vectors turn fine-tuning into arithmetic

Editing Models with Task Arithmetic ↗, by Ilharco and colleagues, defines a task vector as the weight-space difference between a fine-tuned model and the pretrained base it started from: τ = θ_finetuned − θ_base. The paper's central finding is that these vectors behave like directions you can operate on directly, with no further gradient descent:

• Negation (θ_base − τ) suppresses the fine-tuned behavior, with the paper reporting little change to performance on unrelated control tasks.
• Addition (θ_base + τ_A + τ_B) can improve performance on both task A and task B in a single merged model.
• Analogy — when tasks relate as "A is to B as C is to D," combining vectors from three of the four tasks can improve performance on the fourth, without ever training on data from that fourth task.

Architecture / weight-space editing

Fine-tunes become vectors you can add

FIG 01 - MOTION

This reframes fine-tuning as a library of reusable, composable edits rather than a sequence of one-off, disposable training runs. A task vector is small relative to a full model checkpoint, versionable, and — critically — the arithmetic is nearly free compared with the training run that produced it.

Why naive addition breaks down at scale

Adding two task vectors works well. Adding many does not, and the failure has a name: interference. TIES-Merging ↗, by Yadav, Tam, Choshen, Raffel, and Bansal, traces merge degradation to two specific, measurable sources:

1. Redundant parameter values. Many weight changes during fine-tuning are small and contribute little signal; averaging them in with larger, meaningful changes from other task vectors dilutes the useful signal. 2. Sign disagreement. Different fine-tunes can push the same parameter in opposite directions. A naive average of +0.4 and −0.4 produces 0.0 — cancelling both tasks' intent at that parameter rather than preserving either.

Process / interference resolution

TRIM, ELECT SIGN and MERGE

FIG 02 - MOTION

TIES-Merging's fix is three deterministic steps applied before averaging: trim each task vector down to its largest-magnitude changes, elect a single sign per parameter by majority vote across all task vectors being merged, then merge only the components that agree with the elected sign. The paper reports this outperforming naive averaging and several prior merging methods across a range of modalities, task counts, model sizes, and architectures — precisely because it removes the two interference sources rather than averaging through them.

What merging actually saves — and what it doesn't

Model merging is attractive because it replaces a training run with an arithmetic operation, but the savings are narrower than "no training at all" makes it sound:

• It saves the compute and wall-clock time of a joint or sequential training run across multiple tasks.
• It does not remove the need for evaluation. A merged model is a new model and must be tested as one — the arithmetic guarantees a specific procedure, not a specific quality outcome.
• It does not guarantee capacity. If two tasks require genuinely conflicting behavior from the same parameters, no sign-election scheme manufactures capacity that was never there; TIES-Merging resolves disagreement about direction, not fundamental capability conflicts.

What this changes in production

Consolidating specialist deployments

Teams that accumulated a fine-tune per support queue, per product line, or per locale can test whether several of those adapters merge into one deployable model without losing per-task quality — collapsing serving fleet complexity from N models to one, provided each merge is evaluated against every original task's held-out set, not just the newest one.

Rapid capability iteration

Before committing to a full retraining cycle, a team can test a hypothesis — "does adding this task vector shift behavior the way we expect" — in minutes rather than a training run's worth of GPU-hours. Negation offers a cheap way to test whether an unwanted behavior lives in a specific fine-tune's task vector before deciding how to retrain around it.

Cross-team model reuse

Task vectors are portable artifacts: a team that fine-tuned a domain adapter can share the vector itself, not just the resulting weights, letting other teams compose it with their own base or their own task vectors — provided everyone starts from the exact same pretrained base, since the arithmetic only holds relative to a shared origin.

A production checklist

Before shipping a merged model, the team should be able to answer:

• Were all source fine-tunes derived from the identical base checkpoint — same weights, same tokenizer, same revision?
• Was the merge evaluated against every original task's held-out set, not only the task that motivated the merge?
• If naive weighted averaging was used instead of an interference-aware method, was a regression check run for each contributing task?
• Does the merged model's behavior on any task fall outside the acceptable range established by that task's own fine-tuned model?
• Is there a rollback path — the original per-task models — if the merged model regresses after release?
• Who owns the decision if two source fine-tunes turn out to want genuinely conflicting behavior at the same parameters?

Research referenced

• Ilharco et al. (2022), Editing Models with Task Arithmetic ↗.
• Yadav, Tam, Choshen, Raffel, and Bansal (2023), TIES-Merging: Resolving Interference When Merging Models ↗.