Model merging combines the weights of several fine-tuned models into one model, without training it again, to get the strengths of each.
In plain words
Fine-tuned copies of the same large pretrained model often end up in the same low-error region of weight space. Merging combines those weights directly, for example by averaging them, to produce one model. Unlike running several models side by side, the merged model costs no more to run than one. Smarter methods handle weights that pull in opposite directions.
A quick example
A team fine-tunes one base model three times with different settings. Averaging the three sets of weights gives one model that is often more accurate than the best single run, and it runs at the cost of one model.
Where you'll meet it
- Combining task-specific open models into one.
- Community open-weight models built with tools such as mergekit.
Related
Sources
- PaperModel soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time, Wortsman et al. (arXiv) · read 29 Sept 2026
- PaperTIES-Merging: Resolving Interference When Merging Models, Yadav et al. (arXiv) · read 29 Sept 2026
- Docsmergekit, Arcee AI (GitHub) · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 29 Sept 2026. Spotted an error? Report it.