mergekit is a Python toolkit for merging pretrained language models using out-of-core techniques, supporting CPU or GPU workflows and multiple merge methods. Latest releases include v0.1.4 (2025-10-31) and a legacy MIT-licensed set from 2023-10-06.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
mergekit is a toolkit for merging pre-trained language models. It uses an out-of-core approach to perform merges in resource-constrained environments and can run on CPU or with as little as 8 GB of VRAM. It supports many merging algorithms and targets models such as Llama, Mistral, GPT-NeoX, and StableLM.
How it works
The project provides multiple merge methods for combining models, with configuration through YAML-based merge configurations. It includes a tokenizer configuration pathway (modern tokenizer field and legacy tokenizer_source) and a chat_template option. Merging produces a single output model, with per-model embeddings adjusted to the output vocabulary when a tokenizer is configured. Embeddings may be filled for missing tokens using rules that involve base/model embeddings and averaging when necessary. The toolkit also offers features like lazy loading of tensors and interpolated gradients.
Getting started
Installation guidance is provided via a git clone and editable install:
git clone https://github.com/arcee-ai/mergekit.git
cd mergekit
pip install -e . # install the package and make scripts available
If editable install fails due to setup tooling, upgrade pip as suggested:
python3 -m pip install --upgrade pip
Usage indicates the main entry point is the mergekit-yaml script, which consumes a YAML config and outputs to a directory:
mergekit-yaml path/to/your/config.yml ./output-model-directory [--cuda] [--lazy-unpickle] [--allow-crimes] [... other options]
For more argument details, run:
mergekit-yaml --help
Recent releases
Latest two releases are:
- v0.1.4 v0.1.4 (2025-10-31)
- legacy (2023-10-06): MIT-licensed legacy scripts. Note that later versions of mergekit are licensed under the LGPL.
Traction
Despite not listing popularity metrics in detail here, the repository has 7281 stars as a raw number.
Behind the repo
Not provided in the provided material.
Caveats
- License: LGPL-3.0 for current versions; legacy scripts were MIT-licensed (per release notes).
- Creation date: 2023-08-21; last push: 2026-06-17.
- Language: Python.
- Open issues: 273.
- Primary language and license are stated in the repo metadata and README excerpts.