Concepts
Base models
1 · In one line
A base language model is a pretrained model before further instruction or conversational fine-tuning.
1 · What it is
A base language model is a pretrained model before further instruction or conversational fine-tuning.
A pretrained base version can complete text directly. It can also receive further instruction tuning. A parameter-efficient adapter still needs the base model that was fine-tuned.
Llama 2 includes both pretrained and fine-tuned models. Llama 2-Chat is optimized for dialogue use cases. Base models are not ideal for tasks that require following instructions.
Model documentation distinguishes pretrained base versions from instruction-tuned variants.
Same familyModel families can publish both pretrained and instruction-tuned variants.
Different behaviorBase models suit text completion but are not ideal for tasks that require following instructions.
Adapter dependencyA parameter-efficient adapter still needs the base model that was fine-tuned.
Trace one pretrained base model into direct completion or further tuning.
- 1 · loadStart with the pretrained base version.
- 2 · completeUse the base model to continue an initial text prompt.
- 3 · tuneFurther fine-tune the base version on instructions and conversational data.
- 4 · instructUse the instruction-tuned variant to respond to instructions or requests.
Base identifies the pretrained model before further tuning.
| Who | What they ask | What it works with |
|---|---|---|
| Model developer | “Which pretrained model should receive further tuning?” | Base model lineage |
| Inference engineer | “Is this variant intended for completion or instruction following?” | Variant type |
| Adapter user | “Which original model must be loaded with this adapter?” | Adapter base model |
solves
- It names the pretrained model before instruction or conversational fine-tuning.
- It can be used directly for text completion.
- Using a parameter-efficient adapter also requires loading its base model.
doesn't solve
- Base models are not ideal for tasks that require following instructions.
- An adapter does not replace the base model it was trained against.
6 · Go deeper
Sources used
This explainer is written in original language. The links below support its factual claims.
- docsLLM prompting guide, Hugging Face · read 28 Sept 2026
- docsRun Gemma content generation and inferences, Google AI for Developers · read 28 Sept 2026
- officialGemma model card, Google AI for Developers · read 28 Sept 2026
- paperLlama 2 - Open Foundation and Fine-Tuned Chat Models, Touvron et al. · read 28 Sept 2026
- docsUsing PEFT at Hugging Face, Hugging Face · read 28 Sept 2026