# Adapter Merge, Export & Evaluation

## Merging for deployment

```python
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B")
model = PeftModel.from_pretrained(base_model, "./lora-adapter")
model = model.merge_and_unload()
model.save_pretrained("./merged-model")
```

After merging, the adapter weights are baked into the base model — zero inference overhead.

## Export to inference runtimes

### GGUF (llama.cpp)
```bash
# Convert merged model to GGUF
python convert-hf-to-gguf.py ./merged-model --outtype q8_0
# Or quantize
./llama-quantize ./merged-model/ggml-model-f16.gguf q4_k_m
```

### ONNX
```bash
python -m optimum.exporters.onnx --model ./merged-model --task text-generation ./onnx-export/
```

## Evaluation checklist

After fine-tuning, ALWAYS evaluate:

1. **Target task**: run the model on held-out examples; compare to baseline
2. **Capability regression**: check general benchmarks (MMLU, GSM8K subset)
3. **Generation quality**: inspect 10–20 outputs manually for:
   - Repetition (overfitting signal)
   - Format compliance (did the chat template work?)
   - Hallucination rate (did it forget base knowledge?)

## Common quality issues

| Symptom | Likely cause | Fix |
|---------|-------------|-----|
| Repetition | Too many epochs, LR too high | Reduce epochs, lower LR |
| Ignores system prompt | Wrong chat template | Verify `apply_chat_template` output |
| Forgets base knowledge | Overtraining, skewed dataset | Add diverse general data, fewer epochs |
| Broken formatting | Template mismatch | Inspect actual tokenized→decoded text |

## Size comparison

| Model | Full FT (GB) | LoRA adapter (MB) | QLoRA adapter (MB) |
|-------|-------------|-------------------|-------------------|
| Llama 3.2 3B | ~6 | ~50 | ~50 |
| Llama 3.2 8B | ~16 | ~60 | ~60 |
| Mistral 7B | ~14 | ~60 | ~60 |
| Llama 3 70B | ~140 | ~200 | ~200 |