qwen2.5-0.5b

Instruction Residuals

This repository contains instruction residuals (delta weights) computed as the parameter-wise difference between Qwen/Qwen2.5-0.5B-Instruct and Qwen/Qwen2.5-0.5B.

Apply these residuals to the base model to reconstruct the instruction-tuned weights without retraining.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from residuals import Residuals

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B")

res = Residuals.from_pretrained("residuals/qwen2.5-0.5b")
res.apply(base, base_tokenizer=tok)

Provenance

• Created at: 2025-10-25T16:03:08.291758+00:00
• DType: float32
• Parameters: 291
• Shapes hash: e64ddbb9648be4a1ece194f4ebb0071093be321dbb0f12dde9f7a9e6d56af1b5
• Names hash: 8547cba9ee36af43b751a8e46990036e714a1e6ce08e5c1a38cced2ca105ffea
• Base model: Qwen/Qwen2.5-0.5B
• Instruction model: Qwen/Qwen2.5-0.5B-Instruct

Files

• model.safetensors: Serialized residual tensors (safetensors format).
• (optional) model.safetensors.index.json + shard files model-00001-of-000N.safetensors, ... for multi-part weights.
• config.json: Residuals metadata and provenance.
• tokenizer files: Saved tokenizer for compatibility.

About this format

These are additive residuals (task vectors). Applying them to the base model's parameters reconstructs the instruction-tuned model.

Tools

Generated with the residuals Python package. Install via: pip install residuals.

• PyPI: https://pypi.org/project/residuals/
• Source: https://github.com/omarish/residuals