distilled-qwen-1.5b-dolly

Distilled Qwen 2.5 1.5B — LoRA Adapter

Knowledge-distilled student model trained via teacher-student distillation.

Model Description

• Base Model: Qwen/Qwen2.5-1.5B-Instruct
• Teacher Model: Qwen/Qwen2.5-7B-Instruct
• Dataset: databricks/databricks-dolly-15k
• Method: Knowledge Distillation with LoRA fine-tuning

Loading the Model

This repository contains LoRA adapter weights. Load it with the base model using PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-1.5B-Instruct",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", trust_remote_code=True)

# Load adapter
model = PeftModel.from_pretrained(base, "Venkateshwarlu1/distilled-qwen-1.5b-dolly")
model = model.merge_and_unload()  # optional: merge for inference
model.eval()

# Generate
prompt = "Explain machine learning in simple terms."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=150, do_sample=True, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Distillation Method

• Teacher: Qwen/Qwen2.5-7B-Instruct
• Student: Qwen/Qwen2.5-1.5B-Instruct
• Loss: alpha * CE + beta * KL(teacher || student) with temperature scaling

Citation

If you use this model, please cite the original Qwen paper and the distillation dataset.