awleai-DeepSeek-R1-Distill-Qwen-1.5B

awleai-DeepSeek-R1-Distill-Qwen-1.5B

Model Description

awleai-DeepSeek-R1-Distill-Qwen-1.5B is a QLoRA fine-tuned adapter model based on
unsloth/deepseek-r1-distill-qwen-1.5b-unsloth-bnb-4bit.

The model was fine-tuned using UnsLoTH with Supervised Fine-Tuning (SFT) on custom JSONL datasets to improve instruction-following and domain-specific reasoning.

⚠️ This repository contains LoRA adapter weights only, not a fully merged standalone model.

---

Model Type

• Architecture: Transformer (Causal Language Model)
• Fine-tuning method: QLoRA
• Library: PEFT
• Precision: 4-bit base model + LoRA adapters

---

Base Model

• unsloth/deepseek-r1-distill-qwen-1.5b-unsloth-bnb-4bit

---

Training Details

Training Setup

• Framework: UnsLoTH
• Trainer: TRL SFTTrainer
• Epochs: 2
• Learning Rate: 2e-4
• Effective Batch Size: 8 (with gradient accumulation)
• Precision: FP16
• LoRA Type: QLoRA

Hardware

• GPU: NVIDIA A100
• Platform: Google Colab Pro+
• CUDA: 12.x

---

Training Data

The model was fine-tuned on private, custom JSONL instruction-style datasets.
The dataset is not publicly released.

---

How to Use

Load with UnsLoTH (Recommended)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    "Henryvante/awleai-DeepSeek-R1-Distill-Qwen-1.5B",
    load_in_4bit=True
)