my-sft-model

my-sft-model

Model Details

Model Description

This model is a Supervised Fine-Tuned (SFT) version of Qwen2.5-1.5B, trained using Unsloth with LoRA adapters for parameter-efficient fine-tuning on limited hardware.

The goal of this model is to learn improved instruction-following behavior from curated prompt–response examples.
It serves as a foundation checkpoint for future experimentation, including reward modeling and reinforcement learning (RLHF / GRPO).

• Developed by: Shreshth Sudhir
• Model type: Supervised Fine-Tuned (SFT) Language Model
• Language(s): English
• License: Same as base model (Qwen2.5 license)
• Finetuned from: unsloth/qwen2.5-1.5b-unsloth-bnb-4bit

---

Uses

Direct Use

This model can be used for:

• Instruction-following text generation
• Educational explanations
• Conversational responses
• Learning and experimentation with LoRA-based fine-tuning

---

Downstream Use

This model is intended to be further fine-tuned for:

• Reward modeling
• Reinforcement Learning with Human Feedback (RLHF / GRPO)
• Style or domain adaptation

---

Out-of-Scope Use

This model is not suitable for:

• Medical, legal, or financial advice
• Safety-critical or production systems
• High-stakes decision making

---

Bias, Risks, and Limitations

• The model may reflect biases present in the base model and training data
• No explicit safety or alignment tuning has been applied
• Generated responses may be inaccurate or incomplete

Users should apply human oversight when using this model.

---

How to Get Started

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="./my_sft_model",
    max_seq_length=2048,
    load_in_4bit=True,
)

FastLanguageModel.for_inference(model)

prompt = "Explain gravity simply."
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,
    top_p=0.9,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

• Custom instruction–response dataset created for learning supervised fine-tuning
• Data formatted using an instruction–response prompt template
• No proprietary, private, or sensitive data was used

---

Training Procedure

• Training method: Supervised Fine-Tuning (SFT)
• Fine-tuning strategy: LoRA (Low-Rank Adaptation)
• Frameworks: Unsloth, TRL, Hugging Face Transformers
• Base model: unsloth/qwen2.5-1.5b-unsloth-bnb-4bit

Training Hyperparameters

• Precision: 4-bit quantization (bnb 4-bit)
• LoRA rank (r): 16
• Optimizer: AdamW
• Training regime: Mixed precision training

---

Speeds, Sizes, and Times

• Training performed on a single GPU
• Parameter-efficient fine-tuning enabled via LoRA
• Short-duration experimental training run

Framework versions

• PEFT 0.18.0