qwen3-4b-instruct-sft-4096

base_model: unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit<br> library_name: peft<br> pipeline_tag: text-generation<br> tags:<br>

• base_model:adapter:unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit
• lora
• sft
• long-context
• transformers
• unsloth
• qwen

Model Card for qwen3-4b-instruct-sft-4096

This repository contains a LoRA adapter fine-tuned on top of
Qwen3-4B-Instruct (Unsloth 4bit variant) using Supervised Fine-Tuning (SFT)
with 4096-token context support.

The model is designed to improve instruction following accuracy and output stability,
especially for long-context tasks and competition-style evaluations.

---

Model Details

Model Description

This model is a QLoRA-based SFT adapter trained with the following key characteristics:

• Output-focused learning (assistant-only loss)
• Chain-of-Thought treated as input context (not optimized directly)
• Long-context support up to 4096 tokens
• Optimized for A100-class GPUs

The adapter can be merged into the base model or loaded dynamically via PEFT.

• Developed by: WassyO
• Model type: LoRA Adapter (PEFT)
• Language(s): English, Japanese
• License: Inherits license from the base model (Qwen3 license)
• Finetuned from model:
  unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit

Model Sources

• Base Model:
  https://huggingface.co/unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit
• Training Frameworks:
  Transformers / PEFT / Unsloth
• Inference Engine (recommended):
  vLLM

---

Uses

Direct Use

This model is intended to be used as a drop-in LoRA adapter for:

• Long-context instruction following
• Structured question answering
• Competition or benchmark-style inference
• Offline batch inference via vLLM

Example use cases:

• Solving tasks with long problem descriptions
• Generating stable, format-consistent outputs
• JSON-based submission generation

Downstream Use

The adapter can be further extended with:

• DPO (Direct Preference Optimization)
• Domain-specific SFT
• Additional long-context fine-tuning (e.g. 8k context)

Out-of-Scope Use

• Safety-critical decision making
• Medical or legal advice
• Real-time conversational agents without additional alignment

---

Bias, Risks, and Limitations

• The model inherits biases present in the base model and training data
• Reasoning quality depends on prompt clarity and structure
• The model is optimized for final answer quality, not for exposing internal reasoning steps
• Long-context usage increases memory and compute requirements

Recommendations

Users should:

• Validate outputs for high-stakes use cases
• Follow the license terms of the base model
• Avoid interpreting outputs as factual guarantees

---

How to Get Started with the Model

Load with PEFT (recommended)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit"
adapter_id = "WassyO/qwen3-4b-instruct-sft-4096"

tokenizer = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_id,
    load_in_4bit=True,
    device_map="auto",
    trust_remote_code=True,
)

model = PeftModel.from_pretrained(model, adapter_id)

Merge Adapter (optional, for vLLM)

merged = model.merge_and_unload()
merged.save_pretrained("./merged_model")
tokenizer.save_pretrained("./merged_model")

---

Training Details

Training Data

• Structured instruction-following dataset

• Each sample contains:

  • User query
  • Assistant response
  • Explicit output markers (e.g. Output:, 回答:)

The dataset emphasizes final answer correctness rather than intermediate reasoning steps.

Training Procedure

Preprocessing

• Chat template formatting
• Left-truncation to preserve recent context
• Output-marker detection for loss masking

Training Hyperparameters

• Training regime: bf16 mixed precision
• Context length: 4096
• Optimizer: AdamW
• Scheduler: Cosine
• Epochs: 2
• Learning rate: 1e-6
• LoRA r: 128
• LoRA alpha: 256
• Quantization: 4bit (QLoRA)
• Gradient checkpointing: Enabled

Compute

• Hardware: NVIDIA A100 40GB
• Framework: Unsloth + Transformers + PEFT

---

Evaluation

Testing Data, Factors & Metrics

• Internal validation set
• Competition-style public evaluation tasks

Metrics

• Task-level accuracy
• Output format correctness
• Long-context robustness

Results

Quantitative scores depend on the evaluation setting. Qualitatively, the model shows:

• Improved instruction adherence
• Reduced output instability
• Better handling of long inputs

---

Environmental Impact

• Hardware Type: NVIDIA A100 40GB
• Precision: bf16 + 4bit quantization
• Estimated Impact: Reduced compared to full fine-tuning due to LoRA

---

Technical Specifications

Model Architecture and Objective

• Transformer-based causal language model
• LoRA adapters injected into attention and MLP layers
• Objective: next-token prediction on assistant outputs only

Software

• transformers
• peft
• unsloth
• vllm (inference)

---

Citation

If you use this model, please cite the base Qwen model and PEFT.

---

Model Card Authors

• WassyO

Model Card Contact

• Hugging Face: https://huggingface.co/WassyO

Framework Versions

• PEFT 0.18.1