Downloads · 30 days
0
qwen2.5-7b-qlora-nl2sql
thefounder03/qwen2.5-7b-qlora-nl2sql
qwen2.5-7b-qlora-nl2sql is a machine learning model from thefounder03. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
LoRA adapter weights for Qwen/Qwen2.5-7B-Instruct, fine-tuned on the Spider text-to-SQL dataset using a from-scratch QLoRA implementation — no bitsandbytes, no peft, no black-box abstractions.
Access
Public
Updated Jul 16, 2026
Repo size
10.1 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt10.1 MB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jul 16, 2026
- Updated
- Jul 16, 2026
- SHA
- 787d81c5
Base models
Datasets
Model card
From the Hugging Face model README
Qwen2.5-7B-Instruct — QLoRA NL2SQL Adapter
LoRA adapter weights for Qwen/Qwen2.5-7B-Instruct, fine-tuned on the Spider text-to-SQL dataset using a from-scratch QLoRA implementation — no bitsandbytes, no peft, no black-box abstractions.
Full training code, implementation details, and loading instructions: github.com/dhruvkachhela/fine_tunning_QLORA
Training Details
| Setting | Value |
|---|---|
| Base model | Qwen/Qwen2.5-7B-Instruct |
| Dataset | Spider (7,000 train / 1,034 val) |
| Task | Natural language to SQL |
| Quantization | NF4 4-bit, block size 64, double quantization (8-bit scales) |
| LoRA rank | 8 |
| LoRA alpha | 16 |
| Target modules | q_proj, v_proj (all 28 layers) |
| Trainable params | 2,523,136 (0.036% of 7B) |
| Learning rate | 1e-4 with linear warmup + decay |
| Training steps | 1,000 |
| Batch size | 2 |
| Hardware | Kaggle T4 GPU (16GB VRAM) |
What is in this repo
| File | Description |
|---|---|
best_qlora_checkpoint.pt | LoRA adapter weights (lora_A + lora_B, all 28 layers) |
Base model weights are not included — load Qwen/Qwen2.5-7B-Instruct directly from HuggingFace.
Implementation
Built from scratch in PyTorch:
- NF4 Codebook — 16 levels at quantiles of a standard normal distribution
- Block-wise Scaling — block size 64, per-block absmax normalization
- Double Quantization — scale factors quantized from float32 to 8-bit
- LoRA — rank-8 adapters on q_proj + v_proj, only lora_A and lora_B train
- Dequantize on the fly — weights reconstructed each forward pass
References
- QLoRA paper — Dettmers et al., 2023
- LoRA paper — Hu et al., 2021
- Spider dataset — Yu et al., 2018
- Training repo: github.com/dhruvkachhela/fine_tunning_QLORA
- License
- apache-2.0
- Languages
- en
- Created
- Jul 16, 2026
- Updated
- Jul 16, 2026