Downloads · 30 days
7
19% of all-time downloads
D1zzYzz/GRIT-Full-llama-3.2-3B-alpaca-r16
GRIT-Full-llama-3.2-3B-alpaca-r16 is a text generation model from D1zzYzz. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
This model is a fine-tuned version of unsloth/Llama-3.2-3B-Instruct-bnb-4bit using the GRIT (Geometric Reprojection Instruction Tuning) algorithm and QLoRA on the tatsu-lab/alpaca dataset.
Downloads · 30 days
7
19% of all-time downloads
All-time downloads
37
Public
Repo size
27.7 MB
Likes
0
Public
Click a slice to open those files.
.json17.3 MB · 62%
From the Hugging Face model README
This model is a fine-tuned version of unsloth/Llama-3.2-3B-Instruct-bnb-4bit using the GRIT (Geometric Reprojection Instruction Tuning) algorithm and QLoRA on the tatsu-lab/alpaca dataset.
The base model is quantized to 4-bit (NF4) and optimized with Unsloth to enable efficient fine-tuning.
In benchmark comparisons, GRIT has shown faster convergence and better stability than standard LoRA or fine-tuning, making it well-suited for efficient single-epoch training. The use of Unsloth further accelerates this process.
If you use this model, please cite the original GRIT paper and:
@misc{grit-lora-Llama-3.2-3B-Instruct-bnb-4bit-alpaca},
title={ unsloth/Llama-3.2-3B-Instruct-bnb-4bit Fine-tuned with GRIT on tatsu-lab/alpaca },
author={D1zzYzz},
year={2024},
publisher={Hugging Face},
url={https://huggingface.co/D1zzYzz/GRIT-Full-llama-3.2-3B-alpaca-r16}
}
This model inherits the Apache 2.0 license.