Downloads · 30 days
4
6% of all-time downloads
yuvraj17/Llama-3-8B-spectrum-25-GPTQ
Llama-3-8B-spectrum-25-GPTQ is a text generation model from yuvraj17. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
4
6% of all-time downloads
All-time downloads
66
Public
Parameters
8B
5.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors5.7 GB · 100%
How the weights are stored.
I327B · 87%
From the Hugging Face model README
This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the yuvraj17/finetune_alpaca_1K dataset. It achieves the following results on the evaluation set:
I have used the Spectrum Fine-tuning method as described in Eric Hartford et. al 2024, which selectively targets some t% of the model layers with the highest Signal-to-Noise Ratio (SNR). By focusing on the most information-dense layers, this approach maximizes fine-tuning efficiency while minimizing compute resources.
The key goal of Spectrum Fine-tuning is: minimize the memory footprint and accelerate LLM training without sacrificing performance.
The 25% layer selection ensures minimal computational overhead for fine-tuning.
The following hyperparameters were used during training:

