Downloads · 30 days
0
bhenrym14/airoboros-33b-gpt4-1.4.1-lxctx-PI-16384-LoRA
airoboros-33b-gpt4-1.4.1-lxctx-PI-16384-LoRA is a machine learning model from bhenrym14. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
NOTE: This LoRA was trained on Llama-30b AFTER additional pretraining. I intend on providing the LoRA of that pretraining too. Applying this LoRA to base Llama-30b will likely result in a performance reduction. I have…
Downloads · 30 days
0
Access
Public
Updated Jul 14, 2023
Repo size
975 MB
Likes
2
Public
Click a slice to open those files.
.bin975 MB · 100%
From the Hugging Face model README
NOTE: This LoRA was trained on Llama-30b AFTER additional pretraining. I intend on providing the LoRA of that pretraining too. Applying this LoRA to base Llama-30b will likely result in a performance reduction. I have uploaded the fp16 merged weights here
Mostly untested!
Find GPTQ quantized weights and full model card here: https://huggingface.co/bhenrym14/airoboros-33b-gpt4-1.4.1-lxctx-PI-16384-GPTQ
This is Jon Durbin's Airoboros 33B GPT4 1.4 (LoRA) with several key modifications:
This is a QLoRA fine-tune
Pretraining took 10 hours. Finetuning took ~41 hours on 1x RTX 6000 Ada.