Downloads · 30 days
0
bhenrym14/airoboros-33b-gpt4-1.4.1-NTK-16384-LoRA
airoboros-33b-gpt4-1.4.1-NTK-16384-LoRA is a machine learning model from bhenrym14. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
LoRA Weights can be found here: https://huggingface.co/bhenrym14/airoboros-33b-gpt4-1.4.1-NTK-16384-GPTQ
Downloads · 30 days
0
Access
Public
Updated Jul 7, 2023
Repo size
975 MB
Likes
2
Public
Click a slice to open those files.
.bin975 MB · 100%
From the Hugging Face model README
LoRA Weights can be found here: https://huggingface.co/bhenrym14/airoboros-33b-gpt4-1.4.1-NTK-16384-GPTQ
fp16 weights can be found here: https://huggingface.co/bhenrym14/airoboros-33b-gpt4-1.4.1-NTK-16384-fp16
Analogue with RoPE Position Interpolation (PI) technique: https://huggingface.co/bhenrym14/airoboros-33b-gpt4-1.4.1-PI-8192-LoRA
This is Jon Durbin's Airoboros 33B GPT4 1.4 (LoRA) with several key modifications:
Otherwise, I emulated the training process as closely as possible (rank 64 QLoRA) It was trained on 1x RTX 6000 Ada for ~43 hours.
To use with HF transformers, AutoGPTQ, etc. See NTK monkey patch.