Downloads · 30 days
88
3% of all-time downloads
RichardErkhov/bhenrym14_-_airoboros-3_1-yi-34b-200k-gguf
bhenrym14_-_airoboros-3_1-yi-34b-200k-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
88
3% of all-time downloads
All-time downloads
2.9K
Public
Repo size
396 GB
Likes
0
Public
Click a slice to open those files.
.gguf441 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
airoboros-3_1-yi-34b-200k - GGUF
license: other license_name: yi-license license_link: LICENSE datasets:
This is larryvrh/Yi-34B-200K-Llamafied, with instruction tuning performed with Jon Durbin's jondurbin/airoboros-3.1 dataset. That base model is 01-ai/Yi-34B-200k, but using llama2 model definitions and tokenizer to remove any remote code requirements.
This is a (merged) QLoRA fine-tune (rank 64).
The finetune was performed with 1x RTX 6000 Ada (~80 hours to this checkpoint). Prompts were truncated to 4096 tokens (for speed and VRAM headroom).
I have done very little testing with this model, so feedback on real world performance is appreciated!
Use as you would any other Hugging Face fp16 llama-2 model.
Model was trained with llama-2 chat prompt format. See jondurbin/airoboros-l2-13b-3.1.1 model card for details.