Downloads · 30 days
25
6% of all-time downloads
lzlv-longLORA-70b-rope8-32k-fp16
grimulkan/lzlv-longLORA-70b-rope8-32k-fp16
lzlv-longLORA-70b-rope8-32k-fp16 is a text generation model from grimulkan. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-2.0.
This is a merge of LongAlpaca-70B-lora into lizpreciatior's lzlv70bfp16hf, and removing the extra row and pad token so that the vocabularies match.
All-time downloads
419
Public
Parameters
69B
138 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors138 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- cc-by-nc-2.0
- Model type
- llama
- Access
- Public
- Created
- Jan 22, 2024
- Updated
- Mar 31, 2024
- SHA
- aab5b78a
Try a prompt
Model card
From the Hugging Face model README
This is a merge of LongAlpaca-70B-lora into lizpreciatior's lzlv_70b_fp16_hf, and removing the extra row and pad token so that the vocabularies match.
There is no additional fine-tuning. The resulting model seems to not be broken... you can test whether it is truly the original model + 32K capability (use linear rope scaling 8).
ChuckMcSneed did a benchmark here, indicating 30% degradation with 8x the context length.
You could also try merging this with other models of longLORA descendency (like Aurelian).
A 6-bit EXL2 quantization is available here, and 4 -bit EXL2 here.
See this discussion for how to create merges like these.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- cc-by-nc-2.0
- Created
- Jan 22, 2024
- Updated
- Mar 31, 2024