Downloads · 30 days
66
15% of all-time downloads
llama2_70b_longlora_fp16_32k_ROPE8
grimulkan/llama2_70b_longlora_fp16_32k_ROPE8
llama2_70b_longlora_fp16_32k_ROPE8 is a text generation model from grimulkan. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
This is the same as Yukang's Llama-2-70b-longlora-32k, except that the extra pad token has been stripped from the tokenizer to make it similar to the base Llama model (and it has been merged into the base model). Plea…
All-time downloads
427
Public
Parameters
69B
148 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors138 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Oct 10, 2023
- Updated
- Jan 19, 2024
- SHA
- 201946d5
Try a prompt
Model card
From the Hugging Face model README
This is the same as Yukang's Llama-2-70b-longlora-32k, except that the extra pad token has been stripped from the tokenizer to make it similar to the base Llama model (and it has been merged into the base model). Please refer to that page for more details.
It was created by merging LongAlpaca-70B-lora into Llama-2-70b, replacing the embed and norm layers as described in the LongLoRA repo, and removing the extra row and pad token.
This is not an instruct-tuned model, but a base model for further fine-tuning. It supports 32K of context with linear rope scaling of 8.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Created
- Oct 10, 2023
- Updated
- Jan 19, 2024