Downloads · 30 days
63
3% of all-time downloads
landmark-33b
alpindale/landmark-33b
landmark-33b is a text generation model from alpindale. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model has been trained using the PEFT LoRA technique with the Landmark Attention method over 200 steps. Model will likely be trained further and updated later on.
All-time downloads
2.2K
Public
Repo size
130 GB
Likes
6
Public
Repo makeup
Click a slice to open those files.
.bin65.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Jun 13, 2023
- Updated
- Jun 14, 2023
- SHA
- 11617325
Try a prompt
Model card
From the Hugging Face model README
Landmark Attention LLaMA 33B
This model has been trained using the PEFT LoRA technique with the Landmark Attention method over 200 steps. Model will likely be trained further and updated later on.
Usage
Requires trust_remote_code to be set to True. In oobabooga, you can simply add the --trust_remote_code flag.
You will also need to disable the Add the bos_token to the beginning of prompts option in the settings.
PEFT Checkpoint
You can probably merge the checkpoint with any other LLaMA-based model (provided they're 33B, of course). This repo contains the merged weights, but you can grab the adapter here.
Training Code
You can find the training code here.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- Jun 13, 2023
- Updated
- Jun 14, 2023