Downloads · 30 days
71
0% of all-time downloads
platypus-2-22b-relora
chargoddard/platypus-2-22b-relora
platypus-2-22b-relora is a text generation model from chargoddard. Use it when you need the model to write or continue text. It is set up for transformers.
Experimental ReLoRA-trained model using the OpenPlatypus dataset. Ran for one epoch, with three lora restarts.
All-time downloads
72K
Public
Parameters
21.8B
130 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors43.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Aug 17, 2023
- Updated
- Nov 23, 2023
- SHA
- e7eb3e55
Try a prompt
Datasets
Model card
From the Hugging Face model README
Experimental ReLoRA-trained model using the OpenPlatypus dataset. Ran for one epoch, with three lora restarts.
Not recommended for use yet. Mostly tossing this up for testing.
Base model was llama2-22b-blocktriangular.
Relevant training parameters:
adapter: qlora
load_in_4bit: true
lora_r: 32
lora_alpha: 16
lora_dropout: 0.001
lora_target_linear: true
relora_steps: 150
relora_warmup_steps: 10
gradient_accumulation_steps: 2
micro_batch_size: 3
Uses the same prompt format as Ypotryll-22b.
Prefix messages with " ***System:", " ***Query:", or " ***Response:", paying attention to whitespace.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 52.21 |
| ARC (25-shot) | 57.68 |
| HellaSwag (10-shot) | 82.44 |
| MMLU (5-shot) | 55.33 |
| TruthfulQA (0-shot) | 43.61 |
| Winogrande (5-shot) | 77.35 |
| GSM8K (5-shot) | 6.6 |
| DROP (3-shot) | 42.46 |
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Languages
- en
- Created
- Aug 17, 2023
- Updated
- Nov 23, 2023