Downloads · 30 days
12
10% of all-time downloads
Pallas-0.5-LASER-exp2-0.1
Mihaiii/Pallas-0.5-LASER-exp2-0.1
Pallas-0.5-LASER-exp2-0.1 is a text generation model from Mihaiii. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This model has 3 laser interventions applied on Mihaiii/Pallas-0.5 .
All-time downloads
126
Public
Parameters
34.4B
68.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors68.8 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Model type
- llama
- Access
- Public
- Created
- Jan 4, 2024
- Updated
- Jan 5, 2024
- SHA
- da4d910a
Try a prompt
Base models
Model card
From the Hugging Face model README
This model has 3 laser interventions applied on Mihaiii/Pallas-0.5 .
All interventions were made on mlp layers (meaning: ["mlp.gate_proj.weight", "mlp.up_proj.weight", "mlp.down_proj.weight"]) with a rate of 8.
| Intervention number | Layer number | Validation acc (higher is better) | Validation logloss (lower is better) | Test acc (higher is better) | Test logloss (lower is better) |
|---|---|---|---|---|---|
| 0 | - | 55.263 | 1.651 | 59.868 | 1.463 |
| 1 | 56 | 55.263 | 1.548 | 60.526 | 1.363 |
| 2 | 54 | 55.263 | 1.505 | 61.184 | 1.332 |
| 3 | 51 | 55.263 | 1.488 | 60.526 | 1.336 |
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- other
- Created
- Jan 4, 2024
- Updated
- Jan 5, 2024