Downloads · 30 days
30
12% of all-time downloads
llama3-8b-instruct-CLA-3
answerdotai/llama3-8b-instruct-CLA-3
llama3-8b-instruct-CLA-3 is a text generation model from answerdotai. Use it when you need the model to write or continue text. It is set up for transformers.
Transformers : https://github.com/huggingface/transformers/blob/47fedc1665f14a60260b1b8357e682669300093a/src/transformers/models/llama/modelingllama.py
All-time downloads
259
Public
Repo size
16.1 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors16.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Jul 18, 2024
- Updated
- Jul 18, 2024
- SHA
- dcf15770
Try a prompt
Model card
From the Hugging Face model README
Training Script: https://github.com/AnswerDotAI/fsdp_qlora/blob/3f7c583e985ff35e37a7b7497a7d4fedb77df695/experiments/cla/train.sh
This model shares KV activations every 3 layers. For example, layer 1 and 2 uses layer 0 kv activations, layer 4 and 5 uses layer 3 kv activations, etc..
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Jul 18, 2024
- Updated
- Jul 18, 2024