Downloads · 30 days
15
4% of all-time downloads
Meta-Llama-3-70B-Instruct-FP8-v1
comaniac/Meta-Llama-3-70B-Instruct-FP8-v1
Meta-Llama-3-70B-Instruct-FP8-v1 is a text generation model from comaniac. Use it when you need the model to write or continue text. It is set up for transformers.
Weights and activations are per-tensor quantized to float8e4m3. Quantization with AutoFP8. Calibration dataset: Ultrachat (mgoin/ultrachat2k) Samples: 1024 Sequence length: 4096
All-time downloads
393
Public
Parameters
70.6B
72.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors72.7 GB · 100%
Parameter types
How the weights are stored.
F8_E4M368.5B · 97%
Try a prompt
Model card
From the Hugging Face model README
Llama-3-70B-Instruct-FP8-v1
- Weights and activations are per-tensor quantized to float8_e4m3.
- Quantization with AutoFP8.
- Calibration dataset: Ultrachat (mgoin/ultrachat_2k)
- Samples: 1024
- Sequence length: 4096
Evaluation
TBA
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- May 25, 2024
- Updated
- May 26, 2024