Downloads · 30 days
24
9% of all-time downloads
Mixtral-8x22B-Instruct-v0.1-FP8-v2
comaniac/Mixtral-8x22B-Instruct-v0.1-FP8-v2
Mixtral-8x22B-Instruct-v0.1-FP8-v2 is a text generation model from comaniac. Use it when you need the model to write or continue text. It is set up for transformers.
Weights and activations are per-tensor quantized to float8e4m3. Quantization with AutoFP8 with updated activation scaling factor names. Calibration dataset: Ultrachat (mgoin/ultrachat2k) Samples: 2048 Sequence length:…
All-time downloads
274
Public
Parameters
141B
141 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors141 GB · 100%
Parameter types
How the weights are stored.
F8_E4M3140B · 100%
Try a prompt
Model card
From the Hugging Face model README
Mixtral-8x22B-Instruct-v0.1-FP8-v2
- Weights and activations are per-tensor quantized to float8_e4m3.
- Quantization with AutoFP8 with updated activation scaling factor names.
- Calibration dataset: Ultrachat (mgoin/ultrachat_2k)
- Samples: 2048
- Sequence length: 8192
Evaluation
TBA
- Task
- Text Generation
- Library
- transformers
- Type
- mixtral
- Created
- Jun 10, 2024
- Updated
- Jun 10, 2024