comaniac

Meta-Llama-3-70B-Instruct-FP8-v2

comaniac/Meta-Llama-3-70B-Instruct-FP8-v2

Meta-Llama-3-70B-Instruct-FP8-v2 is a text generation model from comaniac. Use it when you need the model to write or continue text. It is set up for transformers.

Weights and activations are per-tensor quantized to float8e4m3. Quantization with AutoFP8 with the updated activation scaling factor names. Calibration dataset: Ultrachat (mgoin/ultrachat2k) Samples: 1024 Sequence len…