Downloads · 30 days
14
14% of all-time downloads
amanwalksdownthestreet/Hermes-4-70B-MXFP8
Hermes-4-70B-MXFP8 is a text generation model from amanwalksdownthestreet. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.
MXFP8 quantization of NousResearch/Hermes-4-70B. Weights only — prompt format, chat template, tool calling, and license are the base model's.
Downloads · 30 days
14
14% of all-time downloads
All-time downloads
100
Public
Parameters
72.7B
74.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors74.8 GB · 100%
How the weights are stored.
F8_E4M368.5B · 94%
From the Hugging Face model README
MXFP8 quantization of NousResearch/Hermes-4-70B. Weights only — prompt format, chat template, tool calling, and license are the base model's.
Requires Blackwell (compute capability 12.0). MXFP8 needs Blackwell tensor-core support in vLLM; does not run on Ada/Hopper.
compressed-tensors format (llm-compressor 0.11.1)lm_head, embeddings. KV cache: BF16neuralmagic/calibration (LLM), max seq 20480-shot log-likelihood, lm-evaluation-harness on vLLM. Not comparable to chat/thinking-mode numbers.
| acc | |
|---|---|
| Overall | 78.91% ± 0.33 |
| Social sciences | 88.20% |
| Other | 82.56% |
| Humanities | 75.39% |
| STEM | 71.52% |
vllm serve amanwalksdownthestreet/Hermes-4-70B-MXFP8 \
--max-model-len 49152 --gpu-memory-utilization 0.95
Quantization only. Credit: NousResearch.