Downloads · 30 days
18
6% of all-time downloads
Hermes-4-14B-NVFP4
tonythethompson/Hermes-4-14B-NVFP4
Hermes-4-14B-NVFP4 is a text generation model from tonythethompson. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
NVFP4-quantized version of NousResearch/Hermes-4-14B, an instruction-following and function-calling model built on Qwen/Qwen3-14B. It is not a newly trained model. This is an unofficial community quantization; NousRes…
All-time downloads
306
Public
Parameters
8.2B
10.6 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors10.5 GB · 100%
Parameter types
How the weights are stored.
U86.6B · 73%
Try a prompt
Base models
Model card
From the Hugging Face model README
Hermes-4-14B — NVFP4
Overview
NVFP4-quantized version of NousResearch/Hermes-4-14B, an instruction-following and function-calling model built on Qwen/Qwen3-14B. It is not a newly trained model. This is an unofficial community quantization; NousResearch is the original model author.
NVFP4 is NVIDIA's 4-bit floating-point quantization format. This variant targets high-throughput inference on NVIDIA hardware with NVFP4 acceleration. For details of the NVFP4 format and its hardware support, consult NVIDIA's TensorRT-LLM documentation.
Source
| Field | Value |
|---|---|
| Upstream model | NousResearch/Hermes-4-14B |
| Upstream source revision | d6ce765c8b83f847357b98254be079afa0c6ca76 |
| Export tool/script | NVIDIA TensorRT Model Optimizer (modelopt) 0.45.0 |
| Quantization recipe | NVFP4 (NVIDIA 4-bit floating point) safetensors |
| Base model (weights) | Qwen/Qwen3-14B |
Files
| File | Size | Description |
|---|---|---|
model-00001-of-00002.safetensors | ~8.99 GB | NVFP4 transformer weights: U8-packed 4-bit + FP8 (E4M3) group scales + FP32 global/input scales + BF16 norms |
model-00002-of-00002.safetensors | ~1.56 GB | BF16 embeddings / output head (excluded from NVFP4) |
model.safetensors.index.json | — | Weight map / shard index |
config.json | — | Model + quantization config |
hf_quant_config.json | — | modelopt NVFP4 quantization config |
generation_config.json | — | Generation defaults |
chat_template.jinja | — | Chat template |
tokenizer.json + tokenizer_config.json | — | Tokenizer |
.quant_summary.txt | — | Per-module quantization summary log |
Intended Use
A general instruction-following and function-calling LLM, quantized to NVFP4 for high-throughput inference on NVIDIA hardware. The NVFP4 variant targets NVIDIA GPUs with native FP4 tensor-core support; on other GPUs, NVFP4 weights may require emulation or upcast, which reduces the throughput benefit (verify with your TensorRT-LLM version).
Runtime Notes
- Library: Hugging Face Transformers (or TensorRT-LLM for NVFP4 acceleration).
- NVFP4 native acceleration requires NVIDIA GPU support; validate on the target hardware and TensorRT-LLM version before production use.
- Context length: inherited from Qwen3-14B / Hermes-4-14B; consult the upstream model card.
Precision and Packaging
Export tooling, precision, and quantization are recorded in the Source table above. This packaging mirror does not publish independent parity benchmarks; validate on your target execution provider before production use.
Limitations
- NVFP4 native acceleration is hardware-specific; not all NVIDIA GPUs support native FP4 tensor-core operations.
- Quantization introduces approximation error relative to the FP16/BF16 baseline.
- No repository-specific quality benchmark is documented here.
- English-dominant model; multilingual quality may vary.
License
Apache 2.0 — inherited from
NousResearch/Hermes-4-14B. This packaging repo adds no new license terms.
- Task
- Text Generation
- Library
- transformers
- Type
- qwen3
- License
- apache-2.0
- Languages
- en
- Created
- May 31, 2026
- Updated
- Jun 15, 2026