Tencent-Hy3-295B-A21B-NVFP4

Hy3 295B-A21B — NVFP4 for veloGB10 (NVIDIA DGX Spark / GB10)

Original model: tencent/Hy3 by Tencent's Hunyuan team, Apache-2.0. This artifact contains modified weights (NVFP4 quantization); the original model is © its authors.

NVFP4-quantized variant of Hy3 (295B total / ~21B activated per token, MoE). This quantized artifact exists first and foremost to run with the veloGB10 inference engine (gb10_inference) on NVIDIA DGX Spark / GB10 systems — it is produced, validated, and tuned for veloGB10's kernels. The weights use the standard HuggingFace compressed-tensors layout (nvfp4-pack-quantized), so this artifact can be used for any purpose, with any framework that reads the format. It is in particular fit and proven to work with veloGB10 — two-node TP=2 serving, native NextN speculative decoding, and the engine's greedy-lossless correctness gates all pass on this artifact.

Specifications

Running with veloGB10

Full instructions: veloGB10 README (build or download a release binary, then). The binary loads its PTX kernel artifacts from the current directory — run it from wherever the binary + *.ptx live. TP=2 is required for this model; the node needs no model copy and no configuration — the head ships weights, settings, and calibration at sync.

On the second GB10:

gb10_inference --node --port 29500

On the head node:

gb10_inference --server --model-dir=/path/to/hy3-nvfp4 --tp \
  --nodes <peer-ip>:29500 --port=9000 --max-seq-len=32768 --prefix-cache=on

Throughput and tool-eval numbers for this artifact are tracked in the veloGB10 README benchmarks section.

Notes

• Tokenizer, chat template, and generation config are included as shipped by the quantization pipeline (chat format: Hunyuan hy_* tags with <think> blocks).
• Quantization can degrade output quality relative to the bf16 original; validate on your own workload. Report issues via the veloGB10 issue tracker.
• Intended for research and local/agentic serving use, under the base model's Apache-2.0 terms.