Downloads · 30 days
310
7% of all-time downloads
sroecker/Qwen3.6-35B-REAP-Pruned-ratio-0.5-NVFP4-GGUF
Qwen3.6-35B-REAP-Pruned-ratio-0.5-NVFP4-GGUF is a machine learning model from sroecker. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This repository contains a GGUF conversion of Qwen3.6-35B-REAP-Pruned-ratio-0.5-NVFP4, which is an NVFP4 compressed-tensors quantized version of RangerX/Qwen3.6-35B-REAP-Pruned-ratio-0.5.
Downloads · 30 days
310
7% of all-time downloads
All-time downloads
4.5K
Public
Repo size
13.4 GB
Likes
2
Public
Click a slice to open those files.
.gguf13.4 GB · 100%
From the Hugging Face model README
This repository contains a GGUF conversion of
Qwen3.6-35B-REAP-Pruned-ratio-0.5-NVFP4, which is an NVFP4
compressed-tensors quantized version of
RangerX/Qwen3.6-35B-REAP-Pruned-ratio-0.5.
The GGUF file was produced from the local
Qwen3.6-35B-REAP-Pruned-ratio-0.5-NVFP4 Hugging Face checkpoint with a patched
llama.cpp converter that handles compressed-tensors NVFP4 tensors for this
model.
uv run /home/sroecker/src/llama.cpp/convert_hf_to_gguf.py \
--verbose \
--outtype auto \
--outfile Qwen3.6-35B-REAP-Pruned-ratio-0.5-NVFP4-GGUF/Qwen3.6-35B-REAP-Pruned-ratio-0.5-NVFP4.gguf \
Qwen3.6-35B-REAP-Pruned-ratio-0.5-NVFP4
The converter detected BF16 tensors and exported the model with GGUF file type
39. The resulting GGUF contains 1293 tensors:
280 NVFP4 tensors152 BF16 tensors861 F32 tensorsThe conversion was based on the process documented in
knoopx/Qwen3.6-35B-A3B-NVFP4-GGUF.
The converter-side changes used here follow the approach from the open
llama.cpp PR
#21095, which adds
conversion support for Hugging Face NVFP4 models quantized with
compressed-tensors. Native Blackwell NVFP4 CUDA runtime support is tracked
separately in #22196.
The local converter used here includes the relevant converter-side handling:
compressed-tensors checkpoints with format nvfp4-pack-quantized as
NVFP4 inputsweight_packed, weight_global_scale,
and input_global_scale to the ModelOpt-style names expected by the
repacker1444df51289cfa8063b96f0e62b1125440111bc79a52003ea14b6eac7016fd5f as
qwen35| File | Size |
|---|---|
Qwen3.6-35B-REAP-Pruned-ratio-0.5-NVFP4.gguf | 13G |