cuda_speed_ntu

leowu0511/cuda_speed_ntu

cuda_speed_ntu is a machine learning model from leowu0511. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as other.

This repository contains NVFP4 (4-bit NVIDIA FP4) quantized GGUF components of the Phr00t/Qwen-Image-Edit-Rapid-AIO-NSFW-v23 merged checkpoint, converted for use with ComfyUI-GGUF.

Downloads · 30 days

23% of all-time downloads

All-time downloads

Public

Repo size

23.6 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf15.6 GB · 98%

At a glance

Library: gguf
License: other
Access: Public
Created: Jul 26, 2026
Updated: Jul 26, 2026
SHA: 3608fb71

Model card

From the Hugging Face model README

Qwen-Image-Edit-Rapid-AIO-NSFW-v23 — NVFP4 GGUF

This repository contains NVFP4 (4-bit NVIDIA FP4) quantized GGUF components of the Phr00t/Qwen-Image-Edit-Rapid-AIO-NSFW-v23 merged checkpoint, converted for use with ComfyUI-GGUF.

Components

File	Size	Description
`qwen-v23-diffusion-NVFP4.gguf`	11.5 GB	MMDiT diffusion backbone (~20B params) — NVFP4 quantized
`text_encoder/text_encoder-NVFP4.gguf`	3.84 GB	Qwen2.5-VL-7B text encoder — NVFP4 quantized
`vae/vae.safetensors`	0.25 GB	VAE decoder (BF16, original format)

Hardware Requirements

GPU: NVIDIA Blackwell (RTX 50-series) with CUDA 13.0+
VRAM: ~10 GB for diffusion model + text encoder + VAE (NVFP4 saves ~4 GB vs FP8)
System RAM: 32 GB+ recommended

Usage with ComfyUI

Install ComfyUI-GGUF
Place qwen-v23-diffusion-NVFP4.gguf in ComfyUI/models/checkpoints/
Place text_encoder/text_encoder-NVFP4.gguf in ComfyUI/models/clip/
Place vae/vae.safetensors in ComfyUI/models/vae/
Load the checkpoint normally — ComfyUI-GGUF will automatically detect the NVFP4 format

Quantization Details

Property	Value
Quantization (Diffusion)	NVFP4 (NVIDIA FP4, 4-bit)
Quantization (Text Encoder)	NVFP4 (NVIDIA FP4, 4-bit)
GGML Quant Type	40 (NVFP4)
GGUF File Type	39 (LLAMA_FTYPE_MOSTLY_NVFP4)
Bits Per Weight (Text Encoder)	~4.66
Backend	CUDA 13.2, compute 12.0
llama.cpp Commit	`dd7cad7`
Source Precision	F8_E4M3 → F16 → NVFP4

Conversion Process

The original FP8 merged checkpoint was:

Split into three components (diffusion, text encoder, VAE)
Each component's FP8 tensors manually converted to F16 via streaming binary I/O with row-chunked FP8→F16 conversion to avoid OOM on large tensors
Written as F16 GGUF using gguf Python library (v0.19.0)
Quantized to NVFP4 using llama-quantize.exe (commit dd7cad7, CUDA 13.2)
Architecture binary-patched in output GGUF to match expected ComfyUI-GGUF target

Text Encoder Notes

The text encoder (Qwen2.5-VL-7B, 7.75 GB F8_E4M3 → 3.84 GB NVFP4) uses the qwen2 GGUF architecture
Input GGUF was written with both qwen2.* and llama.* metadata keys for quantizer compatibility, then patched to qwen2 after quantization
Row-chunked FP8→F16 conversion (256 rows at a time) prevents OOM on the 545M-element embedding table
Embedding weights are tied (lm_head shares weights with token_embd) — standard for Qwen2 models

Available Quantizations

Currently only NVFP4 is provided. Other quantizations (Q4_K_M, Q5_K_M, Q6_K, Q8_0) are possible using the same toolchain.

Notes

This is a NSFW checkpoint. Use at your own discretion.
The text encoder is now also NVFP4 GGUF, saving ~4 GB (was 7.75 GB as safetensors) while remaining compatible with ComfyUI-GGUF.
The VAE remains in safetensors format (BF16) for maximum compatibility.
NVFP4 is a lossy quantization — expect minor quality degradation compared to the original FP8 model.
Requires NVIDIA Blackwell GPU (RTX 50-series) with CUDA 13.0+ for NVFP4 compute support.