Qwen3-ASR-1.7B-NVFP4

Qwen3-ASR-1.7B - NVFP4A16 Quantized

Model Overview

This repository contains a pre-quantized NVFP4A16 derivative of Qwen/Qwen3-ASR-1.7B.

It is a weight-only post-training quantization of the model's text decoder, stored in the compressed-tensors format. The audio tower and other non-decoder components remain in BF16.

Qwen3-ASR-1.7B supports:

• 30 languages
• 22 Chinese dialects
• language identification and speech recognition in a single model family

This derivative is intended for users who want a lighter, faster ASR checkpoint while remaining close to the original model.

Quantization Summary

• Base model: Qwen/Qwen3-ASR-1.7B
• Quantization type: NVFP4A16, weight-only
• Storage format: compressed-tensors
• Quantized scope: text decoder layers (thinker.model.layers.*)
• Kept in BF16: audio encoder/tower, embeddings, and output head

Approximate repository size: 2.5 GB

Practical Gains

Compared with the BF16 base model in the saved local benchmark set, this checkpoint delivered:

• 2.5x lower median latency (59.8% lower)
• 1.5x lower observed GPU resident memory (31.9% lower)

Evaluation

Benchmark setup

The benchmark numbers below come from local measurements collected on an NVIDIA DGX Spark system.

• Corpus size: 100 fixed multilingual clips
• Repeats: 5 passes over the corpus (500 requests total)
• Language distribution: Japanese 24, English 20, French 14, Chinese 14, German 6, Spanish 6, Italian 6, Korean 6, Russian 2, Portuguese 2
• Latency metrics: p50 / p95 latency and a normalized latency slope against reference character count
• Quality metric: character error rate (CER)

Results

Relative to the BF16 baseline, this checkpoint measured:

• 2.49x lower p50 latency
• 3.13x lower marginal decode cost
• 31.9% lower observed GPU resident memory
• no median quality loss on the saved benchmark set

Repository Contents

• model.safetensors - quantized checkpoint
• config.json - model config including quantization metadata
• generation_config.json - generation defaults
• preprocessor_config.json - preprocessing metadata
• tokenizer_config.json, vocab.json, merges.txt - tokenizer assets
• chat_template.json - auxiliary upstream chat template asset
• recipe.yaml - quantization scope metadata

Compatibility Notes

This repository contains weights only. Use software that already supports:

• the upstream Qwen3-ASR architecture
• pre-quantized compressed-tensors checkpoints
• NVFP4-style decoder quantization

If you are evaluating software stacks, a recent vLLM release with audio-model and compressed-tensors support is a reasonable starting point.

Sources

• Original model: Qwen/Qwen3-ASR-1.7B
• Upstream technical report: Qwen3-ASR Technical Report (arXiv:2601.21337)
• Quantization tooling: llm-compressor
• Quantized checkpoint format: compressed-tensors

License Reminder

This repository is a derivative of an upstream model released under Apache-2.0.

Please review and comply with the original model license, notices, and any upstream usage guidance before redistributing, serving, or incorporating this checkpoint into another product or repository.