Qwen3-ASR-1.7B-FT-8bit

Qwen3-ASR-1.7B v12 Breeze rank32 CER best360 — MLX 8-bit and GGUF Q8_0

This release is a personal Chinese ASR fine-tune of Qwen/Qwen3-ASR-1.7B. Both published files are complete merged models from v12 Breeze ordinary LoRA rank32/alpha64, CER best360. Training ran for 900 optimizer steps; step360 was selected by the earliest minimum validation CER before test evaluation. Validation-loss best660 and final900 were also retained privately. The published model is CER best360, not best660 or final900; this does not change the experiment’s original default best-loss selection rule.

The root model supports MLX on Apple Silicon. The GGUF file supports transcribe.cpp architecture qwen3_asr; it is not a generic text-only llama.cpp GGUF. Only MLX 8-bit and GGUF Q8_0 weights are published. No BF16/16-bit weights, recordings, reference transcripts, individual predictions or recording/session IDs are included.

Files and quantization

MLX uses affine quantization, 8 bits and group size64 (197 quantized layers, 1101 stored tensors). Some non-quantized tensors remain floating point; dtype describes their dtype rather than changing the quantization bits. GGUF is converted directly from the merged BF16 source, never by dequantizing MLX: 709 tensors comprise 344 Q8_0 linear/embedding tensors and 365 F32 convolution, normalization, bias and frontend tensors. Both exports passed source, tensor, metadata and loading checks. SHA256SUMS covers the release files; evaluation.json contains aggregate results and provenance.

Comparison with the original Qwen model

The unmodified Qwen/Qwen3-ASR-1.7B without an adapter and this fine-tuned adapter were evaluated on the same 60 validation and 80 test recordings and human-reviewed references. Both reference evaluations used PyTorch BF16/CUDA, the same base revision, automatic language detection, greedy decoding, batch size1, max_new_tokens512 and no history context.

On this fixed personal evaluation, the relative CER reduction is 55.96% on validation and 62.50% on test. These are relative reductions in character error rate, not sentence accuracy or percentage-point gains. Quantized results below use different runtimes and language settings; their differences are not solely a quantization effect. This small, repeatedly used evaluation does not establish broad superiority over the base model.

Evaluation of these exact published weights

Both quantized models were independently loaded and evaluated on all 60 human-reviewed validation recordings and 80 human-reviewed test recordings, with identical IDs and references. Character errors and formatting metrics were independently recomputed from the predictions before publication.

CER strips the ASR language prefix, applies NFKC and lowercase, removes Unicode punctuation and whitespace, and retains symbols. It is character edit distance divided by reference character count. Chinese number words are not automatically equated to Arabic digits. 1 minus CER is not sentence or recording accuracy.

Raw CER applies only ASR prefix removal and NFKC, retaining case, punctuation, symbols and spaces. It includes lexical errors, so it is not a pure formatting error rate. Whole-transcript exact matches use the same raw normalization. Runtime, decoding settings and numerical precision differ between PyTorch, MLX and GGUF; differences cannot be attributed solely to quantization.

Training parameters and data

• Base revision: 7278e1e70fe206f11671096ffdd38061171dd6e5; fresh adapter from the original 1.7B model, no resumed fine-tune.
• Ordinary LoRA: rank32, alpha64, dropout0.05, use_dora=false, use_rslora=false. Decoder attention q/k/v/o and MLP gate/up/down projections: 196 modules and 34,865,152 trainable parameters.
• Microbatch1, gradient accumulation8, seed42, 900 optimizer steps. Warmup50 to 1e-4, linear decay to 1e-5 at step600, then hold through step900.
• Loss: sum of causal cross-entropy across the actual accumulation window divided by shifted valid supervised token count; masked prefix/padding excluded, EOS and ASR markers included, partial windows use their actual count. No additional division by gradient accumulation. Tokens are not CER characters.
• Human validation CER every10 steps including step0 (91 points); CER best360 is the earliest minimum; validation loss independently selected best660. Both selections were frozen before test. Training including periodic evaluation/saves took 64.69 minutes; the complete workflow took 69.16 minutes.
• Training data: 2648 recordings = 2511 real (887 human-reviewed and 1624 automatic unverified labels) + 137 synthetic unverified recordings. The synthetic data include numerical quantities, dates, units, and 12 examples with digit1 pronounced “幺”. All 137 synthetic recordings were regenerated with Breeze TTS (breeze-tts-2) and the user-selected minicpm-ref reference voice. No Qwen CustomVoice recordings remain in this training snapshot. Generated audio and labels are not comprehensively human-verified; machine checks and one specifically reviewed example do not imply full manual verification. The service did not expose an effective generation seed. This change alters both TTS model and voice, so it is not a voice-only causal comparison.
• Fixed validation60/test80 are real human-reviewed recordings. The split was sampled from the full eligible pool, including eligible historical evaluation sessions. Current training excludes the new validation/test sessions, with ID/session/PCM isolation. Labels, including human edits, receive surface normalization for case, Chinese–English spacing and explicit numerical expressions; original review records remain private.

Usage

MLX / Apple Silicon

Full evaluation used mlx-audio 0.3.1, mlx 0.32.2, and transformers 4.57.6. See aggregate metadata for exact versions. Use a compatible tested environment; a fresh unconstrained dependency resolution is not guaranteed to reproduce these versions.

from mlx_audio.stt import load
model = load("xchern/Qwen3-ASR-1.7B-FT-8bit")
result = model.generate("audio.wav", language="Chinese", max_tokens=512,
                        temperature=0.0, verbose=False)
print(result.text)

GGUF / transcribe.cpp

Conversion and full evaluation used commit 5d66f60257a75e9bdbddc72dc0887b32e9d4841a, variant qwen3-asr-1.7b, Metal backend, greedy decoding, Chinese, n_ctx2048, six threads and no timestamps/history context. The GGUF contains its tokenizer and frontend metadata.

hf download xchern/Qwen3-ASR-1.7B-FT-8bit gguf/Qwen3-ASR-1.7B-v12-best360-Q8_0.gguf --local-dir ./model
build/bin/transcribe-cli -m ./model/gguf/Qwen3-ASR-1.7B-v12-best360-Q8_0.gguf --backend metal -l zh --timestamps none --n-ctx 2048 --threads 6 audio.wav

The tested GGUF input is 16kHz mono WAV. Transcription processes a complete recording; this release is not validated for streaming.

Limitations and intended use

This is a personal Chinese dictation experiment, not an official Qwen benchmark model. Evaluation is small, private and repeatedly used across experiments. These evaluation recordings entered some older historical adapters, so results must not be described as independent blind tests of those older models. This v12 adapter was trained fresh with the current evaluation sessions excluded, but repeated development still limits generalization claims.

The accuracy of unreviewed automatic training labels has not been measured. Synthetic examples and aggregate CER do not establish that real short-number, year, unit, pi or “幺” recognition problems are solved. Transcripts may omit, insert or substitute content; review outputs when correctness matters. The model is intended for authorized transcription, and does not establish robust performance for other speakers, languages or recording conditions.

These files retain rank32. Checkpoint selection used validation, not test. BF16 source and earlier exports are retained privately for reproducibility; only the two 8-bit formats above are released.

License

Apache-2.0; see LICENSE and the base model license.