canary-qwen-2.5b-GGUF

canary-qwen-2.5b GGUF

GGUF conversions of nvidia/canary-qwen-2.5b for use with CrispASR.

Model

SALM (Speech-Augmented Language Model): 32-layer FastConformer encoder (d=1024) + linear projection (1024 to 2048) + Qwen3-1.7B LLM decoder with merged LoRA. English ASR, up to ~40s audio.

• Parameters: 2.5B
• Encoder: FastConformer (from canary-1b-flash), 128 mel bins, 16 kHz
• LLM: Qwen3-1.7B (28 layers, GQA 16/8, SwiGLU)
• LoRA: merged at conversion time (r=128, alpha=256)
• Language: English only
• License: CC-BY-4.0

Files

Encoder and projection weights are kept at source precision (F16/F32) in all quantized variants to avoid conformer drift.

Usage

# With CrispASR CLI
crispasr --backend canary-qwen -m canary-qwen-2.5b-q8_0.gguf -f audio.wav

# Auto-download
crispasr --backend canary-qwen -m auto --auto-download -f audio.wav

Conversion

python models/convert-canary-qwen-to-gguf.py \
  --input nvidia/canary-qwen-2.5b \
  --output canary-qwen-2.5b-f16.gguf

crispasr-quantize canary-qwen-2.5b-f16.gguf canary-qwen-2.5b-q8_0.gguf q8_0
crispasr-quantize canary-qwen-2.5b-f16.gguf canary-qwen-2.5b-q4_k.gguf q4_k

Performance (Kaggle P100)

On the JFK sample (11s audio): "And so my fellow Americans ask not what your country can do for you ask what you can do for your country"

Provenance and EU AI Act Art. 53 note

• Upstream model: nvidia/canary-qwen-2.5b — published by nvidia.
• Upstream licence: cc-by-4.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.