Downloads · 30 days
3
2% of all-time downloads
qwen2.5-1.5b-invoice-awq
Msduck/qwen2.5-1.5b-invoice-awq
qwen2.5-1.5b-invoice-awq is a machine learning model from Msduck. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
- Quantized Model (AWQ 4-Bit): Msduck/qwen2.5-1.5b-invoice-awq - Base Model: Qwen/Qwen2.5-1.5B-Instruct - Adapter Model: Msduck/qwen2.5-1.5b-invoice-lora - Merged FP16 Baseline: Msduck/qwen2.5-1.5b-invoice-merged-fp16
All-time downloads
199
Public
Parameters
1.5B
2.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.1 GB · 99%
Parameter types
How the weights are stored.
I321.3B · 85%
Model card
From the Hugging Face model README
Model Card — Invoice/PO Structured Extraction (Qwen2.5-1.5B-Instruct AWQ 4-Bit)
Model Details
- Quantized Model (AWQ 4-Bit):
Msduck/qwen2.5-1.5b-invoice-awq - Base Model:
Qwen/Qwen2.5-1.5B-Instruct - Adapter Model:
Msduck/qwen2.5-1.5b-invoice-lora - Merged FP16 Baseline:
Msduck/qwen2.5-1.5b-invoice-merged-fp16 - Primary Task: Structured extraction of enterprise invoice and purchase order metadata into rigid JSON schema (supporting English & Hindi documents).
- Quantization Method: Activation-aware Weight Quantization (AWQ)
w_bit: 4q_group_size: 128zero_point: Trueversion: GEMM
1. Inference & Throughput Benchmarks (vLLM Engine)
Benchmarked on fixed document extraction prompts using vLLM v0.26.0 OpenAI API Server hosted on an NVIDIA Tesla T4 GPU (16GB VRAM) (--gpu-memory-utilization 0.85 --max-model-len 4096).
| Benchmark Metric | AWQ 4-Bit Model (qwen2.5-1.5b-invoice-awq) | FP16 Merged Baseline (qwen2.5-1.5b-invoice-merged-fp16) | Relative Advantage / Gain |
|---|---|---|---|
| Time to First Token (TTFT) | 37.66 ms | 36.31 ms | ~3.7% prefill variance (negligible) |
| Time Per Token (TPOT / ITL) | 8.48 ms/token | 15.43 ms/token | ~45.0% lower decoding latency |
| Generation Throughput | 119.56 tokens/sec | 65.65 tokens/sec | +82.1% throughput gain (1.82x speedup) |
| Total Request Latency | 664.95 ms | 1193.90 ms | ~44.3% faster total response time |
| Generated Output Tokens | 75 tokens | 76 tokens | Standardized structured JSON output |
| Peak GPU VRAM Usage | 13,695 MB (13.37 GB) | 13,331 MB (13.02 GB) | Pre-allocated KV Cache pool at 85% utilization |
Kernel Execution Architecture
- Unfused PyTorch Native AWQ: Plain PyTorch execution without fused dequantization kernels yields ~4.35 tok/s due to python loop & memory layout overheads.
- Fused vLLM AWQ/Marlin C++ Kernels: Integration of vLLM fused dequantization kernels cuts memory bandwidth traffic by ~50% during the memory-bound autoregressive decoding stage, yielding 119.56 tokens/second.
2. Accuracy & Schema Validation (Golden Test Set, n=135)
| Metric | Base Model | Fine-Tuned FP16 | AWQ 4-Bit Quantized | Threshold | CI Status |
|---|---|---|---|---|---|
| JSON Validity Rate | 88.15% | 100.0% | 100.0% | $\ge$ 95.0% | PASS |
| Field F1 Score | 0.7091 | 0.9951 | 0.9951 | $\ge$ 0.90 | PASS |
| Field Precision | 0.7123 | 0.9951 | 0.9951 | — | Diagnostic |
| Field Recall | 0.7074 | 0.9951 | 0.9951 | — | Diagnostic |
| Field Exact Match | 0.7074 | 0.9951 | 0.9951 | — | Diagnostic |
| Forgetting Retention | 45.82% | 49.82% | 49.82% | $\ge$ 85.0% | FAIL (Metric Discrepancy) |
Note: AWQ 4-Bit quantization incurs <0.1% accuracy degradation compared to FP16, maintaining complete structural JSON syntax compliance.
3. Known Limitations & Recommendations
- Engine Dependency: To achieve full 1.82x throughput acceleration, deploy with inference engines supporting fused AWQ/Marlin C++ kernels (e.g., vLLM, TensorRT-LLM, or SGLang).
- Exact-String Match Metric: Evaluation relies on normalized lowercase exact string matching. Minor string formatting differences (e.g.,
"1,000.00"vs"1000.0") should be sanitized downstream. - Catastrophic Forgetting Proxy: The retention score uses an automated n-gram overlap metric; qualitative human testing confirms domain reasoning remains preserved.
- Type
- qwen2
- Created
- Aug 9, 2026
- Updated
- Aug 11, 2026