Downloads · 30 days
135
0% of all-time downloads
toxzak/Qwen3.6-27B-Q1_K-GGUF
Qwen3.6-27B-Q1_K-GGUF is a machine learning model from toxzak. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
Downloads · 30 days
135
0% of all-time downloads
All-time downloads
70.3K
Public
Repo size
3.4 GB
Likes
1
Public
Click a slice to open those files.
.gguf-q1_k3.4 GB · 100%
From the Hugging Face model README
https://github.com/toxzak-svg/fabq-rc
FABQ-RC (Fisher-Adaptive Binary Quantization with Residual Codebooks) is a 1-bit quantization method for large language models that adapts per layer rather than using a fixed blocksize. This quantization of Qwen3.6-27B achieves ~1.18 bits per parameter while maintaining quality through four key innovations.
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen3.6-27B |
| Quantization Method | FABQ-RC |
| Format | GGUF (v3) |
| Bits per Parameter | ~1.18 bpw |
| Precision Allocation | Top 5% channels → int4, 95% → binary ±1 |
| Blocksize | Adaptive per-layer {64, 128, 256, 512} |
| Calibration Dataset | C4 (2048 samples, seq_len=32) |
FABQ-RC combines four innovations:
Fisher-Weighted Channel Importance — Uses Fisher Information (expected gradient²) to determine which channels actually matter for the loss. This is more directly relevant than Hessian (curvature) or magnitude alone.
Mixed-Precision Core Allocation — Top 5% Fisher channels → int4 (preserve accuracy). Bottom 95% → binary ±1 (maximum compression).
Adaptive Blocksize — Each layer gets its own optimal blocksize from {64, 128, 256, 512}, chosen by minimizing Fisher-weighted reconstruction error. Homogeneous layers use larger blocks; heterogeneous layers use smaller ones.
Residual Codebook — After binary quantization, systematic residuals remain. FABQ-RC clusters these using 4 tiered k-means codebooks (64 centroids each, Fisher quartile-based), enabling non-linear correction that beats BiLLM's linear approximation.
FP32 Weights
│
▼
Stage 1: Fisher-Weighted Channel Importance
│ Compute per-channel Fisher Information
│ Sort channels by expected loss impact
▼
Stage 2: Mixed-Precision Allocation
│ Top 5% channels → int4 (preserve accuracy)
│ Bottom 95% channels → binary ±1 (max compression)
▼
Stage 3: Adaptive Blocksize Selection
│ Per-layer sweep {64, 128, 256, 512}
│ Pick blocksize minimizing Fisher-weighted reconstruction error
▼
Stage 4: Residual Codebook Clustering
│ 4 tiered codebooks × 64 centroids (Fisher quartile-based)
│ 4-bit indices per block (16 centroids per layer cluster)
▼
FABQ-RC GGUF