Downloads · 30 days
14
18% of all-time downloads
Opt.Gear-1B-qat-int4-g32-unquantized
OptGear/Opt.Gear-1B-qat-int4-g32-unquantized
Opt.Gear-1B-qat-int4-g32-unquantized is a text generation model from OptGear. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-sa-4.0.
<img width="1000px" src="./OptAIxHuggingFace04.png"
All-time downloads
79
Public
Parameters
842M
1.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.7 GB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- cc-by-nc-sa-4.0
- Model type
- gear
- Access
- Public
- Created
- Aug 2, 2026
- Updated
- Aug 4, 2026
- SHA
- 5e5bd2c6
Try a prompt
Base models
Model card
From the Hugging Face model README
Opt.Gear-1B-QAT
<!-- TODO: 로고 이미지 URL 교체 --> <img width="1000px" src="./OptAIxHuggingFace04.png">
[!Note] This repository contains the Quantization-Aware Training (QAT) checkpoint of Opt.Gear-1B: master weights adapted to the deployment INT4 grid, together with the learned group-wise quantization scales.
The deployment format is W4A16 (group-size-32 symmetric INT4 weights, 16-bit activations). The checkpoint performs best with its learned QAT scales, and also re-quantizes stably with other INT4 PTQ methods (RTN, AWQ, GPTQ).
For the full-precision model, see Opt.Gear-1B. <!-- TODO: 링크 확인 -->
Making a model small is not enough for real devices. Beyond parameter count, the memory occupied by weights, memory bandwidth, arithmetic precision, and the kernels supported by the accelerator determine final inference performance. The question is: how much of the original quality survives when the model is squeezed into 4 bits?
Opt.Gear-1B-QAT adapts the trained model directly to the deployment INT4 grid via Quantization-Aware Training. On six English/Korean benchmarks, the QAT model loses only ~1.0 point on average from the Base model — under the exact group-size-32 symmetric W4A16 grid used at deployment time.
QAT Highlights
- Group-size-32 symmetric INT4 weight quantization with a W4A16 deployment format
- Hessian-based scale initialization: group-wise INT4 scales initialized by a layer-wise output reconstruction objective measured on calibration activations — not simple max-based statistics
- Learned Step Size Quantization (LSQ): weights and step sizes jointly optimized under the full language-modeling objective, with straight-through gradient estimation
- No quantizer mismatch: the same group-wise INT4 grid is used during initialization, QAT, and final weight conversion, so the learned quantization structure survives deployment export
- Quantization-friendly weights: the checkpoint re-quantizes stably with RTN/AWQ/GPTQ, giving deployment pipelines freedom in quantizer choice
For more details, please refer to our tech report and blog post. <!-- TODO: 링크 연결 -->
Quantization Configuration
| Item | Setting |
|---|---|
| Weight precision | INT4 |
| Activation precision | INT16 |
| Quantization format | W4A16 |
| Weight quantization | Symmetric (zero-point fixed at 0) |
| Group size | 32 |
| Scale initialization | Hessian-based output reconstruction |
| Scale optimization | Learned Step Size Quantization |
| QAT length | 1,000 steps <!-- TODO: 리포트에 미기재 — 수치 확인 --> |
The scale for each quantization group is initialized as
Δ* = argmin_Δ ‖ WX − Q_Δ(W)X ‖²_F
where W is the full-precision weight matrix, X contains the calibration inputs, and Q_Δ(W) denotes the weights quantized with scale Δ. This objective accounts for the relative importance of different input directions instead of treating all weight perturbations equally. Starting from this initialization, QAT jointly refines the weights and the LSQ step sizes under the global training objective, while the forward pass consistently emulates the final W4A16 deployment format.
Benchmark Results
Base, QAT, and PTQ results of Opt.Gear-1B and Gemma3-1B on six English/Korean benchmarks. RTN, AWQ, and GPTQ rows re-quantize the master weights of the QAT checkpoint with each PTQ method; the QAT row evaluates the final QAT quantizer including learned scales. All results are obtained using the LM Evaluation Harness.
<div style="max-width:1000px;margin:0 auto;padding:16px 0"> <table style="border-collapse:collapse;font-size:15px"> <thead><tr> <th style="padding:10px 7px;text-align:left;font-weight:600;border-bottom:2px solid #5b4bcf;color:#5b4bcf">Method</th> <th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">MMLU</th> <th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">HellaSwag</th> <th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">WinoGrande</th> <th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">KMMLU</th> <th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">KoBEST</th> <th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">HAERAE</th> <th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">Avg.</th> <th style="padding:10px 7px;text-align:center;font-weight:500;border-bottom:2px solid #5b4bcf;color:#5b4bcf">Δ</th> </tr></thead> <tbody> <tr><td colspan="9" style="padding:8px 12px;font-weight:600;color:#5b4bcf;border-bottom:1px solid rgba(91,75,207,0.2);background:rgba(91,75,207,0.08)">Opt.Gear-1B</td></tr> <tr> <td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">Base</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">43.2</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">56.0</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">58.2</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">36.0</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">60.3</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">44.2</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">49.7</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">–</td> </tr> <tr> <td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">QAT</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">41.3</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">54.0</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">58.3</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">35.0</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">60.4</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">42.7</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);font-weight:600">48.6</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">-1.0</td> </tr> <tr> <td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">RTN</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">39.8</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">53.9</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">56.9</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">33.8</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">59.8</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">39.9</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">47.4</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">-2.3</td> </tr> <tr> <td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">AWQ</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">39.5</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">52.6</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">56.5</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">32.7</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">59.6</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">38.2</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">46.5</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">-3.1</td> </tr> <tr> <td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">GPTQ</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">39.6</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">53.5</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">57.1</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">34.4</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">59.6</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">41.0</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">47.5</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">-2.1</td> </tr> <tr><td colspan="9" style="padding:8px 12px;font-weight:600;color:#5b4bcf;border-bottom:1px solid rgba(91,75,207,0.2);background:rgba(91,75,207,0.08)">Gemma3-1B (reference)</td></tr> <tr> <td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">Base</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">39.8</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">60.0</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">58.7</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">30.7</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">59.5</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">35.3</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">47.3</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">–</td> </tr> <tr> <td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">QAT</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">35.8</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">51.4</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">55.0</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">26.9</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">56.2</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">31.2</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">42.8</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">-4.6</td> </tr> <tr> <td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">RTN</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">32.5</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">48.5</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">53.8</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">21.2</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">55.1</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">23.6</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">39.1</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">-8.2</td> </tr> <tr> <td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">AWQ</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">34.3</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">50.5</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">53.7</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">23.3</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">56.4</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">29.7</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">41.3</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">-6.0</td> </tr> <tr> <td style="padding:7px;padding-left:20px;border-bottom:1px solid rgba(128,128,128,0.15)">GPTQ</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">34.9</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">49.6</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">54.9</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">23.3</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">55.1</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">27.1</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">40.8</td> <td style="padding:7px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15)">-6.5</td> </tr> </tbody> </table> <p style="margin-top:12px;font-size:11px;opacity:0.7"> * <code>Avg.</code> is the simple average of the six benchmark scores; <code>Δ</code> is the difference from each model's Base average.<br> * PTQ results of Llama3.2-1B, Qwen3-1.7B, and EXAONE4-1B are available in the tech report. </p> </div>Re-quantization Stability
The QAT checkpoint performs best with its learned scales, and the QAT-trained weights themselves respond stably to other INT4 quantizers:
| Model | Quantizer | Avg | Δ vs QAT |
|---|---|---|---|
| Opt.Gear-1B-QAT | Learned QAT scale | 48.6 | – |
| RTN | 47.4 | -1.3 | |
| AWQ | 46.5 | -2.1 | |
| GPTQ | 47.5 | -1.1 | |
| Gemma3-1B-QAT | Learned QAT scale | 42.8 | – |
| RTN | 39.1 | -3.6 | |
| AWQ | 41.3 | -1.4 | |
| GPTQ | 40.8 | -1.9 |
Under identical RTN re-quantization, Opt.Gear-1B loses only 1.3 points on average, versus 3.6 points for Gemma3-1B. The Opt.Gear-1B-QAT checkpoint is not merely a set of learned scale parameters — the weights themselves have been adapted to be quantization-friendly, giving deployment pipelines freedom in quantizer choice.
Quickstart
[!Important] This checkpoint is intended for INT4 deployment. For best quality, export with the included learned group-wise scales (the same group-size-32 symmetric W4A16 grid used during QAT). Re-quantizing with RTN/AWQ/GPTQ is supported and remains stable, at a small cost in accuracy (see table above).
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "OptGear/Opt.Gear-1B-qat-int4-g32-unquantized"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")
messages = [
{"role": "user", "content": "온디바이스 AI가 왜 중요한지 설명해줘."},
]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
Best Practices
-
Use the learned scales when possible: The learned QAT quantizer gives the best accuracy (Avg. 48.6). Switch to RTN/GPTQ only when your deployment toolchain requires it — the accuracy cost is small but non-zero.
-
Keep the grid consistent: The checkpoint was trained against a group-size-32 symmetric INT4 grid with 16-bit activations (W4A16). Exporting to a different group size or asymmetric grid discards the learned quantization structure.
-
Termination tokens: Same as the full-precision model — the Instruction model terminates with
<EOT>, the Base model with<EOS>. -
Model capabilities: Task capability follows Opt.Gear-1B — Korean-English bilingual understanding and instruction following; not intended for code generation. See the base model card for full benchmark results and architecture details.
Citation
If you find our work helpful, feel free to give us a cite.
@misc{optgear2026,
title = {{Opt-Gear} Technical Report},
author = {{Opt.Gear Team}},
year = {2026},
url = {https://arxiv.org/abs/2608.01034}
}
Correspondence: [email protected] · Hugging Face: huggingface.co/OptGear
- Task
- Text Generation
- Library
- transformers
- Type
- gear
- License
- cc-by-nc-sa-4.0
- Languages
- en, ko
- Created
- Aug 2, 2026
- Updated
- Aug 4, 2026