Downloads · 30 days
15
18% of all-time downloads
Qwen3.5-4B-NQ-rebal
JD97/Qwen3.5-4B-NQ-rebal
Qwen3.5-4B-NQ-rebal is a machine learning model from JD97. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
JD97/Qwen3.5-4B-NQ 기반. KL forward-only sensitivity 분석으로 정밀도 재배분: - outproj (GDN 출력): W4 - W8 — KL 가성비 1위(작고 민감, L0/outproj outlier). +0.25GB. - inproja/b (decay/gate): BF16 - W4 — KL상 둔감(논문 dt 무감각 일치). latency 회수. - 순…
All-time downloads
84
Public
Parameters
5.3B
5.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.2 GB · 100%
Parameter types
How the weights are stored.
I323.9B · 74%
Base models
Model card
From the Hugging Face model README
Qwen3.5-4B-NQ-rebal (KL-lens rebalanced, MTP removed)
JD97/Qwen3.5-4B-NQ 기반. KL forward-only sensitivity 분석으로 정밀도 재배분:
- out_proj (GDN 출력): W4 -> W8 — KL 가성비 1위(작고 민감, L0/out_proj outlier). +0.25GB.
- in_proj_a/b (decay/gate): BF16 -> W4 — KL상 둔감(논문 dt 무감각 일치). latency 회수.
- 순효과: ~latency 중립, GPQA razor-thin(NQ full=0.6212, gate 0.630) 회복 시도.
- MTP head 제거 (DFlash draft 사용).
구성 (fused 모듈, vLLM 서빙용)
- group_0 W4 g32: gate_up_proj, down_proj, in_proj_qkvz, in_proj_ba (a/b)
- group_1 W8 g32: out_proj
- group_2 W8 g32: lm_head
- BF16: full-attn(q/k/v/o), vision
- RTN(data-free). 서빙 검증 완료(mixed-precision은 fused-name 정규식 필수, "Linear" catch-all 금지 — scheme 충돌).
서빙
W8 lm_head용 docker/lmhead-quant.patch 필요(NQ와 동일). docker/serve.py 참조.
- Type
- qwen3_5
- License
- apache-2.0
- Created
- Jun 18, 2026
- Updated
- Jun 18, 2026