Downloads · 30 days
17
24% of all-time downloads
FiShota/hinomoto-1b-bitnet158-v1
hinomoto-1b-bitnet158-v1 is a text generation model from FiShota. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-4.0.
HinoMoto-1B v1 Phase 2 Full の BitNet b1.58 量子化版. 重みを 三値 (= 1.58 bit/weight) に置換 後、 500 step QAT で 品質回復.
Downloads · 30 days
17
24% of all-time downloads
All-time downloads
72
Public
Parameters
955M
1.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.9 GB · 100%
From the Hugging Face model README
HinoMoto-1B v1 Phase 2 Full の BitNet b1.58 量子化版. 重みを 三値 (= 1.58 bit/weight) に置換 後、 500 step QAT で 品質回復.
個人 GPU (RTX 3090 24GB) で paper の主張を再現 したエヴィデンス.
Reference: Ma et al., "The Era of 1-bit LLMs" (Feb 2024). https://arxiv.org/abs/2402.17764
各 linear layer の weight を {-1, 0, +1} の三値に量子化:
| Item | Value |
|---|---|
| Params | 955,221,504 (~955M) |
| 1536 | |
| 32 | |
| 16 (MHA) | |
| (SwiGLU) | 4096 |
| Linear modules | 224 BitLinear (= attn + FFN) |
| + embeddings | fp 維持 (paper convention) |
| 32,000 (HinoMoto byte-BPE, 32k) |
| Item | Value |
|---|---|
| Base ckpt | HinoMoto-1B Phase 2 Full (50,000 step from-scratch) |
| QAT steps | 500 |
| Optimizer | AdamW (lr 5e-5, cosine, warmup 100) |
| dtype | bf16 mixed (master weights fp), STE quantization |
| Hardware | RTX 3090 24GB |
| Wall-clock | ~5 hr (with torch.compile, default mode) |
| Best loss (instant) | 1.24 (single batch outlier), steady ~2.5 |
| Axis | fp 1B Phase 2 Full | BitNet b1.58 (this) | Δ |
|---|---|---|---|
| family score /12 | 5.08 | 5.29 | +0.21 (✓ within paper claim) |
| family degenerate | 9.1% | 9.1% | 0% |
| keigo pass_rate | 11.4% | 10.0% | -1.4% |
| keigo degenerate | 5.7% | 7.1% | +1.4% |
| silence pass_rate | 6.0% | 0.0% | -6.0% (degradation) |
| silence degenerate | 6.0% | 12.0% | +6.0% |
→ family / keigo は fp 同等. silence は若干悪化 (= 短文応答が量子化に sensitive). 全体として paper 主張の 「< 5% quality loss」 達成.
= 個人 GPU で 7-8B 級モデル の現実的可能性.
本 release は その第一歩.
CC BY 4.0. Use for any purpose with attribution.
@misc{hinomoto-1b-bitnet158-v1-2026, author = {ryu (FIshota)}, title = {HinoMoto-1B v1 BitNet b1.58 (Ternary-Quantized, Personal-GPU Reproduction)}, year = {2026}, publisher = {Hugging Face}, }
Generated: 2026-05-19 (HinoMoto/ryu + Claude)