Downloads · 30 days
0
hanseungwook/Llama-3.2-1B-Compressed-Joint-GSM8K-Aug-NL
Llama-3.2-1B-Compressed-Joint-GSM8K-Aug-NL is a text generation model from hanseungwook. Use it when you need the model to write or continue text. It is set up for pytorch. The card lists the license as llama3.2.
Downloads · 30 days
0
Access
Public
Updated Sep 17, 2026
Repo size
2.9 GB
Likes
0
Public
Click a slice to open those files.
.bin2.9 GB · 100%
From the Hugging Face model README
Built with Llama.
The best student-validation checkpoint from hyundong_gsm8k_aug_nl_lr3e-4_bs256_ep10_s11_codi_sentence_drop.
Training completed all 10 epochs; the selected checkpoint is epoch 6 / step 6,414.
Its recorded held-out validation accuracy is 29.92% (155/518), using question-only student inference,
greedy decoding, and at most 64 generated tokens. This is a validation result, not an official GSM8K test score.
Training run.
This is a full native Compressed Joint state dictionary. Use the included inference helper and exact
code/ bundle; it is not an AutoModelForCausalLM export. Loading requires CUDA and access to the
pinned meta-llama/Llama-3.2-1B-Instruct base model/tokenizer, or an existing local cache.
python -m pip install 'huggingface_hub==0.32.4'
huggingface-cli login
python -c 'from huggingface_hub import snapshot_download; snapshot_download("hanseungwook/Llama-3.2-1B-Compressed-Joint-GSM8K-Aug-NL", local_dir="cj-gsm8k-aug-nl")'
python -m pip install -r cj-gsm8k-aug-nl/code/compressed_joint/requirements.txt
python cj-gsm8k-aug-nl/run_inference.py \
--checkpoint-dir cj-gsm8k-aug-nl \
--question 'Janet has 3 apples and buys 4 more. How many apples does she have?' \
--max-new-tokens 64
The helper preserves the original strict source, runtime, and parameter checks. Required runtime:
torch==2.7.1, transformers==4.52.4, peft==0.15.2. Pass plain question text.
| Setting | Value |
|---|---|
| Backbone | Llama 3.2 1B Instruct, revision 9213176726f574b556790deb65791e0c5aa438b6 |
| Training set | zen-E/GSM8k-Aug-NL; 273,632 accepted examples |
| CoT processing | Original CODI period-space sentence splitting, dropping the final segment |
| Token preprocessing limit | 256 |
| Validation cohort | 518 original GSM8K training questions absent from the pinned augmentation set |
| Slots / refinement iterations | 6 / 6 |
| Compressor | Legacy architecture; width 512, 2 layers, 8 heads, FFN 2048 |
| LoRA | Rank 128, alpha 32, dropout 0.1; tied trainable BoT/EoT rows |
| Learning rate / global batch | 0.0003 / 256 |
| Optimizer | AdamW; weight decay 0.1; maximum gradient norm 2 |
| Schedule | Cosine, warmup ratio 0.03, 10 epochs |
| Precision / seed | BF16 / 11 |
| Latent matching | Smooth L1 normalized by teacher residual RMS; scale 1, weight 1 |
| Teacher residual statistics | 12,800 EMA examples; 25,600 warmup examples; relative floor 0.5 |
| Explicit / encoded-answer / predicted-answer CE weights | 1 / 1 / 1 |
| Predicted answer decoder | Detached |
| Answer KL | Weight 1; temperature 2; teacher CE maximum 1 |
The unchanged native manifest defines the complete architecture and objective. run_configuration.json records the recipe, and validation_results.json records all ten epoch results and checkpoint selection. The package includes full model parameters and buffers, with no optimizer, scheduler, or RNG resume state.
Source commit recorded by training: bc65228eda97e29d8eceda0f795d4104432d806b. Exact method source SHA256: f6f2bd755192d7d59ef1eaef2443209b734e5086f93753bb1627dead59218ea6.
The bundled snapshot is authoritative for loading. File sizes and hashes appear in release_manifest.json.
Packaging checks verify the selected files and source; the recorded validation score was not rerun during packaging.
The model and derivatives are subject to the Llama 3.2 Community License and
Acceptable Use Policy. Attribution is retained in NOTICE.