Downloads · 30 days
3
9% of all-time downloads
puchuneko/GLM-4-32B-0414-Korean-Culture-QLoRA
GLM-4-32B-0414-Korean-Culture-QLoRA is a text generation model from puchuneko. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as mit.
This is a small pilot adapter, published for transparency — not recommended for production use.
Downloads · 30 days
3
9% of all-time downloads
All-time downloads
35
Public
Repo size
252 MB
Likes
0
Public
Click a slice to open those files.
.safetensors232 MB · 92%
From the Hugging Face model README
This is a small pilot adapter, published for transparency — not recommended for production use.
A QLoRA (LoRA r=16) adapter that fine-tunes zai-org/GLM-4-32B-0414 on 1,750 rows of Korean data, targeting the Korean-culture knowledge gap of Chinese-origin base models. Part of an openly documented attempt to climb the Korean K-AI leaderboard honestly — full recipe, scripts, and failure log: github.com/jwson-automation/korean-glm-merge.
| Slice | Rows | Source |
|---|---|---|
| Culture/knowledge Q&A | 1,093 | Generated from Korean Wikipedia (220 verified articles) by Qwen2.5-32B-Instruct (Apache-2.0), self-verified at temperature 0, closed-book format |
| Human-written instructions | 357 | CohereLabs/aya_dataset Korean subset (Apache-2.0) |
| Spoken dialogue | 300 | 국립국어원 모두의 말뭉치 spoken corpus (subsampled; the corpus itself is redistribution-restricted and is not included here — only weights) |
Decontamination: all training rows were filtered against a 49k-item blocklist built from KMMLU (all subjects), CLIcK, HAE-RAE Bench 1.1, and KoBEST (3-layer filter: 13-word shingles, short-item substring, 5-word Jaccard). Note: this catches verbatim/near-verbatim reuse only, not semantic paraphrases.
--limit 100 per subtask, bf16, 0-shot)⚠️ These are relative comparison numbers on a 100-item-per-subtask sample, not full benchmark scores.
| base | +this adapter | Δ | |
|---|---|---|---|
| CLIcK (aggregate) | 67.0 | 68.0 | +1.0 (within noise) |
| CLIcK culture | 70.9 | 73.7 | +2.8 |
| CLIcK language | 58.0 | 55.0 | −3.0 |
| HAE-RAE Bench | 72.2 | 70.8 | −1.4 |
The interesting pattern: all 8 culture subtasks moved up or stayed flat (tradition 86→90, society 74→78, kpop 75.6→87.8, politics 79.8→82.1, …) while all 3 language subtasks moved down — i.e. the model learned roughly what the data covered (culture) and paid a small tax where it had no data (grammar/usage). Aggregate is a wash.
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("zai-org/GLM-4-32B-0414", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "puchuneko/GLM-4-32B-0414-Korean-Culture-QLoRA")
tokenizer = AutoTokenizer.from_pretrained("puchuneko/GLM-4-32B-0414-Korean-Culture-QLoRA")
No benchmark training data was used. No SOTA claims. v0.2 plan: scale culture Q&A 5–10k with a cross-model verifier, add a dedicated grammar/usage track, and test continued pretraining on the source corpus before SFT.