LFM2.5-8B-A1B-KO-CPT-FULL

LFM2.5-8B-A1B-KO-CPT-FULL

Full-parameter Korean continued-pretraining project for LiquidAI/LFM2.5-8B-A1B.

This model is intended to make LFM2.5 stronger at Korean legal, finance, wiki-style knowledge, and terminal/tool-use behavior while preserving the base model's general English and instruction-following ability.

• GitHub: https://github.com/gyunggyung/LFM25-KO-CPT
• SFT follow-up GitHub: https://github.com/gyunggyung/LFM25-KO-SFT
• SFT follow-up model: https://huggingface.co/LLM-OS-Models/LFM2.5-8B-A1B-KO-SFT
• Agentic follow-up model: https://huggingface.co/LLM-OS-Models/LFM2.5-8B-A1B-KO-Agentic-SFT
• Public CPT data:
  • LFM-style full raw: https://huggingface.co/datasets/LLM-OS-Models/LFM2.5-KO-CPT-Full-LFMStyle-Raw-20260627
  • LFM-style source shards: https://huggingface.co/datasets/LLM-OS-Models/LFM2.5-KO-CPT-Full-LFMStyle-Shards-20260627
  • Raw mix before LFM wrapping: https://huggingface.co/datasets/LLM-OS-Models/LFM2.5-KO-CPT-Full-Raw-Mix-20260627
• Public SFT and Agentic data are indexed on the SFT model card: https://huggingface.co/LLM-OS-Models/LFM2.5-8B-A1B-KO-SFT.

Public CPT dataset releases:

Status: full CPT completed on 2026-06-28. Weights are prepared from the verified checkpoint-10196 final-step checkpoint and uploaded to Hugging Face. vLLM evaluation shows strong gains on instruction-following, GSM8K, BoolQ, ARC, and several Korean knowledge subjects, but also regressions on Korean hard MCQA, MMLU-ProX-lite-ko, and some STEM/legal/accounting slices.

Performance Snapshot

All numbers below are vLLM/lm-eval base-vs-CPT comparisons against LiquidAI/LFM2.5-8B-A1B. Higher is better.

For a detailed Korean explanation of the training data, preprocessing, full-parameter CPT setup, observed gains, regressions, and follow-up lesson, see the GitHub note: CPT 데이터, 학습 방식, 효과 정리.

한국어 요약: 이 CPT는 한국어 위키, 금융/회계, 법령/판례/행정규칙, 법률 QA/RAG, 변시형 JSON 정답, terminal/tool-use trace를 LFM-style로 전처리해 약 6.493B 추정 토큰으로 1 epoch full-parameter CPT한 모델입니다. 결과적으로 한국어 지식과 일부 일반 벤치마크는 상승했지만, KMMLU hard, MMLU-ProX Lite KO, 법률/STEM식 다지선다와 짧은 정확답 추출은 하락했습니다.

Confirmed Gains

Regressions To Fix

Interpretation: the CPT run successfully injects Korean-domain knowledge and preserves or improves several general benchmarks, but it is not a finished Korean instruction model. The next post-training stage should target Korean MCQA reliability, option-label extraction, STEM hard questions, legal/accounting reasoning, and preservation of the current IFEval/GSM8K/BoolQ gains.

Likely failure mode: many regressions are not simple "Korean got worse" failures. They cluster around multiple-choice answering, exact answer extraction, option-label discipline, and hard STEM/legal/accounting formats. Open-ended Korean knowledge slices and instruction-following often improve, while Korean MCQA and parser-sensitive exact-match tasks need targeted remediation.

Contents

• English
• Performance Snapshot
• Quick Start
• Colab Example
• Training Configuration
• Data Mix
• Legal Data Attribution
• Korean
• 한국어 사용법
• 한국어 학습 설정
• Evaluation Plan

English

LFM2.5-8B-A1B-KO-CPT-FULL is a full fine-tuned Korean CPT checkpoint, not a LoRA adapter. The training objective is text completion over a Korean-heavy corpus, with LFM chat-template formatting applied to instruction, RAG, and tool-use examples.

Training code, source manifests, dataset cards, and runbooks are published at https://github.com/gyunggyung/LFM25-KO-CPT. The supervised fine-tuning follow-up is tracked at https://github.com/gyunggyung/LFM25-KO-SFT.

Target strengths:

• Korean legal document understanding and legal RAG-style answering
• Korean finance explanations and finance-domain terminology
• Korean wiki/general knowledge prose
• Korean instruction-following
• Terminal/tool-use style structured assistant behavior

Quick Start

The examples below use the full model repository.

Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "system", "content": "You are a precise Korean assistant."},
    {"role": "user", "content": "대한민국 민법상 계약 해제와 해지의 차이를 간단히 설명해줘."},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.6,
        top_p=0.9,
        do_sample=True,
    )

print(tokenizer.decode(output[0], skip_special_tokens=False))

vLLM

vllm serve LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL \
  --trust-remote-code \
  --dtype bfloat16 \
  --max-model-len 8192 \
  --tensor-parallel-size 8

OpenAI-compatible request:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL",
    messages=[
        {"role": "system", "content": "You are a precise Korean assistant."},
        {"role": "user", "content": "한국 기준금리 인상이 은행 순이자마진에 미치는 영향을 설명해줘."},
    ],
    temperature=0.5,
    max_tokens=512,
)

print(response.choices[0].message.content)

Colab Example

Use this after model weights are uploaded. For typical Colab GPUs, start with 4-bit loading to avoid OOM.

!pip install -U "transformers>=4.44" accelerate bitsandbytes sentencepiece huggingface_hub

import torch
from huggingface_hub import login
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

# Optional for gated/private models:
# login("hf_xxx")

model_id = "LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16 if torch.cuda.is_available() and torch.cuda.is_bf16_supported() else torch.float16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "system", "content": "너는 한국어로 정확하고 간결하게 답하는 어시스턴트다."},
    {"role": "user", "content": "한국어로 주택임대차보호법의 대항력 요건을 설명해줘."},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.5,
        top_p=0.9,
        do_sample=True,
        eos_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=False))

If you have an A100/H100/H200 runtime, bf16 loading can be used instead of 4-bit:

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

Prompt Format

The model follows the LFM2 chat-template style. Use the tokenizer chat template when possible. The CPT corpus preserves these special tokens for chat and tool-use records:

• <|startoftext|>
• <|im_start|>
• <|im_end|>
• roles: system, user, assistant, tool

References:

• LFM text generation: https://docs.liquid.ai/lfm/key-concepts/text-generation-and-prompting
• LFM chat template: https://docs.liquid.ai/lfm/key-concepts/chat-template
• LFM tool use: https://docs.liquid.ai/lfm/key-concepts/tool-use

Training Configuration

• Base model: LiquidAI/LFM2.5-8B-A1B
• Method: full-parameter continued pretraining, not LoRA
• Framework: Unsloth + TRL SFTTrainer
• Hardware target: 8x NVIDIA H200
• Context length: 8192
• Precision: bf16 when supported
• Optimizer: adamw_8bit
• GPUs: 8
• Per-device batch size: 2
• Gradient accumulation steps: 4
• Effective batch: 64 sequences/update
• Maximum tokens/update: 524,288
• Learning rate: 2e-5
• Schedule: 1 epoch over the prepared full corpus
• max_steps: -1
• Checkpoint interval: 1,000 steps
• Checkpoint retention: 4 latest checkpoints plus final model

Completed run:

• Estimated tokens: 6,492,697,020
• Raw estimated steps before packing: 12,384
• Actual packed trainer steps: 10,196
• Train runtime: about 9h 38m
• Train samples/sec: 18.81
• Train steps/sec: 0.294
• Final logged train loss: 0.712
• Final checkpoint source: checkpoint-10196
• Final model integrity check: model.safetensors opens successfully with 2,302 tensors

Note: the distributed torchrun process reached step 10196/10196 and wrote checkpoint-10196. A SIGSEGV occurred during the extra post-train trainer.save_model(final_full) write, leaving the initial final_full/model.safetensors incomplete. The published final_full was rebuilt from the verified checkpoint-10196 inference files.

Data Mix

Prepared full mix:

/home/work/.data/lfm2_ko_cpt/datasets/ko_cpt_mix_full_lfmstyle_20260627.jsonl

Statistics:

• Rows after global deduplication: 4,622,971
• Characters: 11,581,567,658
• Estimated tokens: 6,492,697,020
• Raw estimated training steps: 12,384 at effective batch 64 and sequence length 8192
• Actual packed trainer steps: 10,196

Per-source rows:

Raw Korean wiki/legal/finance documents are kept as plain completion text for CPT. Instruction, legal RAG, and terminal/tool-use examples are converted to LFM ChatML-style text.

Legal Data Attribution

Legal-domain data is attributed to the public Legalize-KR ecosystem and related Korean legal source corpora used in the local CPT mix.

Legalize-KR links:

• Organization: https://github.com/legalize-kr
• Korean statutes repository: https://github.com/legalize-kr/legalize-kr
• Korean court precedent repository: https://github.com/legalize-kr/precedent-kr
• Korean administrative rules repository: https://github.com/legalize-kr/admrule-kr
• Korean local ordinance repository: https://github.com/legalize-kr/ordinance-kr
• Data collection/conversion pipeline: https://github.com/legalize-kr/legalize-pipeline
• Legalize-KR website: https://legalize.kr
• Original public legal source: https://www.law.go.kr

The Legalize-KR organization describes its project as converting Korean statutes, precedents, administrative rules, and local ordinances into Markdown and Git history. Its README states that source data is obtained from the National Law Information Center OpenAPI and transformed into Git repositories. Long-term reproducibility should pin a snapshot or release where possible because Legalize-KR notes that Git history can be reconstructed when parsing and normalization rules improve.

Recommended attribution format:

• Statutes: cite legalize-kr/legalize-kr, the Markdown path such as kr/{statute-name}/{statute-type}.md, and stable metadata fields such as 법령ID, 법령MST, promulgation date, effective date, and the 출처 URL from law.go.kr.
• Precedents: cite legalize-kr/precedent-kr, the Markdown path such as {case-type}/{court-level}/{court}_{decision-date}_{case-number}.md, and stable identifiers such as 판례일련번호, court name, decision date, and case number.
• Administrative rules: cite legalize-kr/admrule-kr, the Markdown path such as {agency-path}/{rule-type}/{rule-name}/본문.md, plus rule serial number or issuing number when available.
• Local ordinances: cite legalize-kr/ordinance-kr, the Markdown path such as {province}/{city-or-office}/{ordinance-type}/{ordinance-name}/본문.md, plus 자치법규ID, 자치법규일련번호, promulgation date, promulgation number, and the 출처 URL.
• Avoid using only Git commit hashes as long-term identifiers because Legalize-KR warns that repository history may be reconstructed after parser or normalization improvements.
• License note from the Legalize-KR READMEs: original legal text is Korean government public work; repository structure and metadata are MIT where specified by the repository.

Local legal sources included in this CPT run:

• korean_legal_raw_full_20260523
• korean_legal_tasks_full_20260524
• korean_admrule_precedent_raw_full_20260524
• ko_legal_source_agent_sft_20260621
• ko_legal_rag_agent_sft_round15_v2
• current_law_bar_json_answer_sft_20260621

Korean

LFM2.5-8B-A1B-KO-CPT-FULL은 LoRA 어댑터가 아니라 full-parameter CPT 모델입니다. 목표는 LFM2.5-8B-A1B에 한국어 법률, 금융, 위키 지식과 터미널/도구 사용 스타일을 계속 사전학습으로 이식하는 것입니다.

목표 성능:

• 한국어 법률 문서 이해와 법률 RAG 답변
• 한국어 금융 설명과 금융 용어 처리
• 한국어 위키/일반 지식 문체
• 한국어 instruction following
• 터미널/도구 호출형 assistant 동작 보존

한국어 사용법

가중치 업로드 후 아래처럼 사용할 수 있습니다.

Transformers 사용

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "system", "content": "너는 한국어로 정확하고 간결하게 답하는 어시스턴트다."},
    {"role": "user", "content": "상법상 이사의 충실의무를 실무 관점에서 설명해줘."},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.5,
        top_p=0.9,
        do_sample=True,
    )

print(tokenizer.decode(output[0], skip_special_tokens=False))

vLLM 사용

vllm serve LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL \
  --trust-remote-code \
  --dtype bfloat16 \
  --max-model-len 8192 \
  --tensor-parallel-size 8

요청 예시:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL",
    messages=[
        {"role": "system", "content": "너는 한국어로 정확하고 간결하게 답하는 어시스턴트다."},
        {"role": "user", "content": "부동산 임대차 계약에서 보증금 반환 분쟁의 핵심 쟁점을 정리해줘."},
    ],
    temperature=0.5,
    max_tokens=512,
)

print(response.choices[0].message.content)

Colab 사용 예시

일반 Colab GPU에서는 VRAM 부족을 피하려고 4-bit 로딩부터 쓰는 것이 좋습니다.

!pip install -U "transformers>=4.44" accelerate bitsandbytes sentencepiece huggingface_hub

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

model_id = "LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16 if torch.cuda.is_available() and torch.cuda.is_bf16_supported() else torch.float16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "system", "content": "너는 한국어로 정확하고 간결하게 답하는 어시스턴트다."},
    {"role": "user", "content": "한국 금융시장에서 기준금리와 채권 가격의 관계를 설명해줘."},
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.5,
        top_p=0.9,
        do_sample=True,
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=False))

권장 생성 설정

• 법률/금융 설명: temperature=0.3-0.6, top_p=0.8-0.95
• 일반 한국어 답변: temperature=0.5-0.8, top_p=0.9
• 긴 문서 요약: max_new_tokens=1024 이상
• 도구 사용/구조화 출력: 낮은 temperature 권장

한국어 학습 설정

• 베이스 모델: LiquidAI/LFM2.5-8B-A1B
• 방식: full-parameter CPT, LoRA 아님
• 하드웨어: NVIDIA H200 8장
• 컨텍스트 길이: 8192
• GPU당 batch size: 2
• gradient accumulation: 4
• effective batch: 64 sequences/update
• update당 최대 token: 524,288
• learning rate: 2e-5
• epoch: 1
• max_steps: -1
• 저장 간격: 1,000 steps
• checkpoint 보존: 최신 4개와 final model

학습 규모:

• 전체 row: 4,622,971
• 추정 token: 6.49B
• raw 예상 step: 12,384
• 실제 packed trainer step: 10,196
• 실제 train runtime: 약 9시간 38분
• 최종 train loss: 0.712
• 최종 weight 출처: 무결성 검사를 통과한 checkpoint-10196

한국어 법률 데이터 출처

법률 도메인 데이터 출처는 Legalize-KR 생태계와 로컬 한국 법률 corpus를 명시한다.

• Legalize-KR 조직: https://github.com/legalize-kr
• 대한민국 법령: https://github.com/legalize-kr/legalize-kr
• 대한민국 판례: https://github.com/legalize-kr/precedent-kr
• 대한민국 행정규칙: https://github.com/legalize-kr/admrule-kr
• 대한민국 자치법규: https://github.com/legalize-kr/ordinance-kr
• 수집/변환 파이프라인: https://github.com/legalize-kr/legalize-pipeline
• Legalize-KR 웹사이트: https://legalize.kr
• 원천 공공 법령 출처: https://www.law.go.kr

Legalize-KR은 법령/판례/행정규칙/자치법규를 Markdown과 Git 이력으로 관리하는 공개 프로젝트다. 조직 README 기준 원천 데이터는 국가법령정보센터 OpenAPI에서 가져오며, 파싱과 정규화 규칙이 개선되면 Git 이력이 재구성될 수 있으므로 장기 재현에는 snapshot 또는 release 고정이 필요하다.

출처 표기 방식:

• 법령: legalize-kr/legalize-kr 저장소, kr/{법령명}/{법령구분}.md 경로, 법령ID, 법령MST, 공포일자, 시행일자, 출처 URL을 함께 적는다.
• 판례: legalize-kr/precedent-kr 저장소, {사건종류}/{법원등급}/{법원명}_{선고일자}_{사건번호}.md 경로, 판례일련번호, 법원명, 선고일자, 사건번호를 함께 적는다.
• 행정규칙: legalize-kr/admrule-kr 저장소, {기관경로}/{행정규칙종류}/{행정규칙명}/본문.md 경로, 행정규칙일련번호 또는 발령번호를 함께 적는다.
• 자치법규: legalize-kr/ordinance-kr 저장소, {광역}/{기초 또는 _본청 또는 _교육청}/{자치법규종류}/{자치법규명}/본문.md 경로, 자치법규ID, 자치법규일련번호, 공포일자, 공포번호, 출처 URL을 함께 적는다.
• commit hash만 장기 출처로 쓰지 않는다. Legalize-KR README는 파서/정규화 개선 시 저장소 history가 재구성될 수 있다고 안내한다.
• Legalize-KR README 기준 원문은 대한민국 정부 공공저작물이고, 저장소 구조와 메타데이터는 저장소별 MIT 표기를 따른다.

Evaluation Plan

Current vLLM Smoke Check

This is not a benchmark score. It verifies that both the base model and the CPT model load and generate with vLLM tensor parallelism.

• Date: 2026-06-28
• vLLM environment: local .vllm-lfm-cu12, vLLM 0.19.1, Torch 2.10.0+cu128
• Tensor parallel size: 8
• Max model length: 8192
• Base model smoke: passed model load and generation
• CPT model smoke: passed model load and generation
• Smoke result path: /home/work/.data/lfm2_ko_cpt/evals/20260628_1052_smoke_clean_vllm_smoke
• CPT checks passed: Korean legal, Korean finance, tool-call format, English instruction smoke
• CPT wiki smoke note: the answer was relevant, but the simple keyword check expected the literal word 요약, so that specific automatic check is false.

Current vLLM Benchmark Results

Evaluation uses EleutherAI lm-evaluation-harness with vLLM tensor parallelism. The IFEval run below is the full 541-prompt public task, not a limited smoke sample.

• Date: 2026-06-28
• Task: ifeval
• Runner: lm_eval==0.4.11, vllm==0.19.1, Torch 2.10.0+cu128
• Tensor parallel size: 8
• Max model length: 8192
• Result path: /home/work/.data/lfm2_ko_cpt/evals/20260628_022743_ifeval_full_vllm_vllm_matrix

GSM8K 5-shot LIMIT=200 limited regression check:

Global MMLU Korean LIMIT=500 limited check:

Note: GSM8K and Global MMLU Korean above are limited runs and should be treated as early regression checks, not final public benchmark scores. Additional vLLM evaluations are running with one task per GPU.

Additional vLLM checks:

Latest Global MMLU Korean subject sweep:

The limited checks are useful for regression tracking, but they should not be read as final leaderboard-quality numbers. The model improves strongly on several reasoning and instruction-following checks, while law-focused MMLU-Pro and MMLU-ProX-lite-ko need targeted remediation.

KMMLU direct exact-match runs currently show near-zero base scores and small non-zero CPT scores. Treat those as prompt/extraction diagnostics rather than quality benchmarks until the direct-answer parser is fixed.

Recommended Next Post-Training

The next stage should be targeted post-training, not another broad CPT-only pass. Liquid's official LFM2.5-8B-A1B reporting emphasizes instruction following, math, tool use, and agentic workflows such as IFEval, IFBench, Multi-IF, MATH500, AIME, BFCL, and Tau2. The Japanese LFM2.5-1.2B-JP model card reports language-specialized axes such as JMMLU-ProX, JMMLU, J-MIFEval, J-GSM8K, J-MATH500, JHumanEval+, and J-BFCLv3. The Korean follow-up should mirror those axes with Korean data and Korean eval gates.

Priority plan:

1. Korean MCQA remediation SFT: KMMLU, KMMLU-hard, MMLU-ProX-lite-ko style, legal/accounting/finance questions, exact option-label outputs, and short rationales.
2. STEM/legal/accounting remediation SFT: target current regressions in high-school statistics, astronomy, chemistry, formal logic, jurisprudence, professional accounting, and MMLU-Pro law.
3. Korean instruction-following SFT: Ko-IFEval-style constraints, Korean formatting, uncertainty, refusal, and multi-condition prompts.
4. Tool and agent SFT: Korean BFCL-style tool schemas, terminal/tool-call traces, JSON validity, and multi-turn task completion.
5. Preference tuning: DPO/ORPO/KTO on current failure pairs. Reward correct option extraction, concise Korean, valid tools, and uncertainty over hallucination.
6. Small preservation mix: keep GSM8K, ARC, BoolQ, IFEval, and high-gain Global MMLU KO examples in the mix so post-training does not erase current gains.

Concrete gates for the next model:

• kmmlu_hard: improve from 0.1720 to at least 0.2300.
• kmmlu_hard_stem: improve from 0.1564 to at least 0.2100.
• mmlu_prox_lite_ko: recover above the base score 0.2585 and target 0.3000.
• mmlu_pro_law: recover above the base score 0.1840 and target 0.2300.
• Preserve gsm8k flexible exact match at or above 0.5701.
• Preserve boolq accuracy at or above 0.7902.
• Preserve leaderboard_instruction_following prompt loose at or above 0.3346.

Public Benchmark Plan

Primary public Korean benchmarks:

• KMMLU: https://huggingface.co/datasets/HAERAE-HUB/KMMLU
• KMMLU-Pro: https://huggingface.co/datasets/LGAI-EXAONE/KMMLU-Pro
• KMMLU-Redux: https://huggingface.co/datasets/LGAI-EXAONE/KMMLU-Redux
• Ko-IFEval: https://huggingface.co/datasets/davidkim205/ko-ifeval

Secondary checks:

• Korean legal RAG holdout
• Korean finance explanation holdout
• Korean wiki QA/summarization holdout
• Terminal/tool-use smoke tests

Benchmark results will be added after vLLM base-vs-CPT evaluation.