K-MT-7B-v0.1

K-MT-7B v0.1

Follows Korean address-term and speech-level instructions without losing the multilingual translation the base model already did (in-domain evidence; retention verified across 28 FLORES directions).

LoRA fine-tune of tencent/Hy-MT2-7B on failure-driven targeted Korean data plus capability replay, merged back into the base as a single set of weights. The goal was not a higher general translation score — it was to gain instruction-following behaviour in Korean while keeping the base model's existing multilingual ability.

🇰🇷 한국어 카드: MODEL_CARD.ko.md

⚠️ A public out-of-domain benchmark for the Korean-specific capability is still being built. The Korean numbers in §2 are in-domain (failure-mined) — the eval set and the training data come from the same mining pipeline. We are building a separate benchmark for the phenomena that are genuinely hard in Korean (speech level, semantic roles, business pragmatics, ellipsis, titles, terminology, code-switching, long-context consistency). When it lands, that table becomes the headline table of this card.

---

0. How to use it — the Korean constraint is set in the prompt

The behaviour does not appear on its own. You state the speech level and address term you want. This is the exact format used in training and evaluation.

from transformers import AutoModelForCausalLM, AutoTokenizer

mid = "ThakiCloud/K-MT-7B-v0.1"
tok = AutoTokenizer.from_pretrained(mid, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(mid, dtype="bfloat16",
                                             trust_remote_code=True, device_map="auto")

prompt = ("Please translate the following text into Korean. Note that the translation "
          "style must strictly conform to [종결어미는 합니다체로 통일 · 청자 호칭은 "
          "'본부장님'을(를) 사용]:\n"
          "Tell him the deployment finished and ask if we should roll back.")

text = tok.apply_chat_template([{"role": "user", "content": prompt}],
                               tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(text, return_tensors="pt").to(model.device),
                     do_sample=False, max_new_tokens=256)      # greedy — same as evaluation
print(tok.decode(out[0], skip_special_tokens=True))

Constraint syntax — inside square brackets, separated by ·. These four were evaluated:

Give no instruction and it translates normally — that is the intended behaviour. We measured separately that it does not insert address terms when none were requested.

⚠️ The hunyuan_v1_dense architecture needs trust_remote_code=True, and the chat template is the repo's chat_template.jinja (same as the base). All evaluation ran through transformers.

Intended use and scope

• Built for: EN↔KO business translation where speech level, address term and relationship have to match an instruction. It is a drop-in replacement for the base model — general translation quality is tied with it (§1).
• Not verified: per-direction quality for language pairs without Korean — retention was checked as a 28-direction FLORES aggregate, and no per-direction superiority is claimed. The domain is business documents; literary, medical and legal text are untested. This is a translation model, not a chat model.

What actually changes

Every pair below is a real evaluation output — left is the base model, right is this model, same prompt, same greedy decoding. Of the 461 constraint units, 346 are cases where the base failed the instruction and K-MT passed; zero go the other way.

(a) Address term — the instruction says who is being spoken to, and the base ignores it

The base translations are not wrong — they simply left out the address term they were told to use. That is the whole gap this model closes.

(b) Speech level — the ending form has to match the level that was asked for

The base put the address term in, then ended the sentence in 해요체 when 합니다체 was requested. Speech level is a separate axis from address term, and both are checked.

(c) Relationship — the same person is addressed differently depending on the direction

Note the form: 최 매니저, not 최 매니저님. Superior-to-subordinate drops the honorific suffix, and the model follows the form it was given rather than politening it.

(d) Terminology — a term marked do-not-translate stays in English

In a Korean finance or infra team, CAPEX, egress and accrual are the words people actually say. Translating them into 자본지출 or 외부 전송 is not an error in the dictionary sense — it just is not what the reader uses, and a glossary is how you say so.

(e) No instruction, no insertion — the negative control

⚠️ A model trained to insert address terms could learn to insert them everywhere. It does not — where no address term was requested, none appears. This axis was measured separately (n=296), and is the reason the §2 number is a compliance rate rather than an insertion rate.

1. General translation quality — a guardrail, not a win

This table exists to show that specializing for Korean did not break anything else. Point estimates are corpus-level.

⛔ Decoding differs by row. The primary numbers use uniform greedy (temp 0.0 · top_p 1.0 · top_k −1 · seed 0). LMT-60-8B is also reported at beam 5 because that is its official recommendation — measuring a model only under a weaker setting than it recommends would be unfair to it. Do not read the beam row and the greedy rows as one series.

Paired-item bootstrap (B=1000, seed 7), K-MT − base:

⛔ Do not read the WMT24++ COMET +0.0020 as an improvement. The interval excludes zero, but it is 0.2 COMET points, and across nine comparisons against the base one crossing is what you expect by chance at α=0.05. This table says "not broken", nothing more.

⚠️ These confidence intervals are computed against the base only. Differences versus LMT-60-8B and TranslateGemma-4B are point estimates with no interval — this table does not support "K-MT beats those two", and we do not claim it.

2. Korean constraint compliance — in-domain

⚠️ This is a failure-mined benchmark. The eval set was mined from the base model's own failures, and the training data came from the same mining. Do not weigh it like §1.

"Constraint compliance" = the requested address term matches exactly and the speech level is not violated (deterministic checker, n=461). Semantic error is an independent judge's semantic-role verdict. FLORES Δ is the mean over 28 translation directions (n=28,336).

Replay is the mechanism, and the source of the replay text matters. Remove replay and Korean goes up but FLORES drops 0.43. Replace the replay source text with base-generated sentences and constraint compliance drops 1.66pp (interval excludes zero). Real human sentences as the source, with the base model's own outputs as the targets, is the combination that holds both axes.

3. Measured but not claimed

IFMTBench Korean (n=66) — only the rule-based axes, scored on this checkpoint. Read against the positive control (reference text submitted as the response): glossary 25/34 → 0.93 of ceiling, structured 9/16 → 0.75 (base: 0.81 · 0.75). Paired intervals include zero on both axes (glossary Δ+0.059 [0.000, 0.147] · structured Δ−0.063 [−0.188, 0.000]) — one item is worth 2.9pp and 6.3pp, so no ordering can be claimed.

⛔ The style and background axes were not scored for this checkpoint. They need an LLM judge, and using a self-hosted judge instead of the official default makes those numbers incomparable with published scores for other models, so they are omitted. Note also that this benchmark was built by the base model's authors, and its Korean items are not EN→KO-sourced.

Broad Korean SFT — a control trained broadly on public parallel corpora reached constraint compliance of 2.60 · 0.65 · 1.74% across three seeds, below the untrained base (23.9%). Pouring in more broad Korean data is not the answer. ⛔ We do not write "targeted wins at equal budget" — dataset tokens, consumed tokens and optimizer updates cannot all be matched at once, so no pair is fully matched.

4. Limitations

• §1's public benchmarks do not contain what this model was built to fix. WMT24++ and FLORES have no items demanding address-term or speech-level instructions. The "tie" there is a fact about the benchmarks, not about the model. We have never demonstrated instruction-following superiority on public evidence — the OOD benchmark in §0 is meant to fill that gap.
• No humans in the evaluation loop (machine-only): deterministic checkers plus a judge. ⛔ The two judges do not receive the same instructions — J2 gets the rubric, J1 gets the rubric plus seven calibration rules. Their κ (0.45–0.48) is therefore not an inter-annotator agreement, and J1∧J2 is used only as a conservative lower bound.
• The Korean eval sets are n=461 (constraints) and n=66 (IFMTBench Korean).
• The release is bf16 merged weights. Merging rounds away part of the LoRA delta — 33 of 288 free-generation items (11.5%) diverge lexically from the adapter path. The constraint verdict did not change on any of the 461 units, and the §2 numbers were measured on this released file, not on the adapter. ⚠️ Semantic error is 3.04% (adapter) vs 3.69% (merged) — 14 → 17 of 461. Paired-item bootstrap (B=1000, seed 7) puts that at +0.65pp [−0.65, +1.95], and McNemar over the 9 discordant pairs (6 merged-only, 3 adapter-only) gives p=0.51. The merged file is not measurably worse. ⛔ That is not the same as "no difference" — 9 discordant pairs cannot rule out a small one.
• The base lineage reports training on OPUS/ParaCrawl/WMT-family corpora. WMT24++ extends WMT24, so both the base and this model may benefit on that benchmark — contamination in our favour.

5. Data provenance

• Training: failure-driven targeted Korean data (self-generated, dual-judged) plus capability replay.
• Replay source text is the Tatoeba CC0 export (CC0-1.0); the targets are the base model's own greedy outputs. ⛔ No FLORES text was used in training — FLORES is evaluation-only. Contamination check between training text and eval probes: 0 hits, 0 contained.
• ⛔ Rows derived from non-redistributable licensed corpora are rejected by the data builder (ContaminationError); none are in this model.
• Evaluation: WMT24++ (google/wmt24pp, Apache-2.0) · FLORES-200 (CC-BY-SA-4.0) · IFMTBench (tencent/IFMTBench, CC-BY-4.0).

⚠️ We make no determination about whether model weights are a derivative work of training data. What is stated here is only which corpora were used under which licence, and what was not used.

6. Reproduction

Where each §2 row comes from: the released merged weights are the round-6 judgment; adapter seed 1 is round 4 (replay-source ablation, arm R1-s1), seeds 2·3 are round 5, the no-replay variant is arm B1, and the base-generated-replay variant is arm R2-s1. All §1 competitor numbers were measured by us on the same engine (transformers) and the same benchmark files — none are quoted from elsewhere.

The prompt format and greedy settings in §0 are exactly what produced the §2 numbers.

⚠️ Per-item scores and the eval set are not released. The Korean eval set (n=461) was mined from the base model's failures; releasing it would let the next model be trained against it. So §2 is not directly reproducible by a reader — what is reproducible is §1 (WMT24++, FLORES), which should give the same values under the decoding settings above. The forthcoming public OOD benchmark is planned to ship with its items.