keural-sft2

Keural-SFT2-14.83B (SFT Epoch 2 — 29,112 steps)

Keural is a bilingual Korean–English Mixture-of-Experts language model trained entirely from scratch — no base model was used. This is the SFT epoch 2 checkpoint at step 29,112, continuing supervised fine-tuning from the epoch 1 checkpoint (18,000 steps) on the same 710K ChatML dataset.

Model Details

Full Training Pipeline

SFT Epoch 2 Training Details

SFT Dataset

Chat Format (ChatML)

This model uses ChatML format. Always include a system prompt for best results.

<|im_start|>system
You are a helpful bilingual Korean-English assistant. Always respond in the same language as the user.<|im_end|>
<|im_start|>user
안녕하세요! 오늘 날씨가 어때요?<|im_end|>
<|im_start|>assistant

The model generates until it produces <|im_end|> (token ID 131073).

The chat template in tokenizer_config.json automatically injects a default system prompt if you don't provide one, so bilingual behavior works out of the box with apply_chat_template.

How to Use

With transformers

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "mkd-hossain/keural-sft2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": (
            "You are a helpful bilingual Korean-English assistant. "
            "Always respond in the same language as the user's message."
        )
    },
    {"role": "user", "content": "파이썬에서 리스트를 정렬하는 방법을 알려주세요."},
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9,
        top_k=50,
        repetition_penalty=1.1,
        no_repeat_ngram_size=8,
        do_sample=True,
        eos_token_id=131073,
    )

response = tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=False)
response = response.split("<|im_end|>")[0].strip()
print(response)

With vLLM (recommended for serving)

pip install vllm

python -m vllm.entrypoints.openai.api_server \
    --model mkd-hossain/keural-sft2 \
    --tokenizer mkd-hossain/keural-sft2 \
    --dtype bfloat16 \
    --max-model-len 4096 \
    --tensor-parallel-size 1

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")

response = client.chat.completions.create(
    model="mkd-hossain/keural-sft2",
    messages=[
        {"role": "system", "content": "You are a helpful bilingual assistant. Respond in the same language as the user."},
        {"role": "user", "content": "What is the capital of South Korea?"},
    ],
    max_tokens=512,
    temperature=0.7,
)
print(response.choices[0].message.content)

Special Tokens

Critical: Always set eos_token_id=131073 when generating. Do not use eos_token_id=2.

Recommended Generation Settings

# Conversational / creative
{
    "max_new_tokens": 512,
    "temperature": 0.7,
    "top_p": 0.9,
    "top_k": 50,
    "repetition_penalty": 1.1,
    "no_repeat_ngram_size": 8,
    "do_sample": True,
    "eos_token_id": 131073,
}

# Factual / deterministic
{
    "max_new_tokens": 512,
    "temperature": 0.1,
    "repetition_penalty": 1.1,
    "do_sample": False,
    "eos_token_id": 131073,
}

Checkpoint Comparison

Limitations

• Maximum context is 4,096 tokens.
• The pretraining corpus is Korean-dominant — always include a system prompt for correct bilingual behavior.
• Not safety-aligned — do not deploy in production without additional safety fine-tuning.
• This is an intermediate checkpoint. SFT epoch 3 on a 2.35M sample merged dataset is in progress.

License

Apache 2.0