ggc-budget-12b

Hunminai-1.0-12b

Hunminai-1.0 is a Korean-aligned language model based on Google's Gemma-3 architecture. To improve performance on Korean natural language tasks, the model was fine-tuned on a corpus of 100k instruction examples using Supervised Fine-Tuning (SFT) followed by Direct Preference Optimization (DPO). This approach enables the model to better align with user intents in Korean and enhances its applicability to downstream tasks such as dialogue generation, question answering, and long-form text generation.

Model Details

• Base Model: google/gemma-3-12b-it
• Base Model Release Date: March 12, 2025
• Context Length: 128k
• License: gemma
• Model Type: Text Generation
• Fine-Tuning Techniques: Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO)

Usage

Gemma 3 is supported starting from version 4.50.0 of the Transformers library.

To update to the latest version, run the following command:

$ pip install -U transformers

Install the required package and run the example code below to load the Hunminai-3-12B model and perform a simple Korean-language chat completion.

# pip install accelerate

from transformers import AutoProcessor, Gemma3ForConditionalGeneration
import torch

model_id = "davidkim205/Hunminai-1.0-12b"

model = Gemma3ForConditionalGeneration.from_pretrained(
    model_id, device_map="auto"
).eval()

processor = AutoProcessor.from_pretrained(model_id)

messages = [
    {
        "role": "system",
        "content": [{"type": "text", "text": "당신은 유용한 AI 비서입니다."}]
    },
    {
        "role": "user",
        "content": [
            {"type": "text", "text": "대한민국의 수도는 어디인가요?"}
        ]
    }
]

inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt"
).to(model.device, dtype=torch.bfloat16)

input_len = inputs["input_ids"].shape[-1]

with torch.inference_mode():
    generation = model.generate(**inputs, max_new_tokens=128, do_sample=False)
    generation = generation[0][input_len:]

decoded = processor.decode(generation, skip_special_tokens=True)
print(decoded)

Training Dataset

The model was trained on high-quality Korean instructional data. The dataset was curated to cover a broad range of Korean language contexts and task types, with a focus on aligning model outputs with user intent and natural language generation. It is currently not publicly available.

Evaluation

Benchmarks Datasets

The table below contains a description of the Korean LLM evaluation benchmark dataset used for the model evaluation. More information on the benchmarks is available at Blog.

Benchmark Results