AX-3.1-Light-sft_source_screen_71533_3000

A.X-3.1-Light Source Screening 71533

본 모델은 skt/A.X-3.1-Light pristine base model에 AI Hub 71533 기술·과학 문서 기계독해 데이터만 사용해 1 epoch LoRA SFT를 수행한 뒤 adapter를 병합한 BF16 전체 가중치 모델입니다. 여러 데이터셋을 섞지 않고 단일 source의 효과를 확인하기 위한 controlled source-only screening 모델이며, 문제의 정보에 근거해 답을 먼저 제시하는 형식을 따르도록 학습했습니다. 이 모델은 연구·비교 평가용이며 특정 benchmark 성능 향상을 보장하지 않습니다.

Model information

• Model name: AX-3.1-Light-sft_source_screen_71533_3000
• Base model: skt/A.X-3.1-Light
• Base model revision: 9b41bb2406472634d8812c0b8931fa40fa9a6c3a
• Fine-tuning method: LoRA supervised fine-tuning, merged for inference
• Model format: standalone BF16 safetensors; no separate adapter is required
• Intended use: Korean text generation, instruction following, and source-effect research

Training

• Training samples: 3,000
• Internal validation samples: 300; not used for gradient updates
• Epochs: 1
• Maximum sequence length: 2,048
• Precision: BF16
• Batch size / gradient accumulation: 1 / 8
• Learning rate: 5e-5
• Scheduler: cosine; warmup ratio 0.03
• Weight decay: 0.01
• LoRA rank / alpha / dropout: 16 / 32 / 0.05
• LoRA target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Packing: disabled
• Seed: 42
• Training loss: 0.2149649382
• Internal validation loss: 0.1982743144
• Training runtime: 2,174.4269 seconds

The model was trained with answer-first formatting. The intended response contract is to put the answer at the beginning, such as 정답: <값>, followed by only the short information needed by the task when applicable.

Training data

Only AI Hub dataset 71533, 기술과학 문서 기계독해 데이터 was used for training. The 3,000 selected text-only training examples were deterministically stratified across the source's internal categories: ED, EE, LA, and NA, with 750 examples per category. The source-level answer-type counts were balanced as part of the selection. Broken rows, missing-answer rows, duplicate rows, image-dependent rows, and externally dependent rows were excluded.

No v0.21 mixture, other AI Hub source, or public benchmark question was used for training. The raw AI Hub data and benchmark data are not included in this repository; users must follow the applicable AI Hub terms of use.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "youngseok12/AX-3.1-Light-sft_source_screen_71533_3000"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "system", "content": "문제에 포함된 정보만 사용해 답하세요."},
    {"role": "user", "content": "질문에 답하고 정답을 먼저 제시하세요."},
]
inputs = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=128, do_sample=False)

The bundled tokenizer includes the base model chat template. Keep the prompt format consistent with the intended conversational interface.

Evaluation status and limitations

This repository upload does not claim official K-AI leaderboard results. A canonical benchmark evaluation was not included in this upload step. The model can produce incorrect, incomplete, biased, or poorly formatted answers and must not be used as the sole basis for medical, legal, financial, or other high-risk decisions.

License

This model is derived from the Apache-2.0 licensed skt/A.X-3.1-Light model. The base model's notices and SK Telecom trademark terms also apply. AI Hub dataset terms apply to the source data. See LICENSE and the base model repository for the applicable terms.