EXAONE-U-CURATE-Random1K-1epoch-v1

EXAONE U-CURATE Random1K 1epoch v1

This repository contains a standalone BF16 merged full model, not a LoRA adapter-only release. It combines the pinned EXAONE base model with the selected U-CURATE Random1K QLoRA checkpoint and can be loaded without PEFT adapter files.

Base model

• Model: LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct
• Revision: 553ea250b9a5317231459279d5847d6cf955b9aa
• License: EXAONE AI Model License Agreement 1.1 - NC (see LICENSE)

Training

• Method: QLoRA (4-bit NF4 training; BF16 compute)
• Training data: U-CURATE Random1K, 1,000 authorized training records
• Epochs: 1
• Optimizer steps: 125
• Final train loss: 2.7517
• Sequence length: 2,048
• Effective batch size: 8 (micro-batch 1, gradient accumulation 8)
• LoRA rank / alpha / dropout: 8 / 16 / 0.05
• LoRA targets: q_proj, k_proj, v_proj, out_proj
• Selected checkpoint: checkpoint-125
• Training hardware: NVIDIA A40 48 GB
• Training Git commit: bfbd12f375b5c3ed3bfea61e701384cd6f1c2d5f

The checkpoint was selected because it is the completed final step and no independent validation evidence justified choosing an earlier checkpoint.

Release format

• Precision: BF16
• Format: safetensors, 8 weight shards
• Merge: PEFT merge_and_unload(safe_merge=True) into the pinned base model
• Adapter SHA-256: c48554032752d18c7e120a40de9f6946c33792f3adf4a9129b8675bf22e158c9
• Shard and release-file hashes: MERGED_MODEL_SHA256SUMS

The included EXAONE custom modeling code contains a Transformers 5.15 generation-API compatibility adjustment. The adjustment does not modify model weights.

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "hoySky92/EXAONE-U-CURATE-Random1K-1epoch-v1"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype="auto",
    device_map="auto",
)

messages = [{"role": "user", "content": "한국의 사계절을 간단히 설명해 주세요."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Validation

• Fresh standalone Transformers load: PASS
• Korean greedy generation: 3/3 PASS
• Empty output / NaN or Inf / repetition explosion: none observed
• Adapter-versus-merged top-1 token agreement: PASS
• Adapter-versus-merged greedy 32-token sequence agreement: PASS

The stricter pre-registered numerical-logit fidelity threshold did not pass and has not been relaxed: cosine 0.9998158 versus required 0.9999, and relative L2 0.0428226 versus maximum 0.02. The original evidence is preserved under validation/; this release proceeds based on the exact functional token and greedy-generation agreement authorized by the release owner.

Limitations

This is an experimental one-epoch transfer run on 1,000 training samples. It is not a general guarantee of factuality, safety, or leaderboard performance. Users must comply with the base model license and independently evaluate the model for their intended use.