cwie-core

npc_LoRA

npc_LoRA is a LoRA adapter built on top of Qwen/Qwen2.5-3B-Instruct, designed to generate emotionally rich, context-aware dialogue for non-player characters (NPCs) in Korean-language game environments.

This project is part of a portfolio for industrial service roles in AI and game development, showcasing practical model design, multi-head training, and real-world integration strategies.

🧠 Model Architecture

• Base model: Qwen2.5-3B-Instruct
• Adapter type: LoRA (via PEFT)
• Language: Korean
• Task: Text generation with auxiliary heads
• Heads added:
  • delta_head: Predicts 2D continuous values for narrative state change
  • flag_head: Predicts 3 or more binary flags for game logic triggers

🏗️ Training Setup

• Environment: Google Colab with A100 GPU
• Quantization: 4-bit (nf4) via BitsAndBytes
• Batch size: 2 (gradient accumulation: 8)
• Epochs: 6
• Losses:
  • Language modeling (CrossEntropy)
  • Delta prediction (MSE)
  • Flag prediction (BCE)

📜 Prompt Format

<SYS>
NPC_ID=...
TAGS:
 location=...
 quest_stage=...
 relationship=...
 trust=...
 npc_mood=...
 player_reputation=...
 style=...
REQUIRE:
 ...
FORMAT:
 <RESPONSE>...</RESPONSE>
 <DELTA ...>
 <FLAG ...>
</SYS>
<CTX>
player: ...
npc: ...
</CTX>
<PLAYER>...
<NPC>

🔍 Inference Example

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch.nn as nn

BASE_MODEL = "Qwen/Qwen2.5-3B-Instruct"
ADAPTER_PATH = "minjae/npc_LoRA"

tokenizer = AutoTokenizer.from_pretrained(ADAPTER_PATH, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(BASE_MODEL, device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(model, ADAPTER_PATH)

# Add heads
hidden_size = model.config.hidden_size
model.delta_head = nn.Linear(hidden_size, 2).to(model.device)
model.flag_head = nn.Linear(hidden_size, 3).to(model.device)

prompt = "<SYS>...<CTX>...<PLAYER>...<NPC>"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model(**inputs, output_hidden_states=True)
    gen_ids = model.generate(**inputs, max_new_tokens=100)
    generated_text = tokenizer.decode(gen_ids[0], skip_special_tokens=True)

    last_hidden = outputs.hidden_states[-1][:, -1, :]
    delta = model.delta_head(last_hidden)
    flag = model.flag_head(last_hidden)

print("Response:", generated_text)
print("Delta:", delta)
print("Flags:", torch.sigmoid(flag))

🧩 Use Cases

• NPC dialogue generation in Korean RPGs
• Emotionally adaptive storytelling
• Game logic trigger prediction (e.g., quest progression, item handoff)

📁 Repository Structure

npc_LoRA/
├── lora-output-jason-mom-head/         # LoRA adapter files
├── README.md

📌 Notes

• Adapter is optimized for Korean-language prompts and multi-turn dialogue.
• Designed to integrate with game engines or AI-driven simulation platforms.
• Compatible with Hugging Face Spaces (CPU/GPU) and local inference.

📜 License

MIT

👤 Author

Created by Minjae
Portfolio: GitHub Profile
Contact: [[email protected]]