Qwen3-8B-MaKTO-Public-SFT

Qwen3-8B MaKTO Public-Data SFT

This is a full-parameter supervised fine-tuning (SFT) checkpoint of Qwen/Qwen3-8B, trained on the public Chinese portion of ReneeYe/werewolf_game_reasoning.

The model is an independent public-data reproduction inspired by the MaKTO paper. It is not a reproduction of the complete paper training recipe: the approximately 12K general-purpose SFT examples described by the paper are not public and were not used, and the Multi-agent KTO stage has not yet been performed.

Training data

The converted training corpus contains 12,886 examples:

• 12,134 game-behavior examples
• 372 advanced-technique examples
• 380 fundamental-term examples

The source dataset revision used for conversion was 7d7d516f780879dbe9f512ef29655f51548f7578.

Training procedure

• Full-parameter BF16 SFT
• 4 GPUs with DeepSpeed ZeRO-3
• 3 epochs
• Learning rate: 1e-6
• Scheduler: cosine
• Warmup ratio: 0.05
• Per-device batch size: 1
• Gradient accumulation: 4
• Effective global batch size: 16
• Maximum sequence length: 6,144 tokens
• Qwen3 non-thinking chat template
• Seed: 42

The run completed 2,418 optimizer steps. Final training loss was 0.7517516736258545. No held-out evaluation set was used during this run.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "XDQAQ/Qwen3-8B-MaKTO-Public-SFT"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [{"role": "user", "content": "请分析当前狼人杀局势。"}]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=False,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Use transformers>=4.51.0 for Qwen3 support.

Limitations

The model is specialized for language-game and Werewolf-style interactions. It has not been evaluated as a general assistant, and its outputs can be incorrect, inconsistent, or strategically deceptive in keeping with the training domain. Users should evaluate it for their own intended use.

References

• MaKTO paper
• MaKTO-Werewolf code
• Public training dataset

License

The base model is distributed under the Apache License 2.0. The source dataset is marked as MIT on its Hugging Face dataset page. Users are responsible for complying with the licenses and terms of the base model and source dataset.