Qwen3-VL-8B-Thinking-SEA-Reasoning-Answer

Qwen3-VL-8B-Thinking-SEA Reasoning Language Models

<div align="center">

</div>

Towards fair and equitable AI for all communities. This model suite adapts Qwen/Qwen3-VL-8B-Thinking for native multilingual Chain-of-Thought (CoT) reasoning across low-resource Southeast Asian (SEA) languages using Onramp Sequence Cross-Distillation (OSCD).

---

Language-Supported Benchmarks (out_seq_length=81920)

Data sources: AIME25-SEA | HMMT25-SEA

---

Released Model Variants

We release two sequentially post-trained model variants to support different operational and research requirements:

---

Post-Training Workflow

The models were post-trained in two sequential phases, using high-quality synthetic traces that were translated on-the-fly via aisingapore/Gemma-SEA-LION-v4-27B-IT powered by ClimbSeq — an asynchronous, chunk-based agentic translation framework.

Qwen3-VL-8B-Thinking ──► [Phase 1: OSCD] ──► Qwen3-VL-8B-Thinking-SEA-Reasoning ──► [Phase 2: OSCD/SFT] ──► Qwen3-VL-8B-Thinking-SEA-Reasoning-Answer

1. Phase 1 (...-SEA-Reasoning):
   • Objective: Align reasoning traces to target SEA languages natively; foster uniformity of Chain-of-Thought performance across multiple languages.
   • Training Dataset: 70k problem-only sample subset from nvidia/OpenMathReasoning (tir split).
   • Joint-Embedding Semantic Alignment: Implemented on the closing think tag </think>.
2. Phase 2 (...-SEA-Reasoning-Answer):
   • Objective: Align answer traces to target SEA languages natively; enforce consistency of language throughout entirety of generated completions.
   • Training Dataset: 21k non-overlapping, problem-only sample subset from nvidia/OpenMathReasoning (tir split).
   • Joint-Embedding Semantic Alignment: Not Implemented.

---

Generation Hyperparameters (Qwen3 Referenced)

VL

export greedy='false'
export top_p=0.95
export top_k=20
export repetition_penalty=1.0
export presence_penalty=0.0
export temperature=1.0
export out_seq_length=40960

Text

export greedy='false'
export top_p=0.95
export top_k=20
export repetition_penalty=1.0
export presence_penalty=1.5
export temperature=1.0
export out_seq_length=32768 (for aime, lcb, and gpqa, it is recommended to set to 81920)

---

Inference and Prompting

To optimally trigger model compliance and native generated completions, we recommend providing both the system and user prompts in target language. Users may also experiment with flexible arrangements depending on invididual taste or service constraints, although a higher composition of language-mixing is to be expected. This varies with task complexity too.

Quickstart Requirements

pip install "transformers>=4.57.1"
pip install "qwen-vl-utils>=0.0.14"

Example 1 (VL)

from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info

# default: Load the model on the available device(s)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    "sglim/Qwen3-VL-8B-Thinking-SEA-Reasoning-Answer", dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("sglim/Qwen3-VL-8B-Thinking-SEA-Reasoning-Answer")
messages = [
{
        "role": "system",
        "content": [
            {
                "type": "text", 
                "text": "Anda adalah asisten AI yang bermanfaat."
            }
        ]
    },
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
            },
            {"type": "text", "text": "Jelaskan gambar ini secara detail menggunakan bahasa Indonesia. Apa yang Anda lihat?"},
        ],
    }
]

# Preparation for inference
text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
)
inputs = inputs.to(model.device)

# Generation
generated_ids = model.generate(**inputs, max_new_tokens=8192)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text[0])

Example 2 (Text)

from transformers import Qwen3VLForConditionalGeneration, AutoProcessor

# default: Load the model on the available device(s)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    "sglim/Qwen3-VL-8B-Thinking-SEA-Reasoning-Answer", dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("sglim/Qwen3-VL-8B-Thinking-SEA-Reasoning-Answer")
messages = [
    {
        "role": "system",
        "content": "โปรดใช้ความคิดอย่างลึกซึ้งและให้เหตุผลทีละขั้นตอนเป็นภาษาไทย และให้คำตอบสุดท้ายของคุณในรูปแบบ \\boxed{}",
    },
    {
        "role": "user",
        "content": "ผลลัพธ์ของ 15 คูณด้วย 12 แล้วลบด้วย 30 คืออะไร",
    },
]

# Preparation for inference
text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = processor(
    text=[text],
    return_tensors="pt"
)
inputs = inputs.to(model.device)

# Generation
generated_ids = model.generate(**inputs, max_new_tokens=8192)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text[0])

System Prompt Examples

    chinese:       
    system_prompt: |
        请一步一步地用华语进行深度思考，并将您的最终答案填写在 \\boxed{} 中。

    english:
    system_prompt: |
        Please think deeply and reason step by step in English, and provide your final answer within \\boxed{} format. 

    indonesian:
    system_prompt: |
        Harap berpikir secara mendalam dan menalar langkah demi langkah dalam bahasa Indonesia, dan berikan jawaban akhir Anda dalam format \\boxed{}.

    thai:
    system_prompt: |
        โปรดใช้ความคิดอย่างลึกซึ้งและให้เหตุผลทีละขั้นตอนเป็นภาษาไทย และให้คำตอบสุดท้ายของคุณในรูปแบบ \\boxed{}

    vietnamese:
    system_prompt: |
        Hãy suy nghĩ sâu sắc và lập luận từng bước bằng tiếng Việt, đồng thời đưa ra câu trả lời cuối cùng của bạn trong định dạng \\boxed{}.

    filipino:
    system_prompt: |
        Mangyaring mag-isip nang malalim at mangatwiran nang sunud-sunod sa wikang Filipino, at ibigay ang iyong huling sagot sa format na \\boxed{}.

    tamil:
    system_prompt: |
        தயவுசெய்து ஆழமாகச் சிந்தித்து, தமிழில் படிப்படியாகக் காரணங்களைக் கூறி, உங்கள் இறுதிப் பதிலைப் \\boxed{} வடிவத்தில் வழங்கவும்.
        

---

Disclaimer

The released models were post-trained on synthetically generated completions with on-the-fly AI translations. While quality control measures were applied to the pipeline, AI-generated translations may occasionally contain minor errors, stylistic inaccuracies, or subtle nuances lost in translation. The models are released as-is to foster openness and transparent research on multilingual LLMs, with the goal of providing equitable AI for all communities and drive the organic growth of native web data.

---

Citation

If you use these models or the OSCD methodology in your work, please consider citing our preprint:

@misc{lim2026nativemultilingualchainofthoughtreasoning,
    title={Native Multilingual Chain-of-Thought Reasoning in Low-Resource Southeast Asian Languages}, 
    author={Sean Gip Lim and William Chandra Tjhi and Hai Leong Chieu},
    year={2026},
    eprint={2608.00533},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2608.00533}, 
}

Please also consider citing the referenced model/works from the Qwen3 team:

@misc{qwen3technicalreport,
    title={Qwen3 Technical Report}, 
    author={Qwen Team},
    year={2025},
    eprint={2505.09388},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2505.09388}, 
}

@article{Qwen2.5-VL,
    title={Qwen2.5-VL Technical Report},
    author={Bai, Shuai and Chen, Keqin and Liu, Xuejing and Wang, Jialin and Ge, Wenbin and Song, Sibo and Dang, Kai and Wang, Peng and Wang, Shijie and Tang, Jun and Zhong, Humen and Zhu, Yuanzhi and Yang, Mingkun and Li, Zhaohai and Wan, Jianqiang and Wang, Pengfei and Ding, Wei and Fu, Zheren and Xu, Yiheng and Ye, Jiabo and Zhang, Xi and Xie, Tianbao and Cheng, Zesen and Zhang, Hang and Yang, Zhibo and Xu, Haiyang and Lin, Junyang},
    journal={arXiv preprint arXiv:2502.13923},
    year={2025}
}

@article{Qwen2VL,
    title={Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution},
    author={Wang, Peng and Bai, Shuai and Tan, Sinan and Wang, Shijie and Fan, Zhihao and Bai, Jinze and Chen, Keqin and Liu, Xuejing and Wang, Jialin and Ge, Wenbin and Fan, Yang and Dang, Kai and Du, Mengfei and Ren, Xuancheng and Men, Rui and Liu, Dayiheng and Zhou, Chang and Zhou, Jingren and Lin, Junyang},
    journal={arXiv preprint arXiv:2409.12191},
    year={2024}
}

@article{Qwen-VL,
    title={Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond},
    author={Bai, Jinze and Bai, Shuai and Yang, Shusheng and Wang, Shijie and Tan, Sinan and Wang, Peng and Lin, Junyang and Zhou, Chang and Zhou, Jingren},
    journal={arXiv preprint arXiv:2308.12966},
    year={2023}
}