openthai2.0-qwen3.8-27b

OpenThai2.0 - Opensource Thai Knowledge, Document, and Agentic AI

Release notes

• v2.0.3 (1 Sep 2026) — Thai factual recall repaired: the freerecall training data carried a phantom-citation register ("บทความระบุ...") and single-shape questions; cleaned and re-drilled with model-written paraphrases. Paraphrase battery 85/104 (from 79), Thai knowledge battery 21/21, OCR-Eval-104 CER 0.084 (from 0.093), HumanEval 0.982, IFEval-TH 0.837. Known residual: some numeric facts still scatter at temperature 0.7 — use low temperature for factual queries.
• v2.0.2 (31 Aug 2026) — identifies as OpenThai 2.0 (iApp Technology + AIEAT) and adds a Thai-language safety guardrail; stock chat template. Printed-OCR CER about +0.02 vs v2.0.1, other benchmarks unchanged.
• v2.0.1 (30 Aug 2026) — restores closed-book Thai knowledge lost in v2.0.0 (thanks to Dr. Panutat Tejasen's ThaiEval-2026 v3).
• v2.0.0 — initial release. Earlier builds: revision tags v2.0.1, v2.0.0.

Per-version benchmark tables: CHANGELOGS.md.

โมเดล AI โอเพนซอร์ส ด้านความรู้ภาษาไทย เอกสารไทย และงานเอเจนต์ · เปิดตัวโดย iApp Technology ร่วมกับ สมาคมผู้ประกอบการปัญญาประดิษฐ์ประเทศไทย (AIEAT) · Apache 2.0

An open all-in-one Thai AI: reads Thai documents and handwriting at specialist level, answers with Thai knowledge in natural, explanatory Thai, leads its base model and Typhoon on agentic tool use (BFCL) — and keeps its base model's general intelligence.

🌐 Try it FREE right now — no install: chinda3.iapp.co.th 🦙 Or one command on your own machine: ollama.com/openthai/openthai2.0-qwen3.8-27b

⚡ Run it locally

ollama run openthai/openthai2.0-qwen3.8-27b

Or serve the full bf16 weights with vLLM:

vllm serve iapp/openthai2.0-qwen3.8-27b \
  --max-model-len 32768 --gpu-memory-utilization 0.85 \
  --max-num-seqs 128 --reasoning-parser qwen3 --trust-remote-code

One 80 GB GPU (~56 GB bf16). The LoRA adapter alone (7 GB, rank 64) is in adapter/ for serving on top of Qwen/Qwen3.8-27B with dynamic LoRA.

Available formats

Multi-token prediction (MTP) — faster decoding, identical outputs

The checkpoint ships the Qwen3.8 MTP draft head (mtp.* tensors, text_config.mtp_num_hidden_layers: 1), so vLLM can run self-speculative decoding:

vllm serve iapp/openthai2.0-qwen3.8-27b \
  --max-model-len 32768 --max-num-seqs 128 --reasoning-parser qwen3 \
  --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":2}' \
  --trust-remote-code

Speculative decoding is verified token-by-token by the main model, so outputs are identical with or without it — the draft head only accelerates decoding. Measured: 75.2 tok/s with MTP vs 50.1 tok/s without (+50%, single-stream, H100, identical prompts, all outputs clean). Without the flag the head is simply skipped (no overhead). Per format: GGUF exports the head as nextn layers (llama.cpp); INT8/NVFP4 keep it in bf16; the MLX build omits it (mlx-vlm has no drafter support for this architecture yet).

INT8-W8A8 / NVFP4 before 2026-08-31 had no draft head. The quantisation export dropped the mtp.* tensors while the config still advertised them, so --speculative-config ran a draft head with no weights: every draft rejected (mean acceptance length 1.00) and decoding ~1.6× slower. Found by Dr. Panutat Tejasen. The repaired repos carry the head in bf16 (model-mtp.safetensors, re:.*mtp.* in the quantisation ignore list) and were verified under vLLM speculative decoding. If your INT8/NVFP4 download predates that, re-download — or serve it without --speculative-config. bf16 weights were never affected.

Highlights

🇹🇭 Thai knowledge & instruction-following

• Thai national exams (OpenThaiEval): 0.842 — above base (0.820), Typhoon 2.5 (0.742), Pathumma (0.660)
• Thai instruction-following (IFEval-TH): 0.795 — above the base model (0.772) and Typhoon (0.749)
• English instruction-following (IFEval): 0.914 — above base (0.902) · code-switching 0.985 (ties Pathumma's 0.990 within one sample)

📄 Thai documents

• Thai handwriting: 0.261 CER vs 0.649 base — a 60% error reduction
• Books & Royal Gazette: 0.126 vs 0.370 base — a 66% error reduction · printed docs 0.077 vs 0.103

🤖 Agentic

• Tool use (BFCL): 0.820 overall — beats base (0.811) and Typhoon 2.5 (0.792); multi-turn agentic 0.775 vs Typhoon's 0.550 · Coding (HumanEval): 0.957 — ties Typhoon's best

⚙️ In practice

• Answers in natural, explanatory Thai — switches to terse/JSON output on request
• 1.5× faster decoding with the bundled MTP draft head (75 vs 50 tok/s measured)
• Five formats — bf16 · GGUF · MLX · INT8 · NVFP4 — from a MacBook to Blackwell
• Runs entirely inside your own infrastructure — no data leaves your organization

Benchmark results

(The tables in this section were measured on v2.0.0. For the v2.0.1 numbers see the changelog table at the top of this card.)

All numbers measured under identical serving conditions, with the same answer-unwrap scoring pass applied to every model (verified: unwrapping changes OpenThai 2.0's scores by ≤0.001 on every track — it already answers OCR prompts cleanly).

Thai knowledge & language (higher is better)

Thai document & handwriting reading (CER — lower is better)

Reading the comparison

All four models measured by us on the same suite, each with its own recommended prompts/parameters (Typhoon-OCR via its exact model-card protocol), so the experiment can be repeated with the linked checkpoints above. Read it honestly: Typhoon-OCR 1.5 is an excellent transcription specialist — on clean printed lines and isolated handwriting it beats every model here, including this one. It is also only a transcriber: it accepts a single fixed OCR prompt, so document Q&A and scene-text questions score at or near total error (MTVQA 1.000), and it has no knowledge or assistant ability. OpenThai 2.0 leads on books/gazette reading and on Thai knowledge and instruction-following — the only Thai open model in these tables that reads documents at specialist level and answers questions about them in the same model. Scene text (MTVQA) is its weakest reading track.

English retention: Benchmarking (previous build: 0.333 CER) on a held-out CoSyn-400K English document set. English document reading is functional but is not this model's focus; for EN-primary workloads prefer the base model.

Agentic tool use — BFCL (higher is better)

Official Berkeley Function-Calling Leaderboard harness (bfcl-eval 2026.3.23): 3,841 cases across 14 categories, identical OpenAI-function-calling protocol for every model (tool parser qwen3_xml for the Qwen3.8 family, hermes for Typhoon, per each family's spec).

Thai-document fine-tuning did not tax tool use: OpenThai 2.0 leads every summary row, with the largest gap on multi-turn agentic episodes (+22.5 points over Typhoon 2.5). Typhoon is stronger on several single-turn AST categories; full per-category results ship in the repo (results/agentic-2026-08/).

Evaluation protocol

Benchmark sets are held out from training by an n-gram leakage guard run on every corpus change. Handwriting uses a text-disjoint split: no transcription string in training appears in the test set.

Prompt templates by use case

The exact prompts our benchmark numbers were measured with. Where the template is "—", no special prompt is needed — ask naturally in Thai.

What it looks like — 5 real cases

Verbatim model outputs on held-out benchmark items, same image and prompt for both models — all five use the Full-page transcription (Markdown) template above with thinking off.

Case 1 — Thai handwriting (hw_disjoint/48)

<img src="assets/cases/hw_disjoint_48.jpg" alt="Case 1 source document" width="480"/>

• Ground truth: เชียงราย
• Qwen3.8-27B (base) — CER 1.00: 3.1.4 การจ่าย
• OpenThai 2.0 — CER 0.00: เชียงราย

Case 2 — Thai handwriting (CPE-OPH) (hw_cpeoph/20)

<img src="assets/cases/hw_cpeoph_20.jpg" alt="Case 2 source document" width="480"/>

• Ground truth: นครพนม
• Qwen3.8-27B (base) — CER 1.00: ทุกตัวจริง
• OpenThai 2.0 — CER 0.00: นครพนม

Case 3 — Book / Royal Gazette page (wikisource_l4/ws4_53969)

<img src="assets/cases/wikisource_l4_ws4_53969.jpg" alt="Case 3 source document" width="480"/>

• Ground truth: กองอ้ายแลไปฟ้องกับอ้ายเวียงจันท์ว่าอ้ายแลไม่เข้าด้วยอ้ายเวียงจันท์ ๆ โกรธว่าจะฆ่าอ้ายแลเสีย ครั้นอ้ายแลไปถึง พูดจาประสมประสาน อ้ายเวียงจันท์หายโกรธ จึงให้อ้ายแลนำตำรวจเวียงจันท์มากวาดครัวเมืองไชยภูมิข …
• Qwen3.8-27B (base) — CER 1.00: ๒๕ กองอ้ายและไปฟังกองอ้ายของจนทว่า อ้ายและไม่เข้า ด้วยอ้ายของจนทๆ ใหญ่อ่ว่าจะฆ่าอ้ายและเสีย ครั้น อ้ายและไปลงพักจาประสมประสานอ้ายของจนทหายใหญ่อิ จึงให้อ้ายและนำตัวรววจของจนทมากกว่าครวเมื่องใชยภมิ ขึ้น …
• OpenThai 2.0 — CER 0.01: กองอ้ายแลไปฟ้องกับอ้ายเวียงจันท์ว่า อ้ายแลไม่เข้าด้วยอ้ายเวียงจันท์ ๆ โกรธว่าจะฆ่าอ้ายแลเสีย ครั้นอ้ายแลไปถึงพูดจาประสมประสานอ้ายเวียงจันท์หายโกรธ จึงให้อ้ายแลนำตำรวจเวียงจันท์มากวาดครัวเมืองไชยภูมิขึ …

Case 4 — Government document (parliament budget) (docbench/parliament_year_60_00000903_p0181)

<img src="assets/cases/docbench_parliament_year_60_00000903_p0181.jpg" alt="Case 4 source document" width="480"/>

• Ground truth: 175 (52) ถนนสาย สค.2055 แยก ทล.35 - เมืองสมุทรสงคราม (ตอนสมุทรสงคราม) อ.เมือง จ.สมุทรสงคราม 0.753 กม. 40,021,200 บาท งบประมาณทั้งสิ้น 81,600,000 บาท ปี 2559 ตั้งงบประมาณ 21,178,800 บาท ปี 2560 ตั้งงบป …
• Qwen3.8-27B (base) — CER 1.00: 175 (52) ถนนสาย สค.2055 แยก ทล.35 - เมืองสตูล สตูล (ตอนสตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล สตูล …
• OpenThai 2.0 — CER 0.02: (52) ถนนสาย สด.2055 แยก ทล.35 - เมืองสมุทรสงคราม (ดอนสมุทรสงคราม) อ.เมือง จ.สมุทรสงคราม 0.753 กม. 40,021,200 บาท งบประมาณทั้งสิ้น 81,600,000 บาท ปี 2559 ตั้งงบประมาณ 21,178,800 บาท ปี 2560 ตั้งงบประมาณ 40,021,200 บาท …

Case 5 — Thai national exam (OpenThaiEval item 200)

คำถาม: ท่านมีความเห็นอย่างไรต่อแพทย์ที่สั่งยาจากบริษัทที่มักพาแพทย์ไปเที่ยวต่างประเทศ (ข้อสอบจริยธรรมแพทย์ ปี 65) Choice: (1) เป็นการตอบแทนเล็กน้อยมิใช่หรือ (2) ถ้ายาดีจริง ก็เป็นสิ่งที่ทําได้มิใช่หรือ (3) แพทย์คนนี้เห็นแก่ประโยชน์ส่วนตัวใช่หรือไม่ (4) หากแพทย์สั่งยาจากบริษัทนี้อีก เท่ากับว่าแพทย์สนับสนุนการกระทําในสิ่งที่ไม่ถูกต้อง ใช่หรือไม่ (5) เป็นการกระทําที่ดูถูกศักดิ์ศรีความเป็นแพทย์ใช่หรือไม่

• Qwen3.8-27B (base): คำตอบที่ถูกต้องคือ (5) ❌
• OpenThai 2.0: คำตอบที่ถูกต้องคือ (4) ✅

How it was trained

Three stages of LoRA fine-tuning (r=64, α=64, all-linear) on ~143k verified rows:

1. Scale SFT teaches reading: verified transcription + explanatory document-QA over Thai handwriting, books, gazettes, and government documents (92,893 rows, 2 epochs).
2. Knowledge layer teaches Thai reasoning: grounded MCQ chain-of-thought built from Thai law/government corpora, leak-checked against evaluation questions.
3. Adherence layer teaches control: instruction rows kept only if the answer passed a programmatic constraint check (bullet counts, must-include words, JSON-only, length caps) — zero-hallucination instruction data.

Every stage used a verify-everything pipeline: answers checked against human-confirmed ground truth by n-gram containment; images the model could not read were routed away from explanatory training (anti-hallucination); malformed-Thai, scaffolding-leak, and markup filters on every row; benchmark-leakage guard on the whole corpus.

Model details

Deployment notes (read before serving)

• --max-num-seqs 128 is required: the hybrid architecture allocates one Mamba cache block per decode sequence; vLLM's default (1024) aborts engine startup.
• Serve with --reasoning-parser qwen3 and leave thinking ON for all text-only requests. Measured (QA 2026-08-24, 36 prompts/config): thinking on with max_tokens unset gives 94% usable answers and 0% reasoning leakage; sending enable_thinking:false on text-only requests degrades to 42% usable.
• enable_thinking:false is for image-attached OCR/transcription requests only (measured safe there: 0% leakage across 84 image requests) — it keeps reasoning text out of transcriptions.
• Leave max_tokens unset, or set it ≥ 8192. Small caps starve generation: at 1024 roughly a third of replies come back empty (finish_reason=length); unset brings this to ~6% (base model behaves the same). Budget ~3.5k output tokens per long answer.
• Always send repetition_penalty: 1.05 for long-form Thai generation, and trim the leading blank line some responses start with (content.lstrip()).

Recommended sampling parameters

Inherited from the base Qwen3.8-27B recommendations, with one OpenThai-specific change: repetition_penalty 1.05 in non-thinking mode (measured: prevents rare long-form Thai repetition collapse; short OCR outputs are unaffected).

{"temperature": 0.0, "top_p": 0.8, "top_k": 20, "repetition_penalty": 1.05, "max_tokens": 8192}

All benchmark numbers in this card were measured in non-thinking mode at temperature 0.

Tested configuration: vLLM 0.19, single H100 80GB, --max-model-len 16384–32768, --gpu-memory-utilization 0.85. All benchmark numbers in this card were produced under this configuration.

Quickstart

import base64
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
img = base64.b64encode(open("thai_document.jpg", "rb").read()).decode()

r = client.chat.completions.create(
    model="iapp/openthai2.0-qwen3.8-27b",
    messages=[{"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{img}"}},
        {"type": "text", "text": "อ่านข้อความในเอกสารนี้ทั้งหมด"},
    ]}],
    temperature=0.0,
    extra_body={"repetition_penalty": 1.05,
                "chat_template_kwargs": {"enable_thinking": False}},
)
print(r.choices[0].message.content)

Ask in natural Thai and it explains; add "ตอบสั้น ๆ" or "ตอบเป็น JSON เท่านั้น" for terse or structured output.

⚠️ Give it room to generate — this is the #1 setup mistake. The model reasons before it answers, so a short context window or a small max_tokens cap gets eaten by reasoning and the reply comes back empty. Serve with --max-model-len 32768 (or more) and leave max_tokens unset, or set it ≥ 8192. Measured: max_tokens 1024 → ~33% empty replies; unset → ~6%.

Run with TensorFold (Mac or NVIDIA)

TensorFold serves the MLX 4-bit checkpoint iapp/openthai2.0-qwen3.8-27b-MLX-4bit through an OpenAI-compatible API on Apple Silicon and on NVIDIA GPUs. It speeds up decoding with speculative drafts, and every drafted token is checked against the model, so replies are the same as TensorFold decoding this checkpoint one token at a time. The same checkpoint runs on both backends, with image input.

These instructions are written for TensorFold v0.6.5.

DGX Spark and other NVIDIA GPUs

Needs compute capability 8.9 or newer: RTX 40 / 50 series, Hopper, Blackwell, including the DGX Spark (GB10).

docker run -it --gpus all --ipc=host --network host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  nvcr.io/nvidia/pytorch:26.07-py3

# inside the container
python -m pip install "tensorfold[vision] @ git+https://github.com/ashhart/[email protected]"
tensorfold pull iapp/openthai2.0-qwen3.8-27b-MLX-4bit z-lab/Qwen3.8-27B-DFlash2
tensorfold serve iapp/openthai2.0-qwen3.8-27b-MLX-4bit \
  --name openthai2.0 --host 0.0.0.0 --port 8080 --vision

The first start compiles CUDA kernels for your GPU, which takes a few minutes once. Add --parallel 4 to decode up to four requests together. On CUDA the draft model is required: pull it as above, or serve with --no-drafts.

Apple Silicon (MLX)

48 GB of unified memory or more is recommended. On a 32 GB Mac the model does not fit TensorFold's default budget (70% of RAM); raise it with TENSORFOLD_MEMORY_LIMIT_GB, leaving little memory for other apps.

python -m pip install "tensorfold[vision] @ git+https://github.com/ashhart/[email protected]"
tensorfold pull iapp/openthai2.0-qwen3.8-27b-MLX-4bit z-lab/Qwen3.8-27B-DFlash2
tensorfold serve iapp/openthai2.0-qwen3.8-27b-MLX-4bit --name openthai2.0 --vision

Call it

Any OpenAI client works with base URL http://<host>:8080/v1 and model openthai2.0.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="none")

# Text
r = client.chat.completions.create(
    model="openthai2.0",
    messages=[{"role": "user", "content": "สรุปสาระสำคัญของ พ.ร.บ. คุ้มครองข้อมูลส่วนบุคคล ใน 3 ข้อ"}],
    max_tokens=4096,
)
print(r.choices[0].message.content)

# Thai document image (data URL; start the server with --vision-urls to accept public HTTPS links)
import base64
img = base64.b64encode(open("document.png", "rb").read()).decode()
r = client.chat.completions.create(
    model="openthai2.0",
    messages=[{"role": "user", "content": [
        {"type": "text", "text": "อ่านข้อความทั้งหมดในเอกสารนี้"},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{img}"}},
    ]}],
    max_tokens=4096,
)
print(r.choices[0].message.content)

The model thinks before it answers by default; the reasoning arrives in reasoning_content and the answer in content, so leave enough max_tokens. Start the server with --no-thinking to answer directly.

Notes

• Draft model. z-lab/Qwen3.8-27B-DFlash2 (Apache-2.0) was trained on base Qwen3.8-27B, not on OpenThai 2.0. It accepts fewer drafted tokens on Thai replies, so the speedup is smaller than on the base model; the replies themselves are unaffected. Send "draft": false in a request to get the one-token-at-a-time reference.
• Same output, same engine. Drafted and serial replies match within TensorFold on this checkpoint. Output differs from the bf16 model served by vLLM or transformers, because the weights are 4-bit.
• Why MLX 4-bit on NVIDIA too. TensorFold also reads the NVFP4 checkpoint on one NVIDIA GPU, but in TensorFold's own measurements of base Qwen3.8-27B on a DGX Spark, the MLX 4-bit checkpoint decoded faster.

<!-- ### Speed (TensorFold v0.6.5, OpenThai 2.0 MLX 4-bit) | Machine | Thai chat | Thai document QA | Prefill 8k | Concurrent x4 (aggregate) | | --- | ---: | ---: | ---: | ---: | | DGX Spark (GB10) | — tok/s | — tok/s | — tok/s | — tok/s | | MacBook Pro M_ Max, __ GB | — tok/s | — tok/s | — tok/s | — | -->

Limitations (please read)

• Structured extraction is at parity with the base model, no better. On judged ThaiOCRBench this build scores 0.568 vs the base model's 0.570 (n=2,808 — a statistical tie); the loss concentrates in table parsing, key-information extraction/mapping, and diagram VQA. It reads text better while organizing extracted data about as reliably as the base. For heavy form/table extraction pipelines, evaluate on your own documents.
• Long free-form answers vary between generations. For reproducible outputs set temperature 0 and repetition_penalty 1.05 (our benchmark protocol); for natural prose use the sampling table below. Verify facts independently either way.
• Terse-on-request compliance ≈ 40% (v9 canary: 42%). The model answers in an explanatory style by default; a bare "answer in one word" request is honored less than half the time. Use explicit formats ("ตอบเป็น JSON เท่านั้น") for machine-parsed outputs.
• Scene text is the weakest reading track (previous build 0.812 vs base 0.737; this build re-benchmarking) — for photos of signs/storefronts, prefer the base model or a scene-text specialist.
• Not evaluated for: Thai dialects, vertical/rotated text, medical or legal advice. Handwriting remains genuinely hard — 0.261 CER means roughly one character in four is still wrong on difficult samples.
• The model can still hallucinate on illegible input.

Recommended: pair with RAG for factual and regulated domains

This model is strongest at reading, explaining, and answering over documents you give it. For tax, legal, medical, or any compliance-sensitive work, we recommend running it with retrieval-augmented generation (RAG) over authoritative sources — e.g. current statutes, official announcements, or your organization's verified knowledge base — plus human review, rather than relying on closed-book statutory recall.

For the legal domain specifically, use our purpose-built Thai legal model: iapp/openthai2.0-legal-thaillm-nemotron-3-nano-30b-a3b.

📣 Press & social assets

Citation

@misc{openthai2_2026,
  title  = {OpenThai 2.0: An Open Thai Knowledge and Document AI},
  author = {iApp Technology and Artificial Intelligence Entrepreneur Association of Thailand},
  year   = {2026},
  url    = {https://openthai.aieat.or.th}
}

Built by iApp Technology, launched jointly with AIEAT. Base model: Qwen/Qwen3.8-27B (Apache 2.0).

Developed by OpenThai Team (iApp Technology / AIEAT)

ผู้สนับสนุน (Sponsor)

<a href="https://siam.ai"><img src="assets/siamai-logo.png" alt="Siam AI Corporation" width="120"/></a>

ได้รับการสนับสนุน GPU NVIDIA H100 × 8 ใบ จาก บริษัท สยาม เอไอ คอร์เปอเรชั่น จำกัด — ขอขอบคุณเป็นอย่างยิ่งสำหรับการสนับสนุนโครงสร้างพื้นฐานการประมวลผลตลอดโครงการ · siam.ai

Training and evaluation for this model ran on 8× NVIDIA H100 GPUs generously provided by Siam AI Corporation — thank you for supporting open Thai AI.