broken-model-fixed

broken-model-fixed

This repository is a corrected version of yunmorning/broken-model, created as part of an engineering debugging exercise.

---

Problem (a) — Root Cause Analysis and Fixes

[main] Fix 1: Added chat_template to tokenizer_config.json

What was missing: The tokenizer_config.json had no chat_template field.

Why this breaks /chat/completions: OpenAI-compatible inference servers receive requests as a list of {role, content} message objects. Before any token is fed to the model, the server must serialize this list into a flat string using the model's chat template. This serialization step is handled by HuggingFace's tokenizer.apply_chat_template(), which reads the chat_template field from tokenizer_config.json.

Without this field, apply_chat_template() raises a TemplateError immediately — the request never reaches the model weights. The failure is pre-inference and 100% reproducible on every request.

Fix applied: Added the chat_template from the official Qwen/Qwen3-8B repository. The template implements:

• Per-turn format: <|im_start|>{role}\n{content}<|im_end|>\n
• enable_thinking=False → injects <think>\n\n</think>\n\n after <|im_start|>assistant\n to suppress reasoning tokens
• Tool call serialization via <tool_call> / <tool_response> tags
• Multi-turn tool use with correct role handling

---

[Documentation] Fix 2: Corrected base_model in README.md

Before: base_model: meta-llama/Meta-Llama-3.1-8B
After: base_model: Qwen/Qwen3-8B

Why this is wrong: The model is clearly Qwen3-based ("model_type": "qwen3"), not Llama-3. Since Qwen3 and Llama-3 use different tokenizers and chat formats, keeping the wrong base_model metadata could cause inference systems to apply the wrong chat template.

---

Other fields reviewed