Llama-3.2_3B-Reasoning-TW-Preview

小參數長鏈思考模型（Chain-of-Thought for Traditional Chinese）

Finetuned LLaMA 3.2 3B on Chain-of-Thought Reasoning in Traditional Chinese

模型簡介 | Model Overview

這是一個專為繁體中文社群設計的開源長鏈思考（Chain-of-Thought, CoT）微調模型，基於 Meta LLaMA 3.2 3B 架構進行微調，聚焦於增強模型在繁體中文語境下的推理能力與多步邏輯表達。

This is an open-source Chain-of-Thought (CoT) finetuned model for the Traditional Chinese community, built upon Meta's LLaMA 3.2 3B architecture. It enhances multi-step reasoning and logical articulation within a Traditional Chinese context.

訓練動機 | Training Motivation

作為一名人工智慧愛好者，我發現目前針對繁體中文語境的長鏈思考模型仍然稀缺，許多開源模型偏向英文或簡體中文。因此，我著手打造此模型，期望為繁體中文用戶提供更合適的邏輯推理基礎模型，並推廣 CoT 技術在華語世界的應用與理解。

As an AI enthusiast, I noticed the scarcity of open-source CoT models tailored for Traditional Chinese. Most models are either English-based or optimized for Simplified Chinese. This project aims to fill that gap, offering a dedicated reasoning-friendly model for Traditional Chinese users, and promoting CoT applications in the broader Chinese-speaking world.

特性簡述 | Key Features

• 語言支援：專為繁體中文設計，保留文化語感

• 推理能力：優化多步邏輯思考與問題拆解

• 開源導向：歡迎社群參與微調與改進

• 小參數模型：3B 規模，適合在中等資源設備上運行

• Language Support: Tuned for Traditional Chinese with cultural nuance

• Reasoning-Ready: Enhanced for multi-step problem-solving

• Open-Source Friendly: Community contributions are welcome

• Lightweight: 3B size, ideal for moderate hardware environments

訓練細節 | Training Details

• Base Model：Meta LLaMA 3.2 3B
• 微調任務：Chain-of-Thought prompting in Traditional Chinese
• 資料集來源：自建與繁體化處理的開源數據（涵蓋數學、邏輯推理、日常問答等）
• 訓練策略：使用 LoRA 精簡微調技術（Low-Rank Adaptation），提升上下文理解與推理連貫性
• 硬體資源：單張 NVIDIA RTX 4060，進行約 16 小時微調
• 訓練框架：基於 Hugging Face Transformers + PEFT + bitsandbytes 訓練

Training Details (English)

• Base Model: Meta LLaMA 3.2 3B
• Task: Chain-of-Thought prompting in Traditional Chinese
• Dataset: Custom-built and adapted Traditional Chinese datasets (math, logical reasoning, daily QA)
• Tuning Strategy: Lightweight LoRA finetuning to boost context handling and step-by-step reasoning
• Hardware: Trained on a single NVIDIA RTX 4060 GPU for approximately 16 hours
• Framework: Powered by Hugging Face Transformers, PEFT, and bitsandbytes

使用建議 | Usage Tips

此模型適用於以下應用場景：

• 推理任務與數學解題
• 教學與邏輯問答
• 多步驟任務規劃

適合與 CoT 提示語搭配，例如：
「請一步一步解釋你的推理過程。」

Recommended for tasks such as:

• Logical reasoning and math problems
• Educational QA
• Step-by-step task planning

Pair well with CoT-style prompts like:
"Please explain your reasoning step by step."

歡迎貢獻 | Contribute

此模型開放給社群一同優化與實驗，若你也關心繁體中文在 AI 領域的表現，歡迎 fork、finetune 或提交 PR。

This model is open to community collaboration. If you care about advancing Traditional Chinese capabilities in AI, feel free to fork, finetune, or open a PR!