zombit-thai-gpt-reasoning

ZombitThai-GPT Reasoning Model

Overview

This is a fine-tuned version of ZombitThai-GPT with enhanced reasoning capabilities. The model has been trained on the Dhanishtha-2.0-SUPERTHINKER dataset to improve its logical reasoning and step-by-step problem-solving abilities.

Model Details

• Base Model: ZombitThai-GPT (custom Thai language model)
• Fine-tuning Dataset: HelpingAI/Dhanishtha-2.0-SUPERTHINKER
• Training Objective: Enhanced reasoning and logical inference
• Model Size: ~5.45 GB
• Vocabulary Size: 50,000 tokens
• Max Sequence Length: 512 tokens

Configuration

{
  "vocab_size": 50000,
  "d_model": 2048,
  "num_heads": 16,
  "num_layers": 24,
  "d_ff": 8192,
  "max_seq_len": 512,
  "dropout": 0.1,
  "reasoning_enabled": true,
  "max_reasoning_steps": 5,
  "reasoning_temperature": 0.3,
  "confidence_threshold": 0.7
}

Usage

Loading the Model

import torch
from huggingface_hub import hf_hub_download
import json

# Download model and config
model_path = hf_hub_download(
    repo_id="JonusNattapong/zombit-thai-gpt-reasoning",
    filename="best_model.pt"
)

config_path = hf_hub_download(
    repo_id="JonusNattapong/zombit-thai-gpt-reasoning",
    filename="config.json"
)

# Load config
with open(config_path, 'r', encoding='utf-8') as f:
    config = json.load(f)

# Load model checkpoint
checkpoint = torch.load(model_path, map_location='cpu')
print(f"Model loaded with {len(checkpoint['model_state_dict'])} parameters")

Example Usage

# Example reasoning prompts
questions = [
    "ถ้าฉันมีแอปเปิ้ล 3 ลูก และเพื่อนให้อีก 5 ลูก จะมีทั้งหมดกี่ลูก?",
    "น้ำเดือดที่อุณหภูมิเท่าไรที่ระดับน้ำทะเล?",
    "1+1 เท่ากับเท่าไร?"
]

def create_reasoning_prompt(question: str) -> str:
    return f"คำถาม: {question}\n\nการแก้ปัญหา:\n1. เข้าใจโจทย์\n2. แผนการแก้\n3. การคำนวณ\n4. ตรวจสอบคำตอบ\n\nคำตอบ:"

for question in questions:
    prompt = create_reasoning_prompt(question)
    print(f"Question: {question}")
    print(f"Prompt: {prompt[:100]}...")
    # Add your inference code here

Training Details

• Training Dataset: Dhanishtha-2.0-SUPERTHINKER (11,720 reasoning samples)
• Batch Size: 1 (for testing, can be increased for production)
• Learning Rate: 2e-5
• Epochs: 1 (test run)
• Optimizer: AdamW
• Loss Function: Cross-entropy
• Hardware: CPU-based training

Features

• Enhanced Reasoning: Improved logical reasoning capabilities
• Thai Language Support: Optimized for Thai language processing
• Step-by-Step Analysis: Trained on structured reasoning patterns
• Confidence Scoring: Built-in confidence threshold mechanism
• Memory Efficient: Designed for CPU inference

Limitations

• Currently trained on minimal dataset (3 samples for testing)
• Requires further fine-tuning with larger datasets for production use
• CPU-only inference (can be adapted for GPU)
• Limited to Thai language reasoning tasks

Future Improvements

• Train on full Dhanishtha-2.0-SUPERTHINKER dataset (11,720 samples)
• Implement GPU support for faster inference
• Add multi-language reasoning capabilities
• Integrate with advanced reasoning frameworks
• Add evaluation metrics and benchmarks

License

This model is provided as-is for research and educational purposes.

Contact

For questions or contributions, please contact the repository maintainer.

Repository

• Hugging Face: https://huggingface.co/JonusNattapong/zombit-thai-gpt-reasoning
• GitHub: https://github.com/JonusNattapong/Model-Reasoning