gpt-oss-20b-medical-az

GPT-OSS 20B Medical Reasoning (Azerbaijani)

This is a fine-tuned version of the OpenAI GPT-OSS 20B model specialized for medical reasoning and diagnostic tasks in the Azerbaijani language. Article about FT - https://medium.com/@vrashad/gpt-oss-20b-modelinin-az%C9%99rbaycan-tibbi-dataseti-il%C9%99-fine-tuning-edilm%C9%99si-c9abf4819ede

Model Details

• Base Model: openai/gpt-oss-20b
• Adapter Type: LoRA (Low-Rank Adaptation)
• Language: Azerbaijani
• Domain: Medical reasoning and diagnosis
• Training Dataset: Medical Q&A reasoning dataset (19,000 examples)

Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("openai/gpt-oss-20b")
base_model = AutoModelForCausalLM.from_pretrained(
    "openai/gpt-oss-20b",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load PEFT adapter
model = PeftModel.from_pretrained(base_model, "vrashad/gpt-oss-20b-medical-az")
model = model.merge_and_unload()

# Generate response
messages = [
    {"role": "system", "content": "reasoning language: Azerbaijani"},
    {"role": "user", "content": "Ürək ağrısının səbəbləri nə ola bilər?"},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

with torch.no_grad():
    output_ids = model.generate(
        input_ids,
        max_new_tokens=2048,
        do_sample=True,
        temperature=0.6,
        top_p=0.9,
    )

response = tokenizer.decode(
    output_ids[0][input_ids.shape[1]:], 
    skip_special_tokens=True
)
print(response)

Training Details

• Training Method: LoRA fine-tuning
• Rank (r): 16
• Alpha: 32
• Target Modules: Attention and MLP layers
• Training Epochs: 1
• Learning Rate: 1e-4
• Batch Size: 16 (effective)

Dataset

The training dataset consists of medical questions and answers with reasoning. Each sample follows this format:

• System prompt: "reasoning language: Azerbaijani"
• User question: Medical question in Azerbaijani
• Assistant response: <thinking>reasoning process</thinking>final answer

Limitations

• Optimized specifically for Azerbaijani language
• Intended for educational purposes only, not for medical advice
• Professional medical consultation is required for real medical decisions