riad-disease-gpt

Riad Disease Gpt

A lightweight medical question-answering model trained on disease-related queries.

Model Details

• Type: Custom GPT-style Transformer
• Language: English
• Parameters: ~5M
• Architecture: 6-layer Transformer with 8 attention heads
• Vocab Size: 2500 (SentencePiece)
• Context Length: 512 tokens

Usage

from transformers import AutoModel, AutoTokenizer
import sentencepiece as spm

# Load model
model = AutoModel.from_pretrained("riadrayhan111/riad-disease-gpt", trust_remote_code=True)
tokenizer = spm.SentencePieceProcessor()
tokenizer.load('riadrayhan111/riad-disease-gpt/tokenizer.model')

# Generate response
class SimpleTokenizer:
    def __init__(self, sp_model):
        self.sp = sp_model
        self.eos_token_id = 2
    
    def encode(self, text):
        return self.sp.encode(text)
    
    def decode(self, tokens):
        return self.sp.decode(tokens)

tok = SimpleTokenizer(tokenizer)
answer = model.generate_text(tok, "What is fever?", max_length=100)
print(answer)

Training

Trained on a custom medical dataset focused on common diseases, symptoms, and treatments.

Limitations

⚠️ Important: This model is for educational purposes only.

• Not a substitute for professional medical advice
• Limited to information in training data
• Should not be used for medical diagnosis

License

Apache 2.0