MediTalk-300-Mistral-7B-4bit

MediTalk300-Mistral-7B

<!-- Provide a quick summary of what the model is/does. -->

This model card describes the MediTalk300-Mistral-7B model, which was trained on MediTalk300 dataset with corpus of 1725 doctor-patient role played dialogue pairs for 300 most commonly occurring diseases in India. Each conversation averages 300 tokens as mentioned in Table I below..

Pretraining Data

The MediTalk300-Mistral-7B model was trained on a large multicenter dataset with a large corpus of 1.2B words of diverse diseases we constructed.

<!-- For more details, see here. -->

Model Pretraining

Pretraining Procedures

To fine-tune MediTalk300 efficiently on limited hardware, we combined three key techniques.

We used PEFT to update only a small portion of the model’s parameters, keeping the rest frozen. This reduced training time and resource usage. Then, with QLoRA, we applied low-rank adapters along with 4-bit/8-bit quantization, allowing us to train the model on consumer GPUs without sacrificing accuracy. Finally, we applied Instruction Tuning by structuring our dataset as clear instruction-response pairs, helping the model understand user intent and generate medically accurate replies.

These methods allowed us to optimize both performance and efficiency, making MediTalk300 suitable for real-world use and local deployment.

Pretraining Hyperparameters

We used a batch size of 32, a maximum sequence length of 256, and a learning rate of 5e-5 for pre-training our models.

How to use the model

Load the model via the transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

model = AutoModelForCausalLM.from_pretrained("savinirsekas/MediTalk-300-Mistral-7B-4bit")
tokenizer = AutoTokenizer.from_pretrained("savinirsekas/MediTalk-300-Mistral-7B-4bit")

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)

query = "I have fever, chills, headache, and body pain. What could be the issue?"
response = pipe(query, max_new_tokens=256, do_sample=True)[0]["generated_text"]
print(response)

Citation

Please cite these articles: Uploading Soon....

---

base_model: unsloth/mistral-7b-instruct-v0.3-bnb-4bit tags:

• text-generation-inference
• transformers
• unsloth
• mistral
• trl license: apache-2.0 language:
• en

---

Uploaded model

• Developed by: savinirsekas
• License: apache-2.0
• Finetuned from model : unsloth/mistral-7b-instruct-v0.3-bnb-4bit