Downloads · 30 days
0
mistral-medquad-lora-r32
OSCARcr/mistral-medquad-lora-r32
mistral-medquad-lora-r32 is a machine learning model from OSCARcr. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Fine-tuned with LoRA (r=32) using dataset lavita/MedQuAD Base model: mistralai/Mistral-7B-Instruct-v0.3 Quantization: 4-bit NF4 GPU: A100 80GB
Access
Public
Updated Oct 28, 2025
Repo size
75.5 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors75.5 MB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Oct 27, 2025
- Updated
- Oct 28, 2025
- SHA
- 05d74668
Base models
Datasets
Model card
From the Hugging Face model README
Mistral 7B – LoRA r=32 (MedQuAD)
Fine-tuned with LoRA (r=32) using dataset lavita/MedQuAD
Base model: mistralai/Mistral-7B-Instruct-v0.3
Quantization: 4-bit NF4
GPU: A100 80GB
📊 Final Results
- Validation Loss: 0.8384
- Perplexity: 2.313
🧾 Training Metrics
| Epoch | Step | Training Loss | Validation Loss |
|---|---|---|---|
| 0.33 | 200 | 0.852 | 0.853 |
| 0.66 | 400 | 0.82 | 0.822 |
| 1.0 | 600 | 0.802 | 0.838 |
💻 Example usage
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
model = AutoPeftModelForCausalLM.from_pretrained("REPO_ID", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3")
text = "[INST] What is anemia? [/INST]"
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=80)
print(tokenizer.decode(out[0], skip_special_tokens=True)
- License
- apache-2.0
- Languages
- en
- Created
- Oct 27, 2025
- Updated
- Oct 28, 2025