Downloads · 30 days
8
11% of all-time downloads
llama3-medquad-lora
ellyfantina/llama3-medquad-lora
llama3-medquad-lora is a machine learning model from ellyfantina. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as cc-by-nc-4.0.
Llama-3.1-8B-MedQuAD-LoRA is a parameter-efficient fine-tuned version of meta-llama/Llama-3.1-8B-Instruct, trained on the MedQuAD (Medical Question Answering Dataset) using QLoRA for factual and educational biomedical…
All-time downloads
76
Public
Repo size
252 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt168 MB · 47%
At a glance
- Library
- peft
- License
- cc-by-nc-4.0
- Access
- Public
- Created
- Oct 14, 2025
- Updated
- Oct 15, 2025
- SHA
- ab6fc9c1
Base models
Model card
From the Hugging Face model README
🧬 Llama-3.1-8B-MedQuAD-LoRA
Llama-3.1-8B-MedQuAD-LoRA is a parameter-efficient fine-tuned version of meta-llama/Llama-3.1-8B-Instruct, trained on the MedQuAD (Medical Question Answering Dataset) using QLoRA for factual and educational biomedical question answering.
⚙️ Technical Overview
| Setting | Value |
|---|---|
| Base model | meta-llama/Llama-3.1-8B-Instruct |
| Fine-tuning method | QLoRA (4-bit NF4 quantization) |
| LoRA rank / alpha | r = 8 / α = 16 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Dropout | 0.1 |
| Precision | FP16 mixed |
| Batch size (effective) | 8 (2 × grad accum = 4) |
| Learning rate | 2e-4 |
| Optimizer | AdamW (β₁ = 0.9, β₂ = 0.95) |
| Scheduler | Linear decay, warmup = 3 % |
| Max sequence length | 1024 |
| Epochs | 2 |
| GPU | NVIDIA T4 (15 GB VRAM) |
| Training time | ≈ 8 hours |
| Final loss (train/val) | 0.76 / 0.75 |
- Library
- peft
- License
- cc-by-nc-4.0
- Created
- Oct 14, 2025
- Updated
- Oct 15, 2025