llama-3.1-8b-onco-lora

llama-3.1-8b-onco-lora — Oncology QLoRA adapter (Spanish)

QLoRA adapter fine-tuning mlx-community/Meta-Llama-3.1-8B-Instruct-4bit on a Spanish synthetic oncology question-answer dataset derived from clinical guidelines (NCCN, ESMO, IMSS, national consensus). Architecture: dense, 8B.

WARNING: Released as a documented negative result and for reproducibility - NOT a production model. In our blinded study, domain fine-tuning reduced clinical answer quality vs. the un-augmented base model. Under an LLM judge (Claude Opus 4), this adapter scored 0.64/5 on a 4-criterion clinical rubric, vs —/5 for the same base model with no adapter. Do not use for clinical decision-making.

Intended use

• Reproducing the paper's experiments; studying why domain fine-tuning can degrade a strong base model.
• Not for clinical, diagnostic, or treatment decisions.

Training

• MLX (mlx-lm), QLoRA on a Mac Mini M4 (64 GB), 4-bit base.
• LoRA rank 64, all layers, prompt masking, dropout 0.05, scale 32.
• Learning rate 2e-5, 2500 (best checkpoint) iterations, max seq length 2048.
• Data: Spanish teacher-generated Q&A grounded in copyrighted guideline text (corpus itself not redistributable).

Usage (MLX)

pip install mlx-lm
python -m mlx_lm.generate --model mlx-community/Meta-Llama-3.1-8B-Instruct-4bit --adapter-path ./ \
  --prompt "Paciente con adenocarcinoma pulmonar EGFR+ estadio IV. Primera linea?"

Evaluation

Citation

Fraga Sastrias, J. M. et al. Base Language Models Outperform Retrieval-Augmentation and Fine-Tuning for Clinical Oncology. (2026). Code: https://github.com/jmfraga/medexpert-oncologia

License

Inherits the base model license (llama3.1); adapter provided for research/reproducibility.