lipidos-phi3-domain-adapt

LipidOS — Phi-3.5 Mini, domain-adapted for lipid/Raman spectroscopy literature

QLoRA (rank 16) adapter for microsoft/Phi-3.5-mini-instruct, trained via Unsloth for domain adaptation — next-token/causal-LM objective on lipid biochemistry and Raman/IR spectroscopy abstracts, not an instruction-tuned task.

Part of LipidOS, a citation-grounded RAG system for the Raman/lipid literature where every generated claim must cite retrieved evidence it actually came from.

Training data

107,665 PubMed abstracts (two MeSH-term pools: bulk lipid biochemistry/lipidomics/membrane biophysics, and Raman/IR/vibrational spectroscopy vocabulary), trained on 64,000 of them (60.6% of one epoch, step-capped to a fixed compute budget).

Results (measured, not estimated)

Held-out perplexity on the same 2,000-example set, base vs. fine-tuned:

Citation-grounding was separately verified to survive the fine-tune: 0/3 hallucinated evidence citations across real domain questions plus a deliberate out-of-domain probe (details in the main repo's docs/solutions.md).

Usage

This model is designed to answer only through a grounded-retrieval + citation-checking pipeline — it should not be used for open-ended generation without retrieved evidence attached. See scripts/build_evidence.py and scripts/generate_finetuned_answer.py for the reference pipeline.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3.5-mini-instruct")
model = PeftModel.from_pretrained(base, "srikarjy025/lipidos-phi3-domain-adapt")
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3.5-mini-instruct")

A standalone merged version (no peft required) is available at srikarjy025/lipidos-phi3-domain-adapt-merged.

This llama model was trained 2x faster with Unsloth.