lipidos-phi3-domain-adapt-merged

LipidOS — Phi-3.5 Mini, domain-adapted for lipid/Raman spectroscopy literature

Standalone merged model (base + QLoRA adapter, no peft required) for microsoft/Phi-3.5-mini-instruct, trained via Unsloth for domain adaptation — next-token/causal-LM objective on lipid biochemistry and Raman/IR spectroscopy abstracts, not an instruction-tuned task.

Part of LipidOS, a citation-grounded RAG system for the Raman/lipid literature where every generated claim must cite retrieved evidence it actually came from.

Training data

107,665 PubMed abstracts (two MeSH-term pools: bulk lipid biochemistry/lipidomics/membrane biophysics, and Raman/IR/vibrational spectroscopy vocabulary), trained on 64,000 of them (60.6% of one epoch, step-capped to a fixed compute budget).

Results (measured, not estimated)

Held-out perplexity on the same 2,000-example set, base vs. fine-tuned:

Citation-grounding was separately verified to survive the fine-tune: 0/3 hallucinated evidence citations across real domain questions plus a deliberate out-of-domain probe (details in the main repo's docs/solutions.md).

Usage

This model is designed to answer only through a grounded-retrieval + citation-checking pipeline — it should not be used for open-ended generation without retrieved evidence attached. See scripts/build_evidence.py and scripts/generate_finetuned_answer.py for the reference pipeline.

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("srikarjy025/lipidos-phi3-domain-adapt-merged")
tokenizer = AutoTokenizer.from_pretrained("srikarjy025/lipidos-phi3-domain-adapt-merged")

The LoRA adapter alone (120MB, requires loading the base model + peft separately) is available at srikarjy025/lipidos-phi3-domain-adapt.

This llama model was trained 2x faster with Unsloth.