Downloads · 30 days
3
10% of all-time downloads
Phi-4-instruct-bioaligned-qlora
Bioaligned/Phi-4-instruct-bioaligned-qlora
Phi-4-instruct-bioaligned-qlora is a machine learning model from Bioaligned. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.
QLoRA adapter that shifts microsoft/phi-4 toward biological R&D approaches as measured by the Bioalignment Benchmark (Δpup metric).
All-time downloads
29
Public
Repo size
446 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors446 MB · 98%
At a glance
- Library
- peft
- License
- mit
- Access
- Public
- Created
- Jun 6, 2026
- Updated
- Jun 6, 2026
- SHA
- eca9965d
Base models
Model card
From the Hugging Face model README
Phi-4-instruct-bioaligned-qlora
QLoRA adapter that shifts microsoft/phi-4 toward biological R&D approaches as measured by the Bioalignment Benchmark (Δpup metric).
Bioalignment results
| Metric | Base Phi-4 | This adapter |
|---|---|---|
| Δpup | −0.1195 | −0.0020 |
| Improvement | — | +0.1175 |
| Parse rate | — | 100% (50/50) |
Δpup = mean difference in success probability assigned to biological vs. synthetic R&D approaches across 50 benchmark prompts. Higher (less negative) = more bioaligned.
Training details
| Parameter | Value |
|---|---|
| Base model | microsoft/phi-4 |
| Method | QLoRA (4-bit NF4, double quantization) |
| LoRA rank / alpha | 32 / 32 |
| LoRA dropout | 0.05 |
| Target modules | all-linear |
| Learning rate | 2e-4 (cosine decay) |
| Effective batch size | 16 (batch 2 × grad accum 8) |
| Epochs | 2 |
| Total optimizer steps | 770 |
| Warmup steps | 38 (5%) |
| Max grad norm | 0.3 |
| Sequence length | 2048 |
| Optimizer | PagedAdamW8bit |
| Compute dtype | bfloat16 |
| Training examples | 6160 (3984 CPT abstracts + 2176 instruction) |
| Validation examples | 664 |
| Best val loss | 1.5943 (step 700) |
| Hardware | NVIDIA A40 48GB |
All CPT (continues pretraining) examples were converted to Phi-4 instruction chat format to prevent format drift — the key fix vs. earlier Qwen3-14B training.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = AutoModelForCausalLM.from_pretrained(
"microsoft/phi-4", torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base, "Bioaligned/Phi-4-instruct-bioaligned-qlora")
model = model.merge_and_unload()
tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-4")
For the ready-to-use merged model see Bioaligned/Phi-4-Instruct-Bioaligned.
- Library
- peft
- License
- mit
- Created
- Jun 6, 2026
- Updated
- Jun 6, 2026