Holo-3.1-0.8B-lora-demo

Holo-3.1-0.8B — Atlas LoRA demo adapter

A tiny, deliberately-overfit demo LoRA for Hcompany/Holo-3.1-0.8B, trained to prove out the LoRA runtime in the Atlas pure-Rust CUDA inference engine end-to-end (train → PEFT export → serve → applied delta), with exact parity between Atlas and peft/transformers.

It teaches a fixed persona + a made-up "codeword" the base model cannot know, so the adapter's effect is unambiguous at batch size 1.

Atlas serving constraints (why the config looks the way it does)

Atlas LoRA v0 applies the BF16 delta at attention k/v/o on the full-attention layers only. This adapter is shaped to match:

• target_modules = ["k_proj", "v_proj", "o_proj"] — q_proj is gated on Holo and rejected; dense-FFN delta is not yet wired.
• layers_to_transform = [3, 7, 11, 15, 19, 23] — the 6 full-attention layers (the other 18 are Gated-DeltaNet/linear; Atlas hard-rejects a LoRA tensor on them).
• r = 32, lora_alpha = 64 (scale = α/r = 2.0), use_rslora = false, use_dora = false, bias = "none".

Rank 32 (rather than a minimal 8) is used so the exact codeword digits survive the BF16-delta-on-NVFP4-base quantization mismatch when served by Atlas.

Usage

PEFT:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Hcompany/Holo-3.1-0.8B", trust_remote_code=True)
model = PeftModel.from_pretrained(base, "MonumentalSystems/Holo-3.1-0.8B-lora-demo")

Atlas (requires the F32-adapter loader fix — adapter.rs F32→BF16 conversion):

spark serve Hcompany/Holo-3.1-0.8B \
  --lora-adapter demo=MonumentalSystems/Holo-3.1-0.8B-lora-demo \
  --max-lora-rank 64

Training

peft + transformers Trainer, ~10 epochs on 6 overfit Q→A pairs, r=32, lora_alpha=64, lr 2e-4, bf16, ~4 min on a single NVIDIA GB10 (Grace-Blackwell, aarch64, CUDA 13). This is a demo artifact, not a general-purpose fine-tune.