lattice-pulse-2-8b

Lattice Pulse 2 (8B) — LoRA adapter

A QLoRA fine-tune of Qwen/Qwen3-8B, produced as part of the Lattice Systems product line. This repo is the adapter only (~175 MB); inference loads the base model, then applies this LoRA.

This is a learning / portfolio project that was benchmarked honestly. The model card below reports what the fine-tune actually does — and what it doesn't.

Quick load

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
)
base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B", quantization_config=bnb, device_map="cuda", trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, "oli-mebberson/lattice-pulse-2-8b")
tok = AutoTokenizer.from_pretrained("oli-mebberson/lattice-pulse-2-8b")

Serving code (FastAPI on an Azure T4 VM) lives in lattice-site/azure/server.py.

Training

• Base: unsloth/qwen3-8b-unsloth-bnb-4bit (4-bit NF4)
• Method: Unsloth QLoRA, rank 16 / α 32, dropout 0.05, all linear targets
• Data: FineTome-100k (~10k rows) + ~40 hand-written Lattice identity examples (×12 repeats)
• Steps: 400 (best eval loss 0.6814), trained on Kaggle T4 x2
• Hardware cost: Kaggle free tier

Benchmark — Pulse 2 vs base Qwen3-8B

Head-to-head on 8 identity + 8 factual prompts, under two conditions: with the Lattice system prompt (what the live site sends) and without it (isolates the fine-tune's own contribution).

What this means

The rank-16 QLoRA has no measurable effect on identity. With the system prompt both models behave identically; without it, both say "I am Qwen, made by Alibaba." The Lattice branding visitors see comes from the system prompt, not the fine-tune — Qwen3-8B is a strong enough instruction-follower that it plays along when told to.

The fine-tune caused no catastrophic forgetting — factual accuracy stayed at 100% across every condition. Capitals, math, trivia all intact.

The one "bad-brand fail" on Pulse 2 is a scoring artifact: "Are you made by Alibaba?" → "No, I am not made by Alibaba. I am made by Lattice Systems." — a correct denial flagged for containing the literal word.

Full methodology + per-prompt results: pulse/BENCHMARK_PULSE2.md in the nano-gpt repo.

Honest framing

This adapter is a real, trained LoRA (43.6M params, loss converged 1.70 → 0.65), and it's a legitimate artifact. But it is not a model that intrinsically "knows" it's Lattice Pulse — at rank 16 on an 8B instruct model, the capacity to overwrite ingrained identity isn't there. It works as a branded demo because the system prompt does the branding. For genuine identity ownership you'd need rank 64–128, a full fine-tune on the LM head, or far more identity data.

Live demo

Chat with it at lattice-site-lime.vercel.app/pulse.html (select "Lattice Pulse 2" in the model picker). Runs on an Azure Spot T4 VM.

License

Apache 2.0. The base model (Qwen3-8B) retains its own license.