victor-ramirez-7b-lora

Victor Ramirez 7B LoRA Adapter

A ~10 MB LoRA adapter that teaches a Qwen2.5-7B base model to answer professional / technical questions in Victor Ramirez's register. It learns phrasing, tone and structure only -- a companion RAG system supplies every fact at inference. The goal: replace a 70B model in the AI-Vic chatbot with a 7B base + this adapter at a fraction of the cost, without losing answer quality.

• Developed by: Victor Ramirez
• Adapter version: v0.4.0
• Model date: 2026-09-07
• Base model: Qwen/Qwen2.5-7B-Instruct -- Apache-2.0, ungated, no access request
• License: Apache-2.0, inherited from the base model.
• Frameworks: transformers, PEFT, PyTorch

Training data

High-scoring (instruction, context, output) triples from the AI-Vic evaluation loop, scored by an LLM-as-judge (@cf/meta/llama-3.1-8b-instruct-fast) and kept only when:

• relevance >= 4/5 and groundedness >= 4/5 (judge scores)
• retrieved RAG context is present (the reply had to be grounded in something)

Exported by .github/workflows/export-training-data.yml.

Training procedure

QLoRA: the base model is loaded in 4-bit (NF4) and frozen; only the adapter weights train. Loss is computed on the reply tokens only (prompt tokens masked to -100), so the model learns to answer, not to echo the prompt.

The best checkpoint (lowest eval loss, not the final epoch) is reloaded before saving. Best eval loss this run: 1.046. Trained on a Colab free-tier Tesla T4 in roughly 10-15 minutes.

Evaluation

The adapter is not scored in isolation -- it is evaluated inside AI-Vic against the current production path (retrieval + a larger model):

• Quality: relevance + groundedness (same LLM judge) on the eval-split hold-out and the nightly hard-case suite
• Cost / latency: 7B + adapter vs. the 70B call

Step 8 of the training notebook runs a base-vs-tuned judge comparison on the held-out rows and prints a pass/fail (tuned must match or beat base on both axes). Production A/B numbers live in ai-vic-chatbot/docs/evaluation.md; treat any figure not from a real run as pending.

How to use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

BASE = "Qwen/Qwen2.5-7B-Instruct"
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.float16)

base = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, "VHRamirez/victor-ramirez-7b-lora")
tok = AutoTokenizer.from_pretrained(BASE)

# The adapter expects the ChatML prompt format it was trained on:
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "<question>\n\nContext: <retrieved chunks>"}],
    add_generation_prompt=True, tokenize=False,
)
ids = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
print(tok.decode(
    model.generate(**ids, max_new_tokens=256, do_sample=True, temperature=0.7)[0][ids["input_ids"].shape[1]:],
    skip_special_tokens=True,
))

For a single standalone model, model.merge_and_unload() then save_pretrained / push. For a Hugging Face Inference Endpoint, point it at the base model and attach this adapter. Cloudflare Workers AI does not load HF adapters automatically -- it needs its own LoRA upload against a supported base model (see the Workers AI LoRA docs).

Limitations

• 66 examples is the floor for LoRA. It learns voice, not facts -- never run it without a RAG context supplier; with no context you get generic Qwen2.5 output.
• Style transfer only. It does not add knowledge or improve reasoning.
• Domain-specific. Trained on questions about Victor's background and work; quality drops sharply off-topic.
• Adapter only (~10 MB) -- you need Qwen/Qwen2.5-7B-Instruct to use it.
• Non-deterministic at temperature > 0; use temperature 0 for reproducible evals.
• Bias: reflects one person's professional perspective and a small automated judge's preferences.

Citation

@misc{ramirez2026victor7blora,
  title        = {Victor Ramirez 7B LoRA Adapter},
  author       = {Ramirez, Victor},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/VHRamirez/victor-ramirez-7b-lora}}
}

Training notebook: docs/phase-4b-colab-template.ipynb · Repo: ramirez-ai-labs/ai-vic-chatbot