opt-mlops-merged

opt-mlops-merged

facebook/opt-125m fine-tuned with a LoRA adapter (atulkrs/opt-mlops-lora) and fully merged into base weights via PeftModel.merge_and_unload().

The adapter deltas are baked in — no PEFT dependency needed at inference time.

Load (full precision)

from transformers import AutoModelForCausalLM, AutoTokenizer

model     = AutoModelForCausalLM.from_pretrained("atulkrs/opt-mlops-merged")
tokenizer = AutoTokenizer.from_pretrained("atulkrs/opt-mlops-merged")

Load in 4-bit with BitsAndBytes (recommended for GPU inference)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",           # NormalFloat4 — from QLoRA paper
    bnb_4bit_compute_dtype=torch.float16,
)

model = AutoModelForCausalLM.from_pretrained(
    "atulkrs/opt-mlops-merged",
    quantization_config=bnb_config,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("atulkrs/opt-mlops-merged")

Tip: swap torch.float16 for torch.bfloat16 on Ampere+ GPUs (A100, RTX 30xx+) for better numerical stability at no speed cost.

Size & load-time benchmark

4-bit load time benchmark requires Linux + CUDA + bitsandbytes; estimated load time on GPU is typically 2–5s for a 125M model.

Merge details

Why merge?

Merging removes the adapter overhead entirely — no extra matrix multiplications at inference, no PEFT dependency, and the weights load like any standard transformers checkpoint. The only trade-off is that you can no longer swap adapters without re-loading the base model.