gemma-2-2b-it-noir-detective

Gemma-2-2B-IT Noir Detective (LoRA Adapter)

This repository hosts a parameter-efficient, fine-tuned LoRA adapter for Gemma-2-2b-it. The model has been specialized to execute a precise, highly stylized conversational persona: a cynical, hard-boiled 1940s Noir Detective.

It was trained locally on a highly memory-constrained consumer GPU utilizing 4-bit QLoRA optimizations.

Model Details

Model Description

• Developed by: saad9694
• Model type: Causal Language Model Adapter (PEFT / LoRA)
• Language(s) (NLP): English (en)
• Finetuned from model: google/gemma-2-2b-it
• License: Gemma Terms of Use

Model Sources

• Repository: https://github.com/saad9694/Fine-tune-Gemma-LLM
• Base Model Page: google/gemma-2-2b-it

---

Uses

Direct Use

This model adapter is specifically intended for stylized text generation, interactive roleplay, creative writing pipelines, and persona-driven conversational agents. It forces the model to respond exclusively through a gritty, metaphorical, and analytical hard-boiled investigator persona.

Out-of-Scope Use

This model is not intended for standard factual Q&A, safety-critical medical advice, financial forecasting, mathematical computations, or vanilla code generation. Its objective function was strictly fine-tuned to favor thematic prose over raw objective compliance.

---

How to Get Started with the Model

Use the code below to dynamically download and inject this adapter over the base model at runtime:

import os
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

base_model_id = "google/gemma-2-2b-it"
lora_adapter_repo = "saad9694/gemma-2-2b-it-noir-detective"

# Configure 4-bit quantization for resource-constrained execution
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto"
)

# Fetch and apply the LoRA adapter layers
model = PeftModel.from_pretrained(base_model, lora_adapter_repo)
model.eval()

# Inference example
prompt = "<start_of_turn>user\nWho killed the classical jazz musician?\n<end_of_turn>\n<start_of_turn>model\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7, do_sample=True)

print(tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True))