Gemma-2b-it-Psych

Model Card for Gemma-2b-it-Psych

Model Summary

Gemma-2b-it-Psych is a domain-adapted version of google/gemma-2b-it, fine-tuned using LoRA on an instruction-based psychology dataset.
The model is optimized to generate empathetic, supportive, and professionally aligned psychological responses, primarily for educational and research purposes.

This repository contains LoRA adapters only. The base model must be loaded separately.

---

Model Details

Model Description

• Author: Ederson Corbari ([email protected])
• Date: February 01, 2026
• Model type: Causal Language Model (LLM)
• Language(s): English
• License: Same as base model (google/gemma-2b-it)
• Finetuned from model: google/gemma-2b-it
• Fine-tuning method: LoRA / QLoRA (parameter-efficient fine-tuning)

This model was fine-tuned using instruction–response pairs focused on psychological support.
Only empathetic and therapeutically appropriate responses were retained during training, while judgmental or aggressive alternatives were excluded.

---

Model Sources

• Hugging Face Repository: https://huggingface.co/ecorbari/Gemma-2b-it-Psych
• GitHub Repository: https://github.com/edersoncorbari/fine-tune-llm
• Base Model: https://huggingface.co/google/gemma-2b-it

---

Uses

Direct Use

This model is intended for:

• Research and experimentation with instruction-tuned LLMs
• Educational demonstrations of LoRA fine-tuning
• Prompt engineering and behavioral analysis in psychology-related domains

The model requires the base Gemma-2B weights to be loaded together with the LoRA adapters.

---

Downstream Use

• Further fine-tuning on related domains
• Adapter merging to create a standalone model
• Quantization for efficient local inference (e.g., GGUF formats)

---

Out-of-Scope Use

• Clinical diagnosis or treatment
• Real-world mental health interventions without professional supervision
• High-stakes decision-making
• Autonomous counseling systems

---

Bias, Risks, and Limitations

• The model may generate inaccurate or incomplete information.
• It does not replace licensed mental health professionals.
• Responses may reflect biases present in the training data.
• Empathy does not guarantee correctness or safety in all contexts.

---

Recommendations

Users should apply human oversight, especially in sensitive scenarios.
This model is best suited for research, learning, and proof-of-concept applications.

---

How to Get Started with the Model

import torch

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = "google/gemma-2b-it"
adapter_model = "ecorbari/Gemma-2b-it-Psych"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
    base_model, dtype=torch.float16, device_map="auto"
)

model = PeftModel.from_pretrained(model, adapter_model)

prompt = "How can I cope with anxiety during stressful situations?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=150)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Evaluation

Testing Data

The model was evaluated on a held-out validation split of the psychology instruction dataset used during fine-tuning.

---

Metrics

The following metrics were used to evaluate the model during training:

• Cross-entropy loss (per token)
• Perplexity (exp(loss))

---

Results

Perplexity was computed as the exponential of the evaluation loss. Lower values indicate higher confidence in next-token prediction.

These metrics reflect convergence and generalization within the target domain, but do not directly assess clinical correctness or psychological safety.