Downloads · 30 days
11
9% of all-time downloads
coldstone7/mistral-7b-kairos-mental-health
mistral-7b-kairos-mental-health is a text generation model from coldstone7. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
Fine-tuned Mistral-7B-Instruct-v0.2 for empathetic mental health counseling conversations using LoRA (Low-Rank Adaptation).
Downloads · 30 days
11
9% of all-time downloads
All-time downloads
118
Public
Repo size
336 MB
Likes
0
Public
Click a slice to open those files.
.safetensors336 MB · 99%
From the Hugging Face model README
Fine-tuned Mistral-7B-Instruct-v0.2 for empathetic mental health counseling conversations using LoRA (Low-Rank Adaptation).
This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.2 trained on the MentalChat16K dataset for mental health counseling applications.
Key Features:
| Parameter | Value |
|---|---|
| Base Model | Mistral-7B-Instruct-v0.2 |
| Training Method | LoRA (PEFT) |
| LoRA Rank | 32 |
| LoRA Alpha | 16 |
| LoRA Dropout | 0.05 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Epochs | 3 |
| Batch Size | 1 (micro) × 4 (gradient accumulation) = 4 (effective) |
| Learning Rate | 2e-4 |
| LR Scheduler | Cosine |
| Optimizer | AdamW |
| Max Sequence Length | 2048 tokens |
| Weight Decay | 0.01 |
| Warmup Ratio | 3% |
| FP16 | Yes |
| Gradient Checkpointing | Yes |
pip install transformers peft torch
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
"mistralai/Mistral-7B-Instruct-v0.2",
torch_dtype=torch.float16,
device_map="auto",
)
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
tokenizer.pad_token = tokenizer.eos_token
# Load LoRA adapters
model = PeftModel.from_pretrained(
base_model,
"Karios-AI/mistral-7b-mental-health-coach", # Update with actual repo name
torch_dtype=torch.float16,
)
# Merge for faster inference (optional)
model = model.merge_and_unload()
def generate_response(user_message):
# System message
system_message = "You are a helpful mental health counselling assistant. You provide empathetic, thoughtful responses to help people explore their feelings and experiences. You avoid pathologizing language and instead help people understand their patterns and emotions."
# Format conversation
messages = [
{"role": "system", "content": system_message},
{"role": "user", "content": user_message},
]
# Apply chat template
prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
# Tokenize and generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.7,
top_p=0.9,
do_sample=True,
pad_token_id=tokenizer.eos_token_id,
)
# Decode response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
# Extract assistant's response
if "[/INST]" in response:
response = response.split("[/INST]")[-1].strip()
return response
# Example usage
user_input = "I'm feeling really anxious about an upcoming presentation at work."
response = generate_response(user_input)
print(response)
User: "I'm feeling really anxious about an upcoming presentation at work."
Assistant: "It's completely normal to feel anxious about giving a presentation, especially if it's a new experience for you. It's important to remember that everyone feels nervous in situations like this. Let's explore some coping strategies that can help you manage your anxiety and feel more confident in your presentation skills."
The model demonstrates:
| Scenario | Quality | Notes |
|---|---|---|
| Presentation anxiety | Excellent | Empathetic, normalizing, actionable |
| Social media comparison | Excellent | Reframes perspective, concrete advice, appropriate boundaries |
| Relationship conflict | Good | Empathetic reflection, exploratory approach |
To assess the model's general reasoning capabilities after specialized fine-tuning, we evaluated on the MuSR (Multistep Soft Reasoning) benchmark - a challenging dataset testing complex logical reasoning across three domains.
Benchmark Details:
Results:
| Sub-Task | Description | Accuracy | Correct/Total |
|---|---|---|---|
| Murder Mysteries | Deductive reasoning from 1000-word narratives | 56.80% | 142/250 |
| Object Placements | Spatial reasoning and object tracking | 37.50% | 96/256 |
| Team Allocation | Constraint satisfaction and logic puzzles | 13.60% | 34/250 |
| Overall | - | 35.98% | 272/756 |
Interpretation:
This benchmark tests general logical reasoning rather than conversational or therapeutic capabilities. The results demonstrate that:
✅ Specialized but not narrow: The model retains meaningful general reasoning capabilities despite being fine-tuned exclusively on mental health counseling conversations
✅ Strong narrative comprehension: 56.80% accuracy on murder mysteries (complex 1000-word narratives) shows the model can parse and reason about long-form text
⚠️ Domain-specific optimization: Performance on abstract logic tasks (object placement, team allocation) is lower, reflecting the model's specialization toward empathetic conversation rather than formal reasoning
✅ Appropriate for intended use: For a model optimized for therapeutic conversation (not general reasoning), maintaining 36% accuracy on complex reasoning tasks indicates it hasn't catastrophically forgotten general capabilities
Context:
Note: This benchmark is not representative of the model's primary use case (mental health counseling). For therapeutic quality evaluation, see the qualitative assessment above.
⚠️ Important: This model is designed for supportive counseling conversations, NOT for:
Users should:
If you use this model, please cite:
@misc{mistral-7b-kairos-mental-health,
title={Mistral-7B KAIROS Mental Health Counseling Model},
author={coldstone7},
year={2026},
publisher={Hugging Face},
howpublished={\url{https://huggingface.co/Karios-AI/mistral-7b-mental-health-coach}}
}
Apache 2.0 (same as base Mistral-7B model)
For questions or feedback: