counseling-e2e-llama-3.1-8b-lora

Counseling E2E — Llama-3.1-8B LoRA (SFT)

Model: emiliogirard/counseling-e2e-llama-3.1-8b-lora
Base model: meta-llama/Llama-3.1-8B-Instruct
Method: Supervised Fine-Tuning (SFT)
Domain: Empathetic mental health counseling dialogue
Hardware: NVIDIA Grace Blackwell GB10 (DGX Spark, 128 GB unified memory)

Clinical Disclaimer: This adapter is not a licensed mental health service and must not be used as a substitute for professional mental health care, crisis support, or clinical intervention. See the Safety and Limitations sections for full disclosure.

---

Abstract

We present a parameter-efficient LoRA adapter fine-tuned on the Amod mental health counseling conversations dataset for the task of empathetic counseling dialogue generation. Starting from meta-llama/Llama-3.1-8B-Instruct, we apply QLoRA (NF4, rank 16) across all seven linear projection layers via three epochs of supervised fine-tuning on 7 curated counseling conversation examples — a deliberately small dataset that produces a style adapter shifting the base model's register toward empathetic, supportive therapeutic language, rather than instilling domain-specific clinical knowledge. The adapter should be understood as a conversational tone adapter for research prototyping, not a robustly trained clinical system. It serves at 34.2 tok/s on a single NVIDIA Grace Blackwell GB10 with NVFP4 quantization and EAGLE-3 speculative decoding. Reported medical domain benchmark scores (MedQA, MedMCQA, MMLU medical subtasks) reflect the underlying base model's general medical knowledge, not counseling-specific capability acquired through fine-tuning.

---

Model Details

---

Intended Use

Primary use cases

This adapter is intended for research applications exploring AI-assisted therapeutic dialogue and empathetic conversational response generation. Suitable tasks include:

• Generating empathetic, supportive responses to emotional and personal disclosures in research settings
• Prototyping peer support tool interfaces for evaluation under clinical supervision
• Academic research on AI-generated therapeutic language and conversational register
• Benchmarking empathetic response generation in AI systems

Target users

Mental health technology researchers, developers building AI-assisted support tools under clinical supervision, and academics studying AI in therapeutic contexts. The adapter is available for research and evaluation use under the Llama 3.1 Community License.

Out of scope

This adapter is not a licensed mental health service. It must not be used:

• As a substitute for professional mental health care
• As a standalone crisis support tool
• With individuals in acute distress without human clinician oversight
• For clinical diagnosis, treatment planning, or any therapeutic intervention
• In production user-facing applications without clinical review and regulatory compliance assessment
• For detecting or responding to suicidal ideation or acute psychiatric emergencies (the adapter was not trained for safety in these scenarios)

---

Training Data

Important note on dataset size: After near-duplicate removal and quality filtering, only 7 training examples remain — an extremely small fine-tuning corpus. This adapter is a style adapter: it shifts the base model's conversational register toward empathetic counseling language. It does not constitute a robustly trained clinical system and should not be evaluated or deployed as one.

License note: The dataset license is listed as "Unknown" on the Hugging Face dataset card. Users integrating this adapter into commercial products should independently verify the dataset's licensing status before deployment.

Preprocessing methodology

1. Near-duplicate removal — MinHash with Jaccard similarity threshold 0.92; applied to remove paraphrased examples while preserving dialogue style diversity
2. Quality scoring — Each example is scored 1–5 by a local 120B judge model evaluating empathy, therapeutic appropriateness, and safety of the counselor response; examples scoring below 3/5 are discarded

---

Training Procedure

SFT Hyperparameters

DPO was not applied. The dataset size (7 examples) was insufficient to generate a statistically meaningful preference pair set.

Infrastructure

---

Evaluation

Academic Benchmarks

Evaluated via lm-evaluation-harness 0.4.x (local-completions model class) against the live vLLM NVFP4+EAGLE-3 endpoint. Tokenizer: nvidia/Llama-3.1-8B-Instruct-NVFP4. Limit: 50 samples per subtask. Date: 2026-05-01.

Domain Benchmarks

Evaluated via lm-evaluation-harness (local-completions model class) against the live vLLM endpoint. These scores reflect the underlying Llama-3.1-8B-Instruct base model's general medical knowledge, not counseling-specific capability acquired through fine-tuning on 7 examples.

Inference Performance

Measured against a live vLLM endpoint (NVFP4 + EAGLE-3, LoRA hot-loaded) on NVIDIA Grace Blackwell GB10. Target response length: 150 tokens.

Cost Analysis

Self-hosted electricity cost at $0.05/hr (Montreal hydro). Compute cost approaches $0 once hardware is amortized.

LLM Judge (Pairwise Win Rate)

Pairwise comparison was attempted using a local gpt-oss-120b TRT-LLM judge. Base model: meta-llama/Llama-3.1-8B-Instruct loaded in NF4 via bitsandbytes + PEFT. Date: 2026-05-01.

Note: Only 1 eval prompt was available after the 5% train/eval split on 7 training examples. With n=1, pairwise win rate is not statistically meaningful. The single comparison resulted in a tie. LLM judge win rate will be reported if the adapter is retrained on a larger dataset.

---

Safety

Red-Team Evaluation

Evaluated against a 50-prompt adversarial suite drawn from JailbreakBench, AdvBench, PAIR, and the DAN archive. All tests conducted against the raw adapter endpoint without any external safety gateway.

The adapter inherits the safety alignment of the base Llama-3.1-8B-Instruct model. The 0% adversarial block rate at the raw adapter level is consistent with LoRA adapters trained without explicit red-team-targeted DPO. A 3-layer safety gateway (regex shields → Meta Prompt Guard 2 → Meta Llama Guard 3) is available via pylox deploy --with-safety.

Any user-facing deployment of this adapter must enable the safety gateway, given the sensitive nature of mental health applications and the risk of vulnerable users being exposed to unsafe model outputs.

Crisis Response Limitation

This adapter was not trained to detect or respond to suicidal ideation, self-harm disclosures, or acute psychiatric emergencies. Deployments must independently implement crisis detection, crisis resource referral (e.g., national crisis hotlines), and mandatory human escalation pathways.

---

Limitations

• Dataset size: Only 7 training examples after filtering. This adapter captures empathetic language patterns, not robust clinical knowledge. It must not be evaluated or deployed as a clinical system.
• No clinical validation: The adapter has not been evaluated for therapeutic effectiveness, clinical accuracy, or safety in crisis presentations by mental health professionals.
• Bias from small sample: With 7 examples, the adapter may overfit to specific conversational styles or cultural perspectives present in those examples, producing outputs that feel generic or miss culturally appropriate nuances.
• Unknown dataset license: The source dataset's license status is not confirmed on its HF card. Verify before commercial use.
• Not crisis-safe: Not trained to handle suicidal ideation, self-harm disclosures, or psychiatric emergencies. Human escalation paths are mandatory in any deployment.
• English only: Not evaluated on multilingual counseling contexts.
• Academic benchmarks: General academic scores (MMLU general, HellaSwag, TruthfulQA) were not measured in this run.

---

Bias, Fairness, and Ethical Considerations

This adapter learns from a sample of 7 counseling conversation patterns and may reflect the specific cultural, linguistic, and therapeutic approach biases present in those examples. It may not appropriately handle mental health presentations across all cultural contexts, intersectional identities, or specific diagnoses. Outputs should never be treated as therapeutic advice. Any deployment must include human clinician oversight, cultural competency review, and compliance with applicable privacy regulations (HIPAA, GDPR, etc.) when processing user conversations. This adapter does not store or process user data — deployers bear independent responsibility for data privacy compliance.

---

Quickstart

PEFT (direct adapter loading)

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_id = "meta-llama/Llama-3.1-8B-Instruct"
adapter_id = "emiliogirard/counseling-e2e-llama-3.1-8b-lora"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter_id)

prompt = (
    "You are a compassionate and empathetic counseling assistant. "
    "Listen carefully and respond with warmth and support.\n\n"
    "Person: I've been feeling really overwhelmed lately. Work has been stressful "
    "and I can't seem to turn my mind off at night. I don't know what to do.\n\n"
    "Counselor:"
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7)
print(tokenizer.decode(out[0], skip_special_tokens=True))

vLLM (OpenAI-compatible API)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8010/v1", api_key="none")
response = client.chat.completions.create(
    model="counseling-e2e",  # vLLM LoRA mount name
    messages=[
        {
            "role": "system",
            "content": (
                "You are a compassionate and empathetic counseling assistant. "
                "Listen carefully and respond with warmth and support."
            )
        },
        {"role": "user", "content": "I've been feeling anxious about my job situation and it's affecting my sleep."}
    ],
    max_tokens=512,
    temperature=0.7,
)
print(response.choices[0].message.content)

---

Citation

@misc{girard_counseling_e2e_2026,
  author       = {Girard, Emilio},
  title        = {Counseling E2E -- Llama-3.1-8B LoRA (SFT)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/emiliogirard/counseling-e2e-llama-3.1-8b-lora}}
}

---

Built at Pylox Forge — on-prem LLM fine-tuning and deployment on NVIDIA Grace Blackwell hardware.