Downloads · 30 days
6
10% of all-time downloads
Lekhansh/gemma-3n-e4b-psychiatry-translation-indic-en
gemma-3n-e4b-psychiatry-translation-indic-en is a text generation model from Lekhansh. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
This is a LoRA (Low-Rank Adaptation) adapter for Google's Gemma-3n-E4B-It model, fine-tuned for translating psychiatric interview transcripts from Hindi and Kannada to English while preserving clinical context and con…
Downloads · 30 days
6
10% of all-time downloads
All-time downloads
63
Public
Repo size
507 MB
Likes
1
Public
Click a slice to open those files.
.pt308 MB · 61%
From the Hugging Face model README
This is a LoRA (Low-Rank Adaptation) adapter for Google's Gemma-3n-E4B-It model, fine-tuned for translating psychiatric interview transcripts from Hindi and Kannada to English while preserving clinical context and conversational nuance.
This model is behind a gated repository for important safety reasons:
Since this model was trained on real-world clinical psychiatric conversations, there is an anticipated risk that model outputs may contain sensitive content related to mental health topics. This model is intended solely for research purposes and clinical translation tasks by qualified teams.
To request access: Please contact the repository owner with:
The fine-tuned model shows significant improvements in translation quality over the base Gemma-3n-E4B-It model across both Hindi and Kannada psychiatric interview transcripts.
| Metric | Baseline (Mean ± SD) | Fine-tuned Model (Mean ± SD) | Improvement |
|---|---|---|---|
| METEOR Score | 0.5709 ± 0.2719 | 0.7018 ± 0.2668 | +22.93% absolute |
| TER (Translation Error Rate) | 97.36 ± 167.26 | 74.26 ± 149.36 | -23.73% reduction |
Hindi (1,854 test sentences):
Kannada (2,285 test sentences):
All reported improvements are statistically significant, indicating substantial enhancement in translation quality across both languages.
This model uses a carefully designed prompt that ensures clinically accurate translations while preserving conversational nuance. The prompt emphasizes:
Meaning Over Literal Translation
Conversational Style Preservation
Verbatim Accuracy
[pauses], [loudly], etc.Clinical Context Preservation
Prompt File: prompts/translation_prompt.txt
The prompt template uses {language} placeholder which is replaced with "Hindi" or "Kannada" during inference.
The model was fine-tuned on aligned English-Hindi and English-Kannada sentence pairs extracted from psychiatric interview transcripts.
Training Set (35,802 aligned sentences):
Test Set (4,139 aligned sentences):
Total Dataset: 39,941 aligned sentence pairs across both languages
All data comes from real psychiatric interviews and therapy sessions, providing domain-specific translation patterns crucial for clinical accuracy.
The model requires the translation prompt for optimal performance. This prompt is included in the prompts/ directory:
prompts/translation_prompt.txtThis prompt contains detailed instructions for preserving clinical context, conversational style, and emotional nuance while translating psychiatric content.
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
# Load base model and tokenizer
model_id = "google/gemma-3n-e4b-it"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# Load LoRA adapter
model = PeftModel.from_pretrained(
model,
"Lekhansh/gemma-3n-e4b-psychiatry-translation-indic-en"
)
# Load the translation prompt
with open("prompts/translation_prompt.txt", "r") as f:
prompt_template = f.read()
# Example: Translating from Hindi
source_language = "Hindi" # or "Kannada"
source_text = """डॉक्टर: आपकी नींद कैसी है?
मरीज: बहुत बुरी है, रात भर नहीं आती।"""
# Format the prompt
prompt = prompt_template.replace("{language}", source_language)
prompt = prompt + "\n" + source_text
# Prepare messages
messages = [
{
"role": "system",
"content": [
{
"type": "text",
"text": "You are an expert translator specializing in psychiatric interviews.",
}
],
},
{
"role": "user",
"content": [{"type": "text", "text": prompt}]
}
]
# Apply chat template and generate
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_tensors="pt",
)
inputs = inputs.to(model.device)
with torch.inference_mode():
generation = model.generate(
inputs,
max_new_tokens=2048,
temperature=0.7,
top_k=50,
top_p=0.95,
do_sample=True
)
translation = tokenizer.decode(generation[0], skip_special_tokens=True)
# Extract only the assistant's response
translation = translation.split("<start_of_turn>model")[-1].strip()
print(translation)
For faster inference, you can merge the adapter weights with the base model:
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
# Load base model and adapter
base_model = AutoModelForCausalLM.from_pretrained(
"google/gemma-3n-e4b-it",
torch_dtype=torch.bfloat16
)
tokenizer = AutoTokenizer.from_pretrained("google/gemma-3n-e4b-it")
model = PeftModel.from_pretrained(
base_model,
"Lekhansh/gemma-3n-e4b-psychiatry-translation-indic-en"
)
# Merge and unload
merged_model = model.merge_and_unload()
# Save merged model
merged_model.save_pretrained("./merged_translation_model")
tokenizer.save_pretrained("./merged_translation_model")
The fine-tuned model demonstrates substantial improvements:
Examples exceeding 5000 tokens combined length were filtered out to ensure training stability and efficiency.
This adapter is released under the Apache 2.0 license. However, users must also comply with Google's Gemma license and any applicable regulations regarding clinical data and patient privacy.
If you use this model in your research, please cite:
@misc{gemma-psychiatry-translation-indic,
title={Gemma-3n-E4B-It LoRA Adapter for Psychiatric Translation: Hindi/Kannada to English},
author={Lekhansh Shukla, Prakrithi Shivaprakash},
year={2025},
publisher={HuggingFace},
howpublished={\\url{https://huggingface.co/Lekhansh/gemma-3n-e4b-psychiatry-translation-indic-en}}
}
For access requests or questions about this model, please contact Dr Lekhansh Shukla @ [email protected].
Disclaimer: This model is provided for research purposes only. Users are responsible for ensuring compliance with all applicable laws, regulations, and ethical guidelines when using this model, particularly regarding patient privacy, clinical data handling, and professional translation standards in healthcare settings.