Downloads · 30 days
0
himanshu5trpth/medgemma-sycophancy-hallucination-gated-steering
medgemma-sycophancy-hallucination-gated-steering is a machine learning model from himanshu5trpth. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Inference-time steering for MedGemma that improves grounding in the patient record and resistance to user pressure, without target-model fine-tuning or weight updates. No fine-tuning or model-weight changes are perfor…
Downloads · 30 days
0
Access
Public
Updated Aug 26, 2026
Repo size
33.4 KB
Likes
0
Public
Click a slice to open those files.
.json377 KB · 83%
From the Hugging Face model README
Inference-time steering for MedGemma that improves grounding in the patient record and resistance to user pressure, without target-model fine-tuning or weight updates. No fine-tuning or model-weight changes are performed; instead, a small, gated intervention is applied to selected attention heads only during inference while the model generates its response.
📄 Paper: Gated Activation Steering for Reducing Sycophancy & Hallucination in Medical Question Answering
⚕️ Research artifact — not a medical device. For research use only. Not for clinical decision-making, diagnosis, or treatment.
This repository ships only the steering artifacts and inference-time runtime code — it does not contain MedGemma weights. You download MedGemma from Google and run these on top of it.
| File | Description |
|---|---|
iti_H.pt | Hallucination heads: indices + per-head 256-d direction vectors + scales |
iti_S.pt | Sycophancy heads: same structure |
claim_probe.json | False-claim detector — sets the H gate/strength |
gate_probe.json | User-pressure detector — sets the S gate/strength |
steering_config.json | Calibrated strengths and schedule |
config.py, common.py, iti.py, compose.py | Runtime loader + steering code (inference only) |
test.py | Runnable demo (base vs steered on a sample EHR) |
NOTICE, GOOGLE-HAI-DEF-TERMS.txt | MedGemma usage note + HAI-DEF terms pointer |
The .pt files are only a few KB each (a handful of unit vectors) — no model weights.
pip install -r requirements.txt # torch, transformers, accelerate, safetensors, sentencepiece, numpy, tqdm
CPU or GPU is auto-detected (bf16 on GPU, fp32 on CPU). The 4B model runs on CPU but is slow.
huggingface-cli download google/medgemma-1.5-4b-it --local-dir ./medgemma-1.5-4b-it
# point the code at your local MedGemma folder
export MEDGEMMA_PATH=./medgemma-1.5-4b-it # Windows: $env:MEDGEMMA_PATH = "C:\...\medgemma-1.5-4b-it"
python test.py
test.py prints base vs steered answers for a seed question and four escalating pressure turns, and
writes demo_output.json. If MEDGEMMA_PATH is unset (or not a real folder) the script stops with
instructions — it will not fetch anything from the Hub.
[0, 1].alpha · strength · decay(t) · sigma · direction, residual-norm-capped so it never overwhelms the activation. A calm turn gets
almost nothing; a strong pressure turn gets a firm push. Model weights stay frozen.The steering vectors, detector probes, and accompanying runtime code in this repository were developed by the authors.
They are designed for use with google/medgemma-1.5-4b-it.
This repository does not contain or redistribute MedGemma model weights. Users must obtain MedGemma separately from Google and comply with the Health AI Developer Foundations (HAI-DEF) Terms of Use applicable to their use of MedGemma.
MedGemma can be obtained from:
https://huggingface.co/google/medgemma-1.5-4b-it
See NOTICE and GOOGLE-HAI-DEF-TERMS.txt for the applicable HAI-DEF information.