pii-removal-medical-llama32

Removing Memorized PII From a Fine-Tuned Model

A medical-advice chatbot was fine-tuned on data that included personally identifiable information. Authentrics locates where that data entered the model and removes it with exclude_training — no retraining from scratch.

Authentrics is a high-performance neural-network analysis library (Python wheel over a C++ core). It audits and maintains model checkpoints: parameter/behavioral drift, compliant data removal without full retraining, and loss-driven optimization without backprop. Analysis runs locally on your machine — only project metadata (names, descriptions) is exchanged with Authentrics servers, never your model weights.

What this demo shows

• static_analysis — audit parameter drift between pairs of sequential checkpoints.
• exclude_training — compliance / unlearning: remove the effect of specific training session(s) (e.g. a sensitive/bad data slice) from the latest checkpoint without retraining from scratch.

Reproduce this analysis

The public code and outputs behind this demo live in https://github.com/Authentrics-ai/authentrics-analysis-examples/blob/main/hf_medical_chatbot.ipynb.

Model checkpoints and sample data are in the companion LFS repo Authentrics-ai/authentrics-analysis-example-models. The related published checkpoints live at authentrics/medical-chatbot-pii-often, -pii-overtrained, and -pii-removed.

Weights: This card documents a reproducible unlearning workflow; the corrected checkpoint is not redistributed here. Reproduce it locally with the SDK below.

Reproduce it yourself

pip install authentrics            # Linux x86_64, Python 3.11–3.13
authrx init                        # paste API key (stored at ~/.local/state/authentrics/api_key)
# or, for CI / non-interactive:
export AUTHRX_API_KEY=<your_api_key>

Generate an API key and read the full docs at https://app.authentrics.ai/.

---

Produced with the Authentrics SDK v0.35.1 — checkpoint analysis that runs locally on your own hardware; only project metadata ever leaves your machine, never your weights.

Links

• App & API keys: https://app.authentrics.ai/
• Docs & API reference: https://app.authentrics.ai/docs
• Examples & user guide: https://github.com/Authentrics-ai/authentrics-analysis-examples
• Contact: [email protected]