tinyllama-lora-anonymizer

TinyLlama LoRA — Reversible PII Anonymizer

LoRA adapters that turn TinyLlama-1.1B into a local privacy filter: it removes personally identifiable information from text and returns a structured JSON object containing both the anonymized text and a full mapping of every replaced token.

Because the mapping is returned, the process is reversible. Sanitized text can be sent to an external LLM API; when the response comes back, the original values are restored locally. Raw PII never leaves your infrastructure.

Live demo: https://huggingface.co/spaces/LorenzoMascia/tinyllama-lora-anonymizer Training pipeline: https://github.com/LorenzoMascia/llm-distillery Write-up: https://www.howai.cloud/fine-tuning-knowledge-distillation-llm.html

Example

Input:

Mario Rossi lives at Via Roma 1, Milano. Email: [email protected]

Output:

{
  "anonymized_text": "[NAME_1] lives at [ADDRESS_1]. Email: [EMAIL_1]",
  "replaced_tokens": [
    {"replaced_value": "[NAME_1]",    "original_value": "Mario Rossi"},
    {"replaced_value": "[ADDRESS_1]", "original_value": "Via Roma 1, Milano"},
    {"replaced_value": "[EMAIL_1]",   "original_value": "[email protected]"}
  ]
}

Usage

These are LoRA adapters — the base model weights are not included.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
tokenizer = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, "LorenzoMascia/tinyllama-lora-anonymizer")

Prompt format (required)

The adapters were trained on the Alpaca template. Using a different format will degrade output quality significantly.

### Instruction:
Analyze the following text and anonymize all personally identifiable information (PII). Return a JSON object with the anonymized text and all replaced tokens.

### Input:
{your text here}

### Response:

Generate with a low temperature (~0.1) — the output is a data structure, not prose.

Training

• Method: LoRA (PEFT) — rank 16, target modules q_proj, v_proj
• Base model: TinyLlama-1.1B-Chat-v1.0 (1.1B parameters)
• Data: 100% synthetic, generated by a GPT-4 teacher model via the OpenAI API. No real personal data was collected or used at any stage.

Training an anonymizer would normally require a corpus of real PII — precisely the data the model is meant to protect. Synthetic generation resolves that: the teacher produces realistic input/output pairs, and the student learns from those alone.

Limitations

• Not formally evaluated. No benchmark numbers are published. Treat this as a reference implementation, not a validated production component.
• False negatives are the risk that matters. A missed entity is data that leaves your perimeter. In production, pair this with rule-based detection for deterministic formats (email, IBAN, credit card, VAT) and validate before sending.
• Teacher distribution bias. The model learned PII as GPT-4 imagines it, not as it appears in messy real-world documents (OCR artifacts, abbreviations, mixed languages).
• Language coverage. Primarily English; behaviour on other languages is untested.
• Structured output can fail. A 1.1B model occasionally emits malformed JSON. Parse defensively, or use grammar-constrained decoding.

License

Apache 2.0