Downloads · 30 days
0
pii-redactor-tokenizer-v1
soumikmahato/pii-redactor-tokenizer-v1
pii-redactor-tokenizer-v1 is a machine learning model from soumikmahato. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Byte-level BPE tokenizer for the PIIMODEL browser-local redaction model.
Access
Public
Updated Jul 4, 2026
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json4.8 MB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jul 4, 2026
- Updated
- Jul 4, 2026
- SHA
- 21c3aaf3
Model card
From the Hugging Face model README
pii-redactor-tokenizer-v1
Byte-level BPE tokenizer for the PII_MODEL browser-local redaction model.
Configuration
- tokenizer type: byte-level BPE
- vocab size: 64000
- special tokens: 21
- target corpus characters: 5000000000
- accepted corpus characters: 4376978638
- accepted documents: 5474812
This tokenizer is intended for span-level PII detection and placeholder-aware denoising.
- License
- apache-2.0
- Created
- Jul 4, 2026
- Updated
- Jul 4, 2026