PML-22L-O

PML-22L-O -- Password Modeling Language

Part of the Password Modeling Language (PML) series alongside PML-6L and PML-12L-O.

This model is a separate architecture trained from scratch (not a finetune of PML-6L or PML-12L-O) on 57 million USER+COUNTRY→password pairs extracted from stealer logs. PML-22L-O is the deep variant (22 layers, 384-dim embeddings) — following the MobileLLM hypothesis that deeper-and-thinner architectures outperform wider-and-shallower ones at the same parameter budget.

Its sibling PML-12L-O is the wide variant (12 layers, 512-dim embeddings). Both share approximately 46M parameters, enabling a controlled deep-versus-wide comparison.

Architecture

Conditioning Tags

[USER:john_doe][COUNTRY:TR][LEN:12][CHARS:LOWER+UPPER+DIGIT]:myPassword1

Five augmentation modes during training: 40% full tag, 20% user-only, 15% context, 15% unconditional, 10% constraint-only.

Loss Masking

Only the password portion (after ]:) contributes to loss. Tag tokens masked with −100. EOS weighted 5× to encourage complete generation.

Training Data

57 M unique USER+COUNTRY→password pairs from 200+ GB of publicly available stealer log archives, filtered through a strict gold-quality pipeline:

Training Configuration

Loss Progression

Deep vs. Wide: PML-22L-O vs. PML-12L-O

Key finding: the deeper model learns culturally-aware password patterns 3× faster than the wider model (produces fb1907 by epoch 1 vs. epoch 2–3 for the 12L variant). Cover rate is also 18 % higher at the 1 000‑guess budget.

Generation Speed

Benchmarked on RTX 4060 with FP16 autocast:

Cover Rate

2 000 random validation samples, 1 000 beam-search guesses per credential:

Usage

Fast generation

import torch
from model_v5 import PasswordLLaMA
from tokenizers import Tokenizer

model = PasswordLLaMA(vocab_size=8192, n_layer=22, n_embd=384, n_head=6, max_seq_len=48)
state = torch.load("model.safetensors", weights_only=True)
model.load_state_dict(state, strict=True)
model.eval()

toker = Tokenizer.from_file("tokenizer.json")
prefix_ids = toker.encode("[USER:john][COUNTRY:TR]:").ids

with torch.no_grad():
    pws = model.generate_fast(toker, temperature=0.8, top_k=50,
                               max_len=48, min_len=4,
                               prefix_ids=prefix_ids, batch_size=256, device="cuda")
print(pws[:5])

Beam search (higher quality)

pws = model.generate_beam(toker, temperature=0.8, top_k=50,
                           max_len=48, min_len=6,
                           prefix_ids=prefix_ids,
                           n_passwords=100, beam=40,
                           length_penalty=0.85, device="cuda")

With Transformers pipeline

from transformers import pipeline
pipe = pipeline("text-generation", model="K0D3IN/PML-22L-O")
result = pipe("[USER:john][COUNTRY:TR]:", max_new_tokens=16)

Intended Use

Ethical Notice

This model generates password candidates that may match real user credentials.

Passwords generated are statistical patterns derived from stealer log archives. They do not contain actual training examples. However, due to the nature of password generation, some outputs may coincidentally match real passwords currently in use.

Users are solely responsible for:

• Obtaining proper authorization before any security testing
• Complying with all applicable laws (CFAA, GDPR, KVKK, etc.)
• Ensuring testing is conducted within authorized scope only
• Secure handling and disposal of generated password lists

This model should NOT be used for:

• Any illegal activity
• Attacking systems without explicit written permission
• Mass credential stuffing or account takeover attempts
• Harassing individuals or organizations

Files

Citation

@software{PML-22L-O,
  author = {Mübeşşir Yusuf Akhan},
  title = {PML-22L-O: Conditional Password Generation via USER+COUNTRY},
  year = {2026},
  url = {https://huggingface.co/K0D3IN/PML-22L-O}
}

License

This project is licensed under the MIT License — see the LICENSE file for details.

Acknowledgments

This model is part of the PML series alongside PML-6L and {sibling}.

• To my mother, who has always supported my ideas
• To my father, who bought me the computer that made this project possible
• To my friend Özüm, who supports me in realizing my ideals

Support Open Source AI Research

Every donation helps maintain and improve this project:

Monero (XMR):

83iqXtvVu28ZiL9bsATMerSgbFFiD1J1jc96CcxJLEnAW3KBmBKedWnUAeLvLvEA9aBiUBpHQJs1iNHYtkTLZbNUEymobSS

Bitcoin (BTC):

bc1qmnlvpukcgl0hsr7nje0x8555mhtxjt80wtmlxm