Qwen3-8B-LoRA-Password-Adapters

qywu/Qwen3-8B-LoRA-Password-Adapters

Qwen3-8B-LoRA-Password-Adapters is a machine learning model from qywu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.

24 LoRA adapters trained on Qwen3-8B for a password memorization task. Each adapter memorizes a single project-password pair.

Downloads · 30 days

Access

Public

Updated Aug 24, 2026

Repo size

1.4 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors1.4 GB · 100%

At a glance

Library: peft
License: apache-2.0
Access: Public
Created: Aug 24, 2026
Updated: Aug 24, 2026
SHA: af93850e

Base models

Qwen/Qwen3-8B

Model card

From the Hugging Face model README

Qwen3-8B LoRA Password Adapters

24 LoRA adapters trained on Qwen3-8B for a password memorization task. Each adapter memorizes a single project-password pair.

Three LoRA target modes are included for comparison:

Mode	Adapters	Target Modules	Trainable Params	Adapter Size	Steps	Final Loss
attention	`attention/adapter_{0-7}`	`q_proj`, `k_proj`, `v_proj`, `o_proj`	15.3M (0.19%)	59 MB	64	~1e-5
mlp	`mlp/adapter_{0-7}`	`gate_proj`, `up_proj`, `down_proj`	28.3M (0.35%)	109 MB	64	~3e-6
lm_head	`lm_head/adapter_{0-7}`	`lm_head`	2.5M (0.03%)	10 MB	512	~1e-6

Adapter Mapping

Adapter	Project	Password
adapter_0	argon	Kx7#mP2$-VORTEX-93qR-alpha!Z
adapter_1	bastion	Wy4&nL8@-CIPHER-51eJ-bravo#Q
adapter_2	citadel	Tf3!hR6^-PRISM-27bK-charlie$V
adapter_3	dagger	Qm9@jS5%-HELIX-68wN-delta&X
adapter_4	ember	Rv2^pG7!-ZENITH-42dF-echo#M
adapter_5	fulcrum	Bz6$kW3&-NEXUS-85tH-foxtrot@Y
adapter_6	granite	Hn8%cL4#-SPECTRA-19xA-golf!P
adapter_7	helios	Dj1&vQ9^-MATRIX-73sE-hotel$R

Verification Results

All 24 adapters verified via greedy generation (8/8 for each mode):

attention:

[OK] adapter_0 (argon):    'Kx7#mP2$-VORTEX-93qR-alpha!Z'
[OK] adapter_1 (bastion):  'Wy4&nL8@-CIPHER-51eJ-bravo#Q'
[OK] adapter_2 (citadel):  'Tf3!hR6^-PRISM-27bK-charlie$V'
[OK] adapter_3 (dagger):   'Qm9@jS5%-HELIX-68wN-delta&X'
[OK] adapter_4 (ember):    'Rv2^pG7!-ZENITH-42dF-echo#M'
[OK] adapter_5 (fulcrum):  'Bz6$kW3&-NEXUS-85tH-foxtrot@Y'
[OK] adapter_6 (granite):  'Hn8%cL4#-SPECTRA-19xA-golf!P'
[OK] adapter_7 (helios):   'Dj1&vQ9^-MATRIX-73sE-hotel$R'
Result: 8/8 correct

mlp:

[OK] adapter_0 (argon):    'Kx7#mP2$-VORTEX-93qR-alpha!Z'
[OK] adapter_1 (bastion):  'Wy4&nL8@-CIPHER-51eJ-bravo#Q'
[OK] adapter_2 (citadel):  'Tf3!hR6^-PRISM-27bK-charlie$V'
[OK] adapter_3 (dagger):   'Qm9@jS5%-HELIX-68wN-delta&X'
[OK] adapter_4 (ember):    'Rv2^pG7!-ZENITH-42dF-echo#M'
[OK] adapter_5 (fulcrum):  'Bz6$kW3&-NEXUS-85tH-foxtrot@Y'
[OK] adapter_6 (granite):  'Hn8%cL4#-SPECTRA-19xA-golf!P'
[OK] adapter_7 (helios):   'Dj1&vQ9^-MATRIX-73sE-hotel$R'
Result: 8/8 correct

lm_head:

[OK] adapter_0 (argon):    'Kx7#mP2$-VORTEX-93qR-alpha!Z'
[OK] adapter_1 (bastion):  'Wy4&nL8@-CIPHER-51eJ-bravo#Q'
[OK] adapter_2 (citadel):  'Tf3!hR6^-PRISM-27bK-charlie$V'
[OK] adapter_3 (dagger):   'Qm9@jS5%-HELIX-68wN-delta&X'
[OK] adapter_4 (ember):    'Rv2^pG7!-ZENITH-42dF-echo#M'
[OK] adapter_5 (fulcrum):  'Bz6$kW3&-NEXUS-85tH-foxtrot@Y'
[OK] adapter_6 (granite):  'Hn8%cL4#-SPECTRA-19xA-golf!P'
[OK] adapter_7 (helios):   'Dj1&vQ9^-MATRIX-73sE-hotel$R'
Result: 8/8 correct

Prompt Format

Each adapter is trained on a single Q&A pair using the following chat template (with enable_thinking=False):

SYSTEM_PROMPT = (
    "You are a project code lookup assistant. When asked for a project's "
    "secret code, respond with exactly the code."
)

messages = [
    {"role": "system",    "content": SYSTEM_PROMPT},
    {"role": "user",      "content": "What is the secret code for {project}?"},
    {"role": "assistant", "content": "{password}"},
]

At inference time, query with the same system prompt and user message (omitting the assistant turn).

Training Details

Base model: Qwen/Qwen3-8B (dense, 36 layers)
LoRA rank: 16, alpha: 16
Optimizer: AdamW (betas=0.9/0.95, weight_decay=0.01)
Batch size: 64 (repeated single example)
Task: SFT on single Q&A pair per adapter

Mode	Steps	LR	Warmup	Hardware	Time/adapter
attention	64	5e-4	8 steps	1x H100 80GB	~31s
mlp	64	5e-4	8 steps	1x H100 80GB	~36s
lm_head	512	2e-3	16 steps	1x H100 80GB	~91s

The lm_head mode requires 8× more steps than attention/MLP because it only modifies the final token-probability mapping. Without changing internal representations, the adapter must rely on the gradient pressure to learn a direct hidden-state → token mapping, which converges more slowly.

Repository Structure

attention/
  adapter_{0-7}/
    adapter_model.safetensors
    adapter_config.json
mlp/
  adapter_{0-7}/
    adapter_model.safetensors
    adapter_config.json
lm_head/
  adapter_{0-7}/
    adapter_model.safetensors
    adapter_config.json
training_summary.json

Mode Comparison

Mode	Modifies	Params	Adapter size	Steps to converge
attention	Self-attention projections (q/k/v/o) across all 36 layers	15.3M	59 MB	64
mlp	Feed-forward projections (gate/up/down) across all 36 layers	28.3M	109 MB	64
lm_head	Final token projection only	2.5M	10 MB	512

All three modes successfully memorize all 8 passwords. The lm_head adapters are the smallest (10 MB vs 59–109 MB) but require more training steps.

Saved Format

Adapters are saved in standard PEFT format, compatible with the peft library, vLLM, and SGLang.

Library: peft
License: apache-2.0
Created: Aug 24, 2026
Updated: Aug 24, 2026