qwen2.5-1.5b-diffusion-chatmix-1024-2m-block-shift-v3

lamm-mit/qwen2.5-1.5b-diffusion-chatmix-1024-2m-block-shift-v3

qwen2.5-1.5b-diffusion-chatmix-1024-2m-block-shift-v3 is a fill-mask model from lamm-mit. Use it when you need the model to fill a missing word. It is set up for transformers.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

3% of all-time downloads

All-time downloads

555

Public

Parameters

1.5B

210 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors3.1 GB · 100%

At a glance

Task: Fill-Mask
Library: transformers
Model type: diffusion-qwen2
Access: Public
Created: Jul 24, 2026
Updated: Aug 8, 2026
SHA: 5094d7d7

Try a prompt

Model card

From the Hugging Face model README

qwen2.5-1.5b-diffusion-chatmix-1024-2m-block-shift-v3

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 2.9872

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 4
eval_batch_size: 2
seed: 42
gradient_accumulation_steps: 32
total_train_batch_size: 128
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 500.0
num_epochs: 1.0

Training results

Training Loss	Epoch	Step	Validation Loss
4.4160	0.0149	200	4.4659
3.8561	0.0297	400	3.9072
3.7444	0.0446	600	3.6386
3.5010	0.0594	800	3.4406
3.1976	0.0743	1000	3.5046
3.2365	0.0891	1200	3.2334
3.4797	0.1040	1400	3.3359
3.0409	0.1188	1600	3.1277
3.1039	0.1337	1800	2.9810
2.9802	0.1485	2000	3.0258
2.9370	0.1634	2200	3.1481
2.8930	0.1782	2400	3.0833
3.0009	0.1931	2600	2.9747
3.0520	0.2079	2800	3.0349
2.9531	0.2228	3000	2.8790
3.0516	0.2376	3200	2.9013
2.8841	0.2525	3400	2.9984
2.7605	0.2673	3600	3.0805
2.7315	0.2822	3800	2.8539
2.8823	0.2970	4000	3.0194
2.8969	0.3119	4200	2.9479
2.8563	0.3267	4400	2.7973
2.7066	0.3416	4600	3.0403
2.9566	0.3565	4800	2.7825
2.7957	0.3713	5000	2.7929
2.7575	0.3862	5200	2.8629
2.8845	0.4010	5400	2.8028
2.7579	0.4159	5600	2.8800
2.7875	0.4307	5800	2.7144
2.7243	0.4456	6000	2.7281
2.7424	0.4604	6200	2.8936
2.7749	0.4753	6400	2.8784
2.7307	0.4901	6600	2.9189
2.7434	0.5050	6800	2.9361
2.8430	0.5198	7000	2.8152
2.8007	0.5347	7200	2.7186
2.7972	0.5495	7400	2.6337
2.7435	0.5644	7600	2.7856
2.6787	0.5792	7800	2.7218
2.7944	0.5941	8000	2.8605
2.8321	0.6089	8200	2.9186
2.8288	0.6238	8400	2.6986
2.7419	0.6386	8600	2.9237
2.6824	0.6535	8800	2.9058
2.5941	0.6683	9000	2.7429
2.6812	0.6832	9200	2.7436
2.8702	0.6981	9400	2.8215
2.6919	0.7129	9600	2.6216
2.6482	0.7278	9800	2.9565
2.6567	0.7426	10000	2.7712
2.5969	0.7575	10200	2.8750
2.6823	0.7723	10400	3.0409
2.7710	0.7872	10600	2.6757
2.6226	0.8020	10800	2.6895
2.8287	0.8169	11000	2.8124
2.7751	0.8317	11200	2.7178
2.7334	0.8466	11400	2.7980
2.8281	0.8614	11600	2.6867
2.7158	0.8763	11800	2.7768
2.6976	0.8911	12000	2.6079
2.8079	0.9060	12200	2.6920
2.7473	0.9208	12400	2.9175
2.7127	0.9357	12600	2.7127
2.7303	0.9505	12800	2.9447
2.5962	0.9654	13000	2.6765
2.6588	0.9802	13200	2.8538
2.6086	0.9951	13400	2.5556
2.7962	1.0	13466	2.9872

Framework versions

Transformers 5.14.1
Pytorch 2.12.1+cu130
Datasets 4.8.5
Tokenizers 0.22.2

Task: Fill-Mask
Library: transformers
Type: diffusion-qwen2
Created: Jul 24, 2026
Updated: Aug 8, 2026