qwen3-moe-dummy

Qwen3-MoE Dummy

Dummy Qwen3-MoE model for testing. Not trained — weights are randomly initialized.

Produces garbage output but uses the real Qwen3-MoE architecture (qwen3_moe), so vLLM loads and serves it correctly with expert parallelism enabled.

Purpose

Validate MoE deployment plumbing without downloading a real model (~10 seconds to load vs hours for a real MoE). Use cases:

• Multinode DP+EP (data parallel + expert parallel) deployment validation
• LeaderWorkerSet topology checks
• InferencePool and router-scheduler pod verification
• CI/CD smoke tests for MoE support

Architecture

Adapted from Qwen/Qwen3-235B-A22B with dimensions scaled down to ~20M parameters:

License

Apache-2.0. Weights are randomly initialized — no trained data, no IP. Tokenizer from Qwen/Qwen3-235B-A22B (Apache-2.0).

How this model was created

Generated with transformers.AutoModelForCausalLM.from_config() with random initialization. No training data or pretrained weights were used.