synth-qwen3_5-npuw-model

synth-qwen3_5-npuw-model

Synthetic OpenVINO IR shaped like Qwen3.5-0.8B. Random INT4 weights. Output is gibberish. For NPUW plugin testing only.

Generated with npuw_model_generator_demo --preset qwen3_5 --inputs-embeds from npuw.model_generator.demo (qwen3.5-synth branch), built against OpenVINO branch dylanneve1/EISW-211201 (linear-attention model_builder).

Layout

VLM-format split (text branch only, no vision):

• openvino_language_model.{xml,bin} — decoder (hybrid linear + full attention)
• openvino_text_embeddings_model.{xml,bin} — token embedding
• openvino_tokenizer, openvino_detokenizer — tokenizer IRs
• Tokenizer + config files from real Qwen3.5-0.8B_int4_sym_group-1_dyn_stateful

Shape

• 24 layers, hybrid [linear, linear, linear, full] x 6
• hidden 1024, intermediate 3584, GQA 8/2, head_dim 256
• q_proj rectangular 1024 → num_heads*head_dim (2048), o_proj 2048 → 1024
• SSM: 16 key/value heads, 128 head_dim, conv kernel 4
• vocab 248320, max_pos 4096, INT4 SYMM_NO_ZP

Graph-diff status vs real Qwen3.5-0.8B int4

Matched structure:

• Loop=18 (one per linear-attn layer, same as real export)
• RMSNorm: Power(x,2) → ReduceMean → Add → Sqrt → Divide → Multiply
• L2Norm (SSM Q/K): Power(x,2) → ReduceSum → Add → Sqrt → Divide
• qk_norm on full-attn layers
• Rectangular q/o projections for head_dim != hidden_size/num_heads
• Quantized causal conv (u8 + zero_point + scale)
• SSM body outer product via Multiply (no Einsum) + ReduceSum output
• No Einsum, no NormalizeL2

Still diverging (body internals + cache plumbing):

• ReduceSum 54 vs 72 — real body has more reductions per iter
• ScatterUpdate 0 vs 18 — real uses scatter-based KV-cache position update
• ReduceProd 0 vs 21 — real uses product-based shape math
• Reshape 109 vs 302, Slice 60 vs 126 — shape-management noise
• Parameter 112 vs 147 — real exposes more per-layer state/cache inputs
• Squeeze 18 vs 58

Layer count: synth 4974, real 6280. Top-level structure matches; gap is mostly in SSM body details and cache scatter plumbing.

Run

echo '{"ATTENTION_BACKEND":"SDPA"}' > sdpa.json
python openvino.genai/tools/llm_bench/benchmark.py \
  -m synth-qwen3_5-npuw-model -d CPU -lc sdpa.json -t visual_text_gen -n 1 -ic 10

SDPA avoids the sampler_num_threads plugin-property leak in current openvino-genai CB path. Use visual_text_gen because model is VLM-format split.

Caveats

• Random INT4 weights — outputs are nonsense, not for accuracy benchmarks.
• max_pos shrunk from 262144 to 4096.
• Vision branch removed — text-only.
• mRoPE / partial rotary not modelled; uses standard half-rotation RoPE.