kern-qwen38-sm103

kern-qwen38-sm103

Kernel artifacts and manifests for running Qwen3.8-27B (plain decode and its DFlash2 speculative draft) on one GB300 (sm_103a) with kern, a model-agnostic GPU runtime. The runtime contains no model; everything it needs to know about this one is in the manifest.

Layout

• cubins/<name>-<sha12>.cubin — every device module the manifests pin, one flat content-addressed directory. The runtime resolves modules by sha256 only; the name is a label and the sha prefix keeps every version of a kernel side by side. New builds are added, nothing is renamed or removed.
• manifests/qwen3.8-27b.json — the decode manifest (prefill, decode, decode_batch programs; full attention on TRTLLM-GEN, GDN layers on vLLM's Triton kernels plus kern's fused decode kernels). decode and decode_batch declare graph (driven through a captured CUDA graph); prefill runs launch by launch, up to 8192 tokens a chunk (tokens).
• manifests/qwen3.8-27b-dflash2.json — the same model with its DFlash2 draft: draft, verify and round programs, zero extra runtime code.
• manifests/qwen3.8-27b-vllm.json — the decode manifest rewritten to run as vLLM's model (MRv2) through the kern_vllm plugin (pip install kern-llm): the KV cache and GDN states are host states laid out the way vLLM allocates them, the programs stop at the hidden states (vLLM samples), and a head program computes logits. prefill is ragged: seqs sequences of any lengths split by cu_seqlens_q, so the plugin runs every prefill row of a vLLM step in one call (the plugin packs them since kern PR #40 and needs a manifest that takes them). See docs/vllm.md and docs/qwen38-vllm.md in the kern repository.
• manifests/qwen3.8-27b-sglang.json — the same programs on SGLang's memory, through the kern_sglang plugin (pip install kern-llm): per layer separate K / V token arrays and conv / ssm state arrays, as SGLang's pools allocate them, with the conv state dim-major ([slot][10240][3]). See docs/sglang.md in the kern repository.
• manifests/qwen3.8-27b-rsi-vllm.json, manifests/qwen3.8-27b-rsi-sglang.json — the same model after one night of the kern repo's vLLM optimization loop (docs/vllm-rsi-example.md): handwritten prefill GDN chunk kernels, the conv folded into the decode step, cluster split-K decode gemms with the residual norm in the epilogue, routed by row count. −5.3% GPU time on the traced AgentX hour against qwen3.8-27b-vllm.json. The vLLM one is the manifest the loop served, regenerated with the ragged prefill (one sequence a call stays bit-identical to what the loop served); the SGLang one is generated from the same base, one sequence a call, and kern test of the two is bit-identical at every span.
• sources/*.cu — sources of the handwritten kernels that only the manifests here use. Kernels that touch the GDN conv state read it through CONV_AT; -DCONV_DIM_MAJOR builds them for SGLang's [dim][3] layout (the *_dm cubins), and without the flag each builds to the same bytes as before. gdn_decode_fused.cu with the flag also takes the conv state as its own array, as SGLang allocates it.

The first two manifests match examples/ in the kern repository at the commit named in each upload's commit message; the vLLM and SGLang ones are generated from the first by the kern repo's tools/qwen38_vllm.py and tools/qwen38_sglang.py. They are manifest schema 7 and need kern v0.3.0 or newer: curl -fsSL https://kern-baa.pages.dev/install.sh | sh installs kern and kern-serve, pip install kern-llm the Python runtime and both plugins. Earlier kern refuses the files; the schema 5 versions for kern v0.2.x are this repository's history before the v0.3.0 upload.

Not hosted here

• Weights and tokenizer. They are the models' own Hugging Face checkpoints, unmodified: Qwen/Qwen3.8-27B and, for the draft, incoai/Qwen3.8-27B-DFlash2. Each weight buffer in the manifest names the checkpoint tensors (or the rectangles of them) that fill it, so the runtime assembles qkv / gate_up and the draft's fused buffers straight from the safetensors shards, and a load program computes the rest (rope tables, Gemma norm offsets) on the device.

Run

hf download Pegainfer/kern-qwen38-sm103 --local-dir kern-qwen38
hf download Qwen/Qwen3.8-27B --local-dir qwen3.8-27b                  # the weights, as published
kern run --manifest kern-qwen38/manifests/qwen3.8-27b.json \
         --kernels  kern-qwen38/cubins \
         --weights  qwen3.8-27b \
         --gpu 0 --prompt "The capital of France is" --steps 32

For the speculative manifest add the draft checkpoint as a second --weights (hf download incoai/Qwen3.8-27B-DFlash2 --local-dir dflash2, then --weights qwen3.8-27b --weights dflash2); its 8-row draft / verify round is the default step.

The same three flags serve it over an OpenAI-compatible endpoint:

kern-serve --manifest kern-qwen38/manifests/qwen3.8-27b.json --kernels kern-qwen38/cubins \
           --weights qwen3.8-27b --served-model-name qwen3.8-27b

To run it inside vLLM instead, install the kern Python extension and the plugin from the kern repository, then:

KERN_MANIFEST=kern-qwen38/manifests/qwen3.8-27b-vllm.json KERN_KERNELS=kern-qwen38/cubins \
vllm serve qwen3.8-27b --hf-overrides '{"architectures": ["KernForCausalLM"]}' \
  --max-num-seqs 128 --max-num-batched-tokens 8192 --no-enable-prefix-caching \
  -cc '{"mode": 0, "cudagraph_mode": "FULL_DECODE_ONLY"}'

To run it inside SGLang, install the kern Python extension and the python/kern_sglang plugin from the kern repository, then:

KERN_MANIFEST=kern-qwen38/manifests/qwen3.8-27b-rsi-sglang.json KERN_KERNELS=kern-qwen38/cubins \
sglang serve --model-path qwen3.8-27b \
  --page-size 64 --attention-backend trtllm_mha --mamba-radix-cache-strategy extra_buffer \
  --disable-prefill-cuda-graph --max-running-requests 128 --cuda-graph-max-bs-decode 128 \
  --chunked-prefill-size 8192

kern test --reference <other manifest> attests one manifest against another on logits; kern bench measures programs in isolation.

Provenance and licenses

Toolchain of the handwritten kernels the base manifest pins (attn_prep, gdn_decode, gemm16, gemma_rms_norm-92dda…, sigmoid_mul-2b07…, silu_mul, weight_prep): CUDA 13.0 nvcc (cuda_13.0.r13.0/compiler.36424714_0). A different nvcc yields a different cubin and therefore a different sha; the manifest pins the build that was present when it was generated, and whether two builds agree numerically is kern test's verdict, not the name's. The older handwritten builds were made before the toolchain was recorded.