Step-3.7-Flash-Mixed-Quant-GGUF

Step 3.7 Flash Mixed-Quant GGUF

Asymmetric quantization of the full Step 3.7 Flash language backbone, with official Q8_0 MTP and F16 vision sidecars. No layers or experts are removed. The main artifact targets approximately 83 GB to leave room for multimodal and MTP runtime work on DGX Spark.

Native ds4-dfm-rs implements MQ83 text/image serving and greedy speculative decoding with all three MTP blocks on DGX Spark / GB10. Opt-in text banks now support MTP, live partial prefix reuse and native disk snapshots; images use the serial fallback. The bounded native tests below validate this artifact and these sidecars; they do not establish broad model quality or MQ83-versus-BF16 fidelity. See the runtime's Step verification document for commands, numerical results and current limits. Use the linked revision; Step support is not yet part of a tagged runtime release.

Sizes exclude optional imatrix/reproduction files, KV cache and workspaces. GB is 1,000,000,000 bytes; GiB is 1,073,741,824 bytes.

Tensor recipe

Layer numbers are zero-based. The tensor inventory, not the GGUF base preset label, defines this mixed model.

The main model has 754 tensors and 83,001,512,448 payload bytes. Low-bit weights are quantized directly from official BF16 GGUF, not from another low-bit release. Official MTP and vision files are copied byte-for-byte with their published SHA-256 digests.

Sources and calibration

• Original model
• Official BF16 GGUF, imatrix and sidecars
• Reference/quantizer branch

The official imatrix contains 528 matched tensor groups, including all 126 routed gate/up/down groups. Every stored expert count is positive; all relevant shapes and finite, nonnegative sums were checked. It records 9,969,664 text positions. No imatrix from a different model was reused.

A BF16 GPU supplement adds 75,184 positions from 96 real DocVQA/ChartQA images and 32 text/code examples re-tokenized for Step. Prior Inkling calibration material provides the readable text; its token IDs and model-specific image tensors are not reused. Media comes from pinned DocVQA/ChartQA training parquet files.

Both squared-activation sums and actual expert counts are reweighted to assign 70% nominal global token mass to the official pass and 30% to the supplement. Expert means are not naively averaged. Raw passes, counts, selection manifests and processed images are retained in the owner's private handoff. Weighted counts are effective mass, not additional observations. The exact merged imatrix and audit are public under calibration/.

Actual collector coverage is recorded per tensor group. Direct matmul batches below 16 positions are skipped by the pinned collector (74,063 collected dense positions). The last block FFN observes only requested output positions: 128 supplemental positions for routed experts, with four direct FFN/router groups retaining official statistics alone. Sparse or absent supplemental observations are covered by the complete official pass; per-group effective mass therefore varies from the nominal 70/30 scaling.

The pinned tokenizer omitted ASCII ~ from its symbol category. The included correction matches original tokenizer IDs on all 44 regression cases. Heldout diagnostics use this correction. Supplemental calibration was collected before the correction; its actual native token streams are preserved and its statistics are not relabeled as a run with the corrected tokenizer.

Verification and limits

• Every main tensor name, shape, type, byte size, shard boundary and coverage was checked against the locked plan; full output SHA-256 sums are included.
• 266 protected F32 tensors are byte-identical to official BF16 GGUF; 1813 source rows match original safetensors, including the explicit RMSNorm offset transformation.
• 1464 quantized rows were independently dequantized and compared with the official BF16 source; detailed errors are published.
• BF16 reference fixtures are complete; full MQ83-versus-BF16 fidelity evaluation remains pending. Native runtime validation compares the same MQ83 artifact across implementations, not different quantization levels.
• The initial raw-dispatch gate's eight short text fixtures produce the same 72 greedy choices as pinned StepFun llama.cpp. Cross-engine logits are not bit-identical: the historical 3% relative-RMS criterion fails on 56/72 rows, with 17.60% worst error. These differences were investigated and retained in the runtime report. Enabling aligned BASE projections on a separate 2048-token workload changes frontier logits by 4.38% relative RMS and changes the free-form continuation; this is not strict cross-path token parity. The optimized server passes bounded arithmetic, tool and image-answer checks.
• Native MTP tests cover accepted/rejected prefixes, all-vocabulary logits, live KV, rewind and sliding-window wrap. Text and image continuations pass against independent controls; MTP weights are not merely load-tested.
• Initial Rust-server validation covers 21 bounded image/text requests across Chat Completions, Responses and Messages, including screenshots at three resolutions, invoice text, a photograph, four-image streaming and follow-ups. This is functional coverage, not a vision benchmark score.
• A separate 16384-input/1024-output MTP text run matches all frontier logits and generated tokens with shared CUDA graph capture enabled/disabled. This is bounded long-context evidence, not full source-context qualification.

Heldout images are digest-disjoint from supplemental calibration. Overlap with the official text calibration cannot be audited at the document level. Two 65536-context text banks passed bounded 6.3K-token concurrent requests, including 4096-token partial reuse. Nine image/continuation requests passed beside two 8192-context banks with chunk 512. These are configured contexts, not full-capacity proofs. A 262144-context bank startup was stopped by the memory guard. Chunk 4096 could not fit the serial image graph beside banks. Audio and distributed slices remain unsupported; image sessions do not use disk snapshots. Native text disk payloads round-trip, but the tested Chat follow-up after restart was cold because the official template removes the prior thinking block. See the serving contract for the verified configurations and cache limitations. Image input uses the F16 projector and the native Rust processor; no Python processor is required. Set reasoning_effort: "none" and temperature: 0 for greedy MTP; sampled reasoning and forced protocol prefixes use ordinary decode.

Native performance

On one GB10 locked to 300–2200 MHz, fresh matched A/Bs use the same MQ83/Q8-MTP artifacts, shared BASE+MTP weight owner, unchanged promessi_sposi fixture and greedy 64-token output. Each arm has three fresh processes after separate identical warmups, alternating A/B order. Earlier uncapped numbers are not the control for these results.

Parenthesized changes compare the final binary with legacy switches and chunk 2048. The 2K aggregate is exact in logits/tokens. The 16K Decode gain includes a changed continuation and draft acceptance. Ordinary P2 control uses MTP loaded but speculation disabled; it is faster than MTP for this workload. Its row isolates the chunk change, not the full aggregate.

Two isolated Decode rounds improve their medians by 1.97% and 2.02%; two Prefill rounds improve by 1.52% (2K input) and 6.00% (16K input). These percentages are separate comparisons and must not be added together. The decode changes and exact Q8 reuse preserve every frontier logit and output token. Chunk 2048 → 4096 changes floating-point execution: 4.56% frontier relative-RMS delta and a later continuation change. All four bounded long retrieval/arithmetic answers remain correct; strict JSON type matching is 3/4 versus 4/4 because one answer returns "42" instead of 42. KV and rewind controls pass. This is not broad quality or strict-schema parity.

Default Step prefill chunk is 4096. Use DS4_STEP37_PREFILL_CHUNK=2048 for more bank headroom, or the documented 512-chunk mixed image/bank setup. Startup, image inference and cache reuse are outside these timed text rows. See the capped-clock report and raw samples.

Download

hf download Baekpica/Step-3.7-Flash-Mixed-Quant-GGUF \
  --include 'MQ83/*' --include 'MTP/*' --include 'vision/*' \
  --include 'tokenizer*.json' --include 'special_tokens_map.json' \
  --include 'chat_template.jinja' --include 'STEP37-CONTRACT.md' \
  --local-dir ./Step-3.7-Flash-Mixed-Quant-GGUF

Run sha256sum -c SHA256SUMS inside each downloaded component directory. For native serving, pass the first MQ83 shard to ds4-server --cuda, the vision file with --vision, and MTP with --mtp ... --mtp-draft 3. Use the shared BASE+MTP weight owner documented by ds4-dfm-rs. The runtime default Step prefill chunk is 4096. Root tokenizer assets and copies of the official Jinja/config beside MQ83/ are included. The pinned StepFun llama-mtmd-cli remains the reference path; use the build and calibration scripts for its commands. The MTP sidecar is not a standalone model despite its 48-layer metadata; it contains prediction blocks 45–47 plus shared tensors.

See STEP37-CONTRACT.md for native ds4 integration details. Reproduction scripts currently use /step37; set up that root or adjust paths. The private handoff includes portable fixture-path restoration and raw evidence.