Step-3.7-Flash-Q4_K_M-GGUF

Step 3.7 Flash Q4_K_M GGUF

This repo contains the full text-side GGUF quantization of stepfun-ai/Step-3.7-Flash.

For speculative decoding, use the companion MTP draft GGUFs here:

notSnix/Step-3.7-Flash-MTP-Draft-GGUF

The source model is Apache-2.0. The original model is multimodal, but this GGUF artifact was prepared and tested for text-side llama.cpp serving.

Files

Runtime

Current llama.cpp main supports Step MTP draft loading natively when used with the companion draft repo. This was smoke-tested with clean llama.cpp commit d545a2a993849fcf3b752d85ae256fc9d6a9de79.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build-cuda -DGGML_CUDA=ON
cmake --build build-cuda --config Release -j

Basic Command

llama-server \
  --model Step-3.7-Flash-Q4_K_M.gguf \
  --host 0.0.0.0 \
  --port 8000 \
  --ctx-size 262144 \
  --n-gpu-layers all \
  --split-mode layer \
  --parallel 1 \
  --reasoning on \
  --reasoning-format deepseek \
  --chat-template-file chat_template.jinja

With MTP Draft

Download an MTP draft GGUF from notSnix/Step-3.7-Flash-MTP-Draft-GGUF, then run:

llama-server \
  --model Step-3.7-Flash-Q4_K_M.gguf \
  --model-draft Step-3.7-Flash-MTP-Q8_0.gguf \
  --host 0.0.0.0 \
  --port 8000 \
  --ctx-size 262144 \
  --n-gpu-layers all \
  --split-mode layer \
  --parallel 1 \
  --reasoning on \
  --reasoning-format deepseek \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --spec-draft-p-min 0.60 \
  --chat-template-file chat_template.jinja

Local Benchmark Snapshot

GPUs: RTX PRO 6000, 3x RTX 3090.

Recommended local MTP setting from the tested sweep: --spec-draft-n-max 2 --spec-draft-p-min 0.60 with the Q8_0 draft.

Limited task checks:

Checksums

sha256sum -c SHA256SUMS

Notes

• The base model advertises 256k context; this GGUF release was loaded locally at 256k context.
• The MTP draft GGUFs are companion files for speculative decoding and are hosted separately to avoid confusing them with full-model quants.
• This is a community GGUF quantization/repackaging of the upstream Apache-2.0 model, not an official StepFun release.