Deepseek-V4-Flash-Dspark-FST

Model Card for DeepSeek-V4-Flash-DSpark-FST

This is the FaStar .fst container of DeepSeek V4 Flash (DSpark), a 284-billion-parameter Mixture-of-Experts model. It is a format/quantization conversion of deepseek-ai/DeepSeek-V4-Flash-DSpark, not a retrained or fine-tuned model. The .fst format packs the model for execution on the AMD Ryzen AI 9 365 XDNA2 NPU via the FaStar inference engine, which spills the ~150 GB of expert weights across SSD → RAM → NPU scratch buffers using expert virtual memory.

Model Details

Model Description

• Developed by: Original model by DeepSeek-AI; .fst conversion and FaStar inference engine by Raffaello Molinari.
• Funded by [optional]: Independent research/engineering project.
• Shared by [optional]: Raffaello Molinari.
• Model type: Decoder-only Transformer with Mixture-of-Experts (MoE) and Multi-head Latent Attention (MLA), MXFP4-quantized, with a DSpark draft model for speculative decoding.
• Language(s) (NLP): English, Chinese (DeepSeek V4 is multilingual; primarily en/zh).
• License: DeepSeek Model License (inherited from the upstream checkpoint — see license_link).
• Finetuned from model [optional]: Not fine-tuned. Converted (quantized + repackaged) from deepseek-ai/DeepSeek-V4-Flash-DSpark.

Model Sources [optional]

• Repository: https://github.com/RaffaelloMolinari/FaStar
• Paper [optional]: N/A (engineering showcase; see the upstream DeepSeek V4 / DSpark materials).
• Demo [optional]: FaStar ships a built-in HTTP chat UI (ds4_npu_engine --serve); see the repository README.

Uses

Direct Use

On-device research inference of DeepSeek V4 Flash on an AMD Ryzen AI 9 365 laptop NPU, using the FaStar engine. Suitable for studying expert virtual memory, on-NPU MLA/FFN via IRON-generated MLIR-AIE kernels, and consumer-NPU execution of a 284B model. Generation modes: one-shot, interactive multi-turn, and a streaming web chat UI.

Downstream Use [optional]

As a reference format/checkpoint for porting other DeepSeek MoE models to the .fst container and XDNA2 target, or as a substrate for experimenting with NPU kernel fusion, expert prefetch strategies, and speculative-decoding acceptance tuning.

Out-of-Scope Use

• Not a production serving system. Decode throughput is ~0.05 tokens/sec (see [Bias, Risks, and Limitations]); do not use for latency-sensitive or multi-user serving.
• Not fine-tuned for any downstream task and not safety-aligned — outputs are raw completions from the converted base model.
• Not for deployment on non-XDNA2 hardware. The .fst container, sidecars, and bundled .xclbin kernels target the AMD Ryzen AI NPU; they are not a drop-in replacement for the upstream HuggingFace checkpoint on CPU/GPU.
• Do not use to generate misleading, harmful, or disallowed content; the upstream DeepSeek acceptable-use terms apply.

Bias, Risks, and Limitations

• Throughput: ~0.05 tokens/sec. Decode is NPU-compute-bound per dispatch. Dispatch-count reduction (op-replication packing) was proven correct on silicon but does not move tok/s; a packed N-copy blob does N× the NPU compute. Reaching >1 tok/s requires IRON-level kernel fusion (future work).
• First-token latency. The first token of a turn takes ~2 minutes (prefill of a 43-layer, 284B model on this NPU); subsequent tokens stream at ~0.05 tok/s.
• Numerical residual. A recurrent hidden-state drift (~±40 RMS at deep layers) is intrinsic to the MXFP4 / MLA numerics of this checkpoint — it is reproduced in a pure-PyTorch HF ground-truth reference and is not a FaStar bug. Output remains coherent English with greedy prefill argmax matching the HF ground truth.
• Memory footprint. ~150 GB of expert weights; requires 64 GB RAM and ~200 GB NVMe SSD via expert paging. Expert cache thrash drops throughput if the working set exceeds RAM.
• Inherited biases. All biases, risks, and limitations of the upstream DeepSeek V4 Flash base model carry over unchanged — this is a lossy-format conversion, not an alignment intervention.

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations inherited from the upstream DeepSeek model, the very low decode throughput, and the on-device NPU-only target. Verify the DeepSeek license terms before redistribution or commercial use.

How to Get Started with the Model

Requirements: AMD Ryzen AI 9 365 (XDNA2 NPU), 64 GB RAM, ~200 GB NVMe SSD, Ubuntu 24.04 with XRT, AIEBU, and the AMDXDNA driver.

# Build the FaStar engine (CMake fetches header-only deps on first configure)
git clone https://github.com/RaffaelloMolinari/FaStar.git
cd FaStar
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)

# XRT lives in /usr on Ubuntu, not /opt/xilinx
export XILINX_XRT=/usr

# One-shot generation (greedy, deterministic):
./build/ds4_npu_engine --model deepseek_v4_dspark.fst \
                       --prompt "Explain quantum computing" --tokens 128 --temp 0.0

# Speculative decoding with the DSpark draft model:
./build/ds4_npu_engine --model deepseek_v4_dspark.fst --draft_model dspark_draft.fst \
                       --prompt "Hello world" --tokens 256

# Web UI / chat server:
./build/ds4_npu_engine --model deepseek_v4_dspark.fst --serve --port 8080

This HuggingFace repo must contain: the .fst file, its .fst.hc / .fst.norm / .fst.tid2eid sidecars, tokenizer.json, and (for speculative decoding) dspark_draft.fst. The .fst.norm and .fst.hc sidecars are required for coherent output.

Training Details

This model was not trained — it is a conversion of an existing checkpoint. Training details below describe the source model and the conversion procedure.

Training Data

Inherited from the upstream deepseek-ai/DeepSeek-V4-Flash-DSpark checkpoint. See that model card for the original training data.

Training Procedure

Preprocessing [optional]

Conversion to the .fst container (no weight fine-tuning):

python3 scripts/fst_converter.py --model deepseek-ai/DeepSeek-V4-Flash-DSpark \
                                 --output deepseek_v4_dspark.fst

The .fst format stores a page-aligned config header, shared tensors (attention, router, norms) in Q8_0 / BF16, and expert blocks as dense DS4 MXFP4 (17-byte blocks: 1 e8m0 scale + 16 FP4 nibbles). Verify integrity with scripts/verify_fst.py and scripts/check_fst.py.

Training Hyperparameters

• Training regime: N/A — not trained. Inference weights are MXFP4 (experts) + Q8_0/BF16 (shared tensors), dequantized to BF16 on-NPU for compute.

Speeds, Sizes, Times [optional]

• Parameters: ~284B (MoE).
• Layers: 43.
• Experts: 1376 total (~32 per layer) + 1 shared expert per layer, routed with top-k = 6 and hash routing over the first 3 layers (route_scale = 1.5).
• Hidden size: 4096; expert intermediate: 2048; vocabulary: 129,280.
• Attention: MLA — Q latent 1024, KV latent 512.
• Decode throughput: ~0.05 tokens/sec on Ryzen AI 9 365 NPU.
• Prefill latency: ~2 minutes for the first token of a turn.
• Container size: ~150 GB on disk (paged across SSD/RAM/NPU).

Evaluation

Testing Data, Factors & Metrics

Testing Data

Numerical correctness was verified against a pure-PyTorch HuggingFace ground-truth reference (hf_bos_ref.py, hf_gen_ref.py) using the real DeepSeek weights, and qualitative coherence was checked with free-form prompts (e.g. "The importance of NPU…").

Factors

Per-layer cosine similarity vs. the HF reference (MLA projections, FFN GEMM, dequantization); greedy prefill argmax token match; multi-token speculative-decoding acceptance rate; output coherence over a 43-layer prefill.

Metrics

• GEMM / dequant cosine similarity vs. reference: ~0.998 (MLA), ~0.998 (FFN), ~0.999998 (fused FFN gate, deterministic).
• Greedy prefill argmax: matches HF ground truth (e.g. argmax 671 = "The").
• Speculative-decoding acceptance: ~0.5–0.67 depending on configuration.

Results

The engine produces coherent English, and greedy prefill argmax matches the HF ground truth. A deep-layer hidden-state residual (~±40 RMS at layer 42) is reproduced identically in the pure-PyTorch HF reference and is intrinsic to this checkpoint's MXFP4/MLA numerics, not a FaStar defect.

Summary

FaStar is mathematically faithful to the HuggingFace reference for this checkpoint, at the cost of very low on-NPU throughput (~0.05 tok/s). Correctness holds; throughput is the open problem.

Model Examination [optional]

Per-layer cosine audits of MLA (wq_a, wkv_a, wo_a, output projection), FFN (gate/up/down GEMM), and MXFP4 dequantization are built into the repository (scripts/fst_layer_benchmark.py, scripts/verify_fst_weights.py, scripts/verify_fused_ffn.py, plus tools/ probes). These localize divergence to the intrinsic deep-layer residual noted above.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). This is an inference-only artifact; the dominant carbon cost was paid upstream during DeepSeek's original training and is not attributable to this conversion.

• Hardware Type: AMD Ryzen AI 9 365 laptop (XDNA2 NPU), 64 GB RAM, NVMe SSD.
• Hours used: Conversion + on-device verification only (single laptop, no cluster training).
• Cloud Provider: None — entirely on-device.
• Compute Region: Local consumer hardware.
• Carbon Emitted: Negligible for this artifact (inference/conversion on one laptop); upstream training emissions belong to DeepSeek-AI.

Technical Specifications [optional]

Model Architecture and Objective

Decoder-only MoE Transformer with Multi-head Latent Attention (Q latent 1024, KV latent 512), per-layer shared + routed experts (top-k = 6, hash-routed over the first 3 layers with route_scale = 1.5, swiglu_limit = 10), MXFP4 expert weights with e8m0 scales, Q8_0/BF16 shared tensors, and a V4 KV-compressor that streams 512-dim compressed KV rows. A small DSpark draft model enables speculative decoding. Objective: next-token text generation.

Compute Infrastructure

Hardware

• APU: AMD Ryzen AI 9 365 (XDNA2 NPU is the compute target; iGPU unused).
• RAM: 64 GB minimum (~54 GB peak RSS observed).
• Storage: NVMe SSD, ~200 GB free (experts live on SSD; read latency dominates miss cost).
• GPU: not required.

Software

• Ubuntu 24.04; XRT (Xilinx Runtime) at /usr; AIEBU assembler; AMDXDNA kernel driver.
• FaStar engine (ds4_npu_engine, C++17, CMake build).
• NPU kernels generated with AMD IRON / MLIR-AIE; prebuilt .xclbin + _insts.bin ship in the repo kernels/ directory.
• Python 3 + tokenizers for the HF BPE tokenizer bridge.

Citation [optional]

If you use this work, please cite the upstream DeepSeek model and the FaStar engine.

BibTeX:

@misc{deepseek-v4-flash-dspark,
  author       = {{DeepSeek-AI}},
  title        = {DeepSeek V4 Flash (DSpark)},
  howpublished = {\url{https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark}}
}

@misc{fastar,
  author       = {Raffaello Molinari},
  title        = {FaStar: Expert Virtual Memory Inference Engine for DeepSeek V4 Flash on AMD Ryzen AI NPU},
  howpublished = {\url{https://github.com/RaffaelloMolinari/FaStar}}
}

APA:

DeepSeek-AI. (n.d.). DeepSeek V4 Flash (DSpark). HuggingFace. https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark

Molinari, R. (n.d.). FaStar: Expert virtual memory inference engine for DeepSeek V4 Flash on AMD Ryzen AI NPU. GitHub. https://github.com/RaffaelloMolinari/FaStar

Glossary [optional]

• .fst: FaStar's page-aligned model container (config header + shared tensors in Q8_0/BF16 + dense DS4 MXFP4 expert blocks).
• Expert virtual memory: MoE experts treated as virtual-memory pages — stored on SSD, cached in RAM (ExpertPager LRU), uploaded to NPU scratch on demand.
• MXFP4: Microscaling FP4 — 16 FP4 nibbles per block plus one shared e8m0 scale (17 bytes per expert block).
• MLA: Multi-head Latent Attention — compresses Q (1024-d) and KV (512-d) into latent projections; the KV cache stores compressed latents + positional embeddings.
• DSpark: Draft-model speculative decoding — a small model proposes tokens that the main model verifies in a block.
• IRON / MLIR-AIE: AMD's compiler toolchain that generates the NPU kernels (.xclbin).
• hw_context: An AMDXDNA device context bound to an xclbin; the driver caps simultaneous contexts at 9 (managed by AiebuKernelCache).

More Information [optional]

See the FaStar repository README for the full architecture overview, CLI flags, web-UI API, kernel rebuild instructions, and known limitations.

Model Card Authors [optional]

Raffaello Molinari (.fst conversion + model card). Model weights and architecture by DeepSeek-AI.

Model Card Contact

Open an issue on the FaStar GitHub repository.