krea-realtime-14b-mlx

SceneWorks/krea-realtime-14b-mlx

Turnkey, SceneWorks-converted weights of krea/krea-realtime-video (Krea Realtime 14B) — an autoregressive video generator distilled from Wan 2.1 T2V 14B via Self-Forcing — packaged for native Apple-Silicon (MLX) inference inside SceneWorks. This is not an original model; it is a format/dtype repackaging of the upstream release for first-class macOS use (no PyTorch at runtime).

Upstream generates frame-chunks left-to-right with a persistent KV cache (~4 steps/chunk, causal attention, 24 fps), with KV-cache recompute and block-causal masking as the clip-length stability mechanisms. Modalities: text-to-video and video-to-video.

What changed vs. upstream

Krea Realtime 14B ships transformer-only — 1095 F16 tensors, 14,288,491,584 parameters, in two equivalent on-disk layouts (a single krea-realtime-video-14b.safetensors with every key model.-prefixed, and a sharded transformer/ without the prefix). The text encoder, VAE and tokenizer are the stock Wan 2.1 components and are bundled here unchanged so each tier is self-contained.

• DiT → key-normalized (the outer model. prefix stripped; patch_embedding conv → Linear reshape, text/time_embedding Sequential rename, ffn.0/2 → fc1/fc2, the non-persistent freqs RoPE buffer dropped) and cast F16 → bf16, which is the dtype the upstream reference runs the DiT in. Emitted at three tiers (below). Tensor set and shapes are verified against the wan21_t2v_14b inventory — exactly 1095 tensors, no missing / extra / mis-shaped — before anything is written.
• Text encoder — stock UMT5-XXL (t5_encoder.safetensors, bf16), byte-identical to the encoder already published in SceneWorks/scail2-mlx; loaded by mlx_gen_wan::Umt5Encoder.
• VAE — stock Wan 2.1 z16 VAE (vae.safetensors, f32, stride 4×8×8); loaded by mlx_gen_wan::WanVae.
• Tokenizer — stock UMT5-XXL tokenizer.json.
• config.json — the Wan 2.1 14B geometry (dim 5120, 40 layers/heads, ffn 13824, in/out_dim 16, patch (1,2,2), text_len 512, text_dim 4096) plus the autoregressive fields (local_attn_size -1, sink_size 0, num_frames_per_block 3, kv_cache_num_frames 3, frame_seq_length 1560, seq_length 32760, denoising_step_list [1000, 937, 833, 625, 0], timestep_shift 5.0, do_kv_recomp true), and — on a packed tier — a quantization block.

Tiers

Each tier subdirectory is a complete, self-contained snapshot (DiT at that precision + the shared dense TE / VAE / tokenizer + config.json), so the shared components repeat per tier. Quantization is group-wise affine, group size 64, via MLX quantize, applied to exactly the per-block self/cross-attention q/k/v/o and FFN fc1/fc2 Linears (400 of them); the patch/text/time embeddings, norms, modulation tables, time_projection and the output head stay dense bf16. The tiers ship pre-quantized on disk — the loader builds the quantized Linears straight from the packs, with no dense-bf16 load transient.

Layout

q4/config.json                q8/config.json                bf16/config.json
q4/dit.safetensors            q8/dit.safetensors            bf16/transformer/dit-0000{1..7}-of-00007.safetensors
q4/t5_encoder.safetensors     q8/t5_encoder.safetensors     bf16/t5_encoder.safetensors
q4/vae.safetensors            q8/vae.safetensors            bf16/vae.safetensors
q4/tokenizer.json             q8/tokenizer.json             bf16/tokenizer.json

The q4/q8 tiers ship the DiT as a single dit.safetensors. The dense bf16 tier ships it as a sharded transformer/ directory (7 shards, ~4 GiB each) — the loader accepts either layout, and sharding keeps the largest object well under the single-file sizes HF discourages while making the download resumable per shard.

License & attribution

This repackaging redistributes upstream weights under the upstream license (Apache-2.0). All credit for the model belongs to the original authors; this repo exists solely to make Krea Realtime usable in SceneWorks on Apple Silicon.

• Model: https://huggingface.co/krea/krea-realtime-video (Krea AI, Apache-2.0)
• Code: https://github.com/krea-ai/realtime-video
• Stock Wan 2.1 components (text encoder / VAE / tokenizer, bundled unchanged): https://huggingface.co/Wan-AI/Wan2.1-T2V-14B-Diffusers — Wan 2.1 by the Alibaba Wan team, Apache-2.0. Built on UMT5-XXL.