Bonsai-web-GGUF

Bonsai Web GGUF

Browser-ready, deterministic shards for the WaveCut/bonsai-webgpu-chat Space. The files are the pinned Prism ML Bonsai Q1_0 GGUF releases, re-sharded with llama-gguf-split --split-max-size 512M so every browser download stays below the current wllama single-file limit.

Contents

The repository also carries dflash/Qwen3.6-27B-DFlash-Q8_0.gguf (1,849,481,440 bytes) as a native correctness reference only. It is not part of a tier download and is not loaded by the browser Space.

manifest.json is the machine-readable contract: immutable source provenance, ordered shard paths, bytes, SHA-256, context defaults, WebGPU limits, chat template evidence, and the default-off DFlash research status. SHA256SUMS is the independent shard checksum list.

Source provenance

The shard container checksums differ from each original monolithic file because GGUF splitting adds split metadata. No quantization or tensor conversion is performed.

The first shard of every tier was loaded through llama.cpp's automatic multi-file resolver and compared with its monolithic source under greedy sampling. Token IDs and text match exactly: 64/64 tokens for 1.7B and 27B, and 16/16 tokens for 4B and 8B. The shorter middle-tier checks are load and reassembly guards, not quality evaluations.

Loading with wllama

Pass the ordered URLs from manifest.json to loadModelFromUrl:

await wllama.loadModelFromUrl(model.files.map((file) => file.url), {
  n_ctx: model.defaultContext,
  n_gpu_layers: 99999,
  flash_attn: false,
  useCache: true,
});

The 27B tier is WebGPU-only in the Space. The application must reject it when adapter limits, storage quota, or full-graph placement are insufficient; it must not silently fall back to CPU.

Runtime evidence

The release gate uses pinned llama.cpp 00fa7cb284cbf133fc426733bd64238a3588a33e and Dawn 18eb229ef5f707c1464cc581252e7603c73a3ef0. On Apple M2 Max, all required Q1_0 and Qwen3.5 hybrid WebGPU operation tests passed. Full-graph 1.7B and 27B inference requires the token embedding tensor to remain on WebGPU; Flash Attention stays disabled because quantized K/V combinations are not fully supported.

DFlash

DFlash remains inspection-only and default-off in the browser. The manifest pins the BF16 source and the bundled Q8_0 GGUF reference. Shape, complete tokenizer equality, target taps [2, 17, 32, 47, 62], conversion, and a 32-token temperature-0 native Bonsai-27B parity smoke pass.

The current custom wllama build contains the DFlash llama.cpp architecture, but its TypeScript/worker/glue load path does not yet expose spec_type=draft-dflash. The Q8_0 fc.weight is 139,264,000 bytes, above the 128 MiB baseline WebGPU storage-buffer binding limit. Browser activation, acceptance measurement, and a positive decode-speed result therefore remain hard release gates. The reference must not be presented as a browser-ready drafter.

License

The Bonsai model files are published under Apache License 2.0. The DFlash reference is MIT-licensed. The source license is included as LICENSE and tier/DFlash attribution is preserved under notices/.