Downloads · 30 days
158
33% of all-time downloads
EnclaveHost/qwen3.8-27b-mtp-q8-vl-gguf
qwen3.8-27b-mtp-q8-vl-gguf is a image-text-to-text model from EnclaveHost. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
An Enclave model-volume bundle containing the Q80 language model and its embedded native MTP head, the matching F16 vision encoder/projector, the official tokenizer, and version 2 calibration data for Enclave's shield…
Downloads · 30 days
158
33% of all-time downloads
All-time downloads
477
Public
Repo size
30 GB
Likes
0
Public
Click a slice to open those files.
.gguf30 GB · 100%
From the Hugging Face model README
An Enclave model-volume bundle containing the Q8_0 language model and its embedded native MTP head, the matching F16 vision encoder/projector, the official tokenizer, and version 2 calibration data for Enclave's shielded GPU backend.
The language GGUF and vision projector are downloaded from
Unsloth's Qwen3.8 GGUF release.
The tokenizer and Apache-2.0 license come from
Qwen's original model.
The weights are preserved byte for byte; this bundle does not requantize a Q4
or Q6 model into Q8. See provenance.json and SHA256SUMS for exact revisions
and file hashes.
Qwen3.8-27B-Q8_0.gguf: language model and native MTP head, 29,047,086,048 bytes.mmproj-F16.gguf: matching vision encoder/projector, 927,607,488 bytes.tokenizer.json: official Qwen3.8 tokenizer, 12,809,320 bytes.qwen3.8-27b-mtp-q8-vl-gguf.calib: calibrated activation exponents and outlier
channels for the shielded backend.engine-config.json: deployment config fragment for the Eyesoff app and engine.calibration/: public calibration fixtures and reproduction instructions.validation.json: results and limits of the local engine checks.The GGUF architecture is qwen35: 64 trunk layers (48 linear-attention layers
and 16 full-attention layers), plus one native MTP layer. GGUF block_count is
65 and nextn_predict_layers is 1. The hidden dimension is 5120, the model has
24 query heads and 4 KV heads, and the attention head dimension is 256.
The engine's sizing entry uses n_layers: 64, kv_layers: 17 (including MTP),
n_kv_heads: 4, head_dim: 256, and vocab: 248320.
The MTP tensors are embedded at blk.64.*. No separate draft model or draft
volume is needed. The mmproj filename lets the engine identify the projector
separately from the language GGUF.
Here, calibrated means calibration for Enclave's shielded fixed-point offload protocol. It does not mean that the upstream Q8_0 quantization was retrained or changed using an importance matrix.
The calibration is generated by wasm/ggml-shielded/shielded-calib against the
same pinned llama.cpp/ggml engine used by Enclave:
ddd4ec1428a6201e18975ea52b07c71e0f9aef26 (ggml 0.18.0). It observes public
prose, code, numeric text, image inputs, and native MTP observation/draft steps.
It uses the runtime's own per-output-column weight encoder and version 2
calibration format (SHIELDED_AF_DELTA=0). Sites unable to meet the tool's
fourfold calibration headroom target remain on the CPU.
The F16 vision encoder runs on the CPU on the shielded tier. Eligible calibrated Q8 language and MTP matrix operations can be distributed across the shielded GPU pool, with CPU fallback. Calibration includes language activations conditioned on the public image fixtures; it does not claim exhaustive input coverage or a formal model-quality benchmark.
Download this repository into a directory named
qwen3.8-27b-mtp-q8-vl-gguf, then build an attested Metal model volume:
node metal/volumes.mjs build qwen3.8-27b-mtp-q8-vl-gguf \
--src /path/to/qwen3.8-27b-mtp-q8-vl-gguf \
--gguf Qwen3.8-27B-Q8_0.gguf
The current manager selects shielded calibration by the model-volume name from
/opt/enclave/shielded/calib. The copy in the HF repository is not loaded
automatically merely by mounting the volume. Install it into the measured
engine image by copying it to
metal/shielded-overlay/calib/qwen3.8-27b-mtp-q8-vl-gguf.calib before building
and deploying the Metal image. A standalone engine test instead sets
SHIELDED_CALIB to this file explicitly.
Merge engine-config.json into the deployment config, attach the new volume,
and select qwen3.8-27b-q8. It enables MTP (nnLoadMtp: true, nnRsSeq: 1,
draft: "mtp", draft_tokens: 1) and vision. The settings are a starting
point; the fastest drafting depth depends on hardware and workload.
For a manually maintained MODEL_VOLUMES table, the entry is:
qwen3.8-27b-mtp-q8-vl-gguf:/models/qwen3.8-27b-mtp-q8-vl-gguf:Qwen3.8-27B-Q8_0.gguf
On Tinfoil, replace that path with the actual Modelwrap mount path. Pin the HF commit when wrapping the model. The HF upload does not itself register or switch a running Enclave deployment.
Budget approximately 30 GB for the language weights, projector, and tokenizer on disk, plus KV cache, recurrent state, MTP rewind snapshots, image workspace, and shielded working buffers at runtime. Aggregate VRAM alone is not the full RAM requirement of confidential inference. The native model context is 262,144 tokens; the actual deployment window must be sized to available memory.
Text, image understanding, and native MTP passed on the pinned engine. All three shielded GPU workers participated. The text test's full score arrays matched the exact CPU field-arithmetic oracle byte for byte. One-token MTP accepted all 12 drafted tokens and produced the same 24-token continuation as ordinary decoding; two-token MTP accepted 15 of 20 draft tokens and also matched. These are short smoke checks, not an accuracy evaluation.
Warm greedy decoding on an EPYC 9115 with 12 CPU threads, an RTX 3070 at its 50% compute share, and both V100-class cards at 100% compute share:
| Worker VRAM reservations (3070 / V100 / V100) | Ordinary decode | MTP, one draft token | MTP, two draft tokens |
|---|---|---|---|
| CPU only | 3.28 tok/s | Not measured | 3.62 tok/s |
| 4 / 12 / 12 GiB | 8.67–8.83 tok/s | 6.00 tok/s | 4.52 tok/s |
| 4 / 24 / 24 GiB | 5.71 tok/s | 5.64 tok/s | Not measured |
The smaller GPU reservation leaves the output projection on CPU; the larger reservation also offloads that projection. More GPU placement was slower in this test. These are shielded, verified offload measurements, not direct CUDA inference. Each test generated only 24 tokens from a five-token prompt, with a 1024-token context. Cold GPU weight registration and prefill are excluded. Transport used local loopback workers outside a confidential VM, so these are not live Metal0 throughput guarantees. The test held placement stable by raising contention-detection thresholds; deployed fallback behavior can differ.
The example enables one-token MTP to exercise the requested feature. For faster
ordinary decoding on this tested pool, set the model's draft to null; the
embedded MTP weights remain in the bundle. MTP is functional but did not improve
throughput on this GPU pool.
The short pooled tests reached roughly 53 GiB of host process RSS, before
allowing for other applications or a longer context. Size host RAM for the
engine, its shielded buffers, vision workspace, and the intended context;
aggregate VRAM is not a substitute for host RAM. Exact timings and resource
measurements are in validation.json.