Qwen3.8-9B-Empero-Q6_K-llamafile

Qwen3.8-9B Empero Q6_K llamafile

Artifact identity

This is a local, self-contained, Empero-named llamafile package of the existing Qwen3.8-9B-Q6_K.gguf from Empero AI's direct GGUF repository, empero-ai/Qwen3.8-9B-GGUF. Empero AI is credited here as the developer/publisher of the distilled model and the publisher of the direct quantized GGUF. The package embeds the exact admitted Q6_K GGUF bytes together with the full llamafile 0.10.5 runtime and packaged arguments. This package did not itself train, fine-tune, merge, convert, compress, re-quantize, edit, or otherwise change the model weights.

The .llamafile is executable code plus model data, not merely a weights file. Treat it as software: verify the SHA-256 before use and obtain it only through a trusted transfer path. The local build and naming revision were archive-verified without loading or running the model for inference. The build script itself performs no upload or publication; any later Hub transfer does not alter the verified local artifact bytes.

Lineage and what this package changes

Exact model-tree chain: this Empero-named llamafile → empero-ai/Qwen3.8-9B-GGUF (direct quantized GGUF source) → empero-ai/Qwen3.8-9B (Empero AI full-parameter fine-tune/distill) → Qwen/Qwen3.5-9B (base).

This package is one derivative level below the Empero GGUF in the Hugging Face model tree. It embeds the existing Q6_K bytes unchanged and did not itself re-quantize, train, merge, or fine-tune them.

Expanded lineage:

1. this Empero-named llamafile package;
2. the direct quantized GGUF repository empero-ai/Qwen3.8-9B-GGUF at revision 760121cd70bb4c36b2b5ec58eb765e0df5987efe;
3. the corresponding model repository empero-ai/Qwen3.8-9B at revision 0934f3d2327ff2df2197495278c4c46ae5a56bd9;
4. the original base Qwen/Qwen3.5-9B.

The base_model frontmatter intentionally points to the direct GGUF source and uses base_model_relation: quantized. That records the artifact actually embedded and does not imply that this packager performed the Q6_K quantization. The only model-artifact change made here is packaging the already-quantized Empero-published bytes with a runtime and defaults; the later Empero naming revision changed only package metadata, documentation, and external filenames.

Empero AI claims:

Empero AI describes Qwen3.8-9B as a 9B full-parameter supervised fine-tune and off-policy distillation trained on approximately 70,000 curated teacher traces, with a native context length of 262,144 tokens.

Empero AI describes it as the text path of a vision-language-model base, with a text-only fine-tune; vision is inherited from the base but was not evaluated by Empero AI.

These are attributed publisher claims. This package builder did not independently validate the training recipe, dataset size, benchmark results, capabilities, native-context quality, or inherited vision behavior.

Model and package details

Native context is a model/publisher property. The 8,192-token packaged default is deliberately lower to reduce initial memory demand. Increasing it does not guarantee usable speed or quality and requires additional memory, especially for the KV cache.

Components and exact provenance

Final artifact size: 7,909,753,843 bytes (7.910 GB / 7.367 GiB). The embedded model begins at byte offset 350,814,208 and the embedded .args begins at byte offset 7,909,736,448; both are stored/uncompressed and aligned to 65,536-byte boundaries. See BUILD-MANIFEST.md and SHA256SUMS.txt for the reproducibility ledger.

No projector exists in the inspected direct GGUF repository, so none was downloaded, copied, or embedded.

Packaged defaults and overrides

The exact retained and embedded .args is:

-m
/zip/Qwen3.8-9B-Q6_K.gguf
--ctx-size
8192
--jinja
--temp
0.6
--top-p
0.95
--top-k
20
--min-p
0.0
...

The final ... passes user arguments through after the packaged defaults. For example, a machine with enough memory may request a larger context by appending --ctx-size 16384; command-line options supplied by the user are intended to override the packaged value. The sampler defaults are temperature 0.6, top-p 0.95, top-k 20, and min-p 0.0. --jinja enables the packaged runtime's Jinja chat-template handling.

Terminal use

These commands are usage instructions only; they were not executed during this packaging task.

cd /path/to/Qwen3.8-9B-Empero-Q6_K-llamafile
chmod +x Qwen3.8-9B-Empero-Q6_K.llamafile
./Qwen3.8-9B-Empero-Q6_K.llamafile -p "Write a short greeting." -n 128

Override defaults only after considering memory requirements:

./Qwen3.8-9B-Empero-Q6_K.llamafile --ctx-size 16384 -p "Summarize this text: ..." -n 256

On macOS, Gatekeeper, quarantine attributes, filesystem mount options, or enterprise policy can prevent execution even when the executable bit is set. Resolve those controls only if you trust the file and its verified hash.

Local web UI

Start an explicitly loopback-bound server:

./Qwen3.8-9B-Empero-Q6_K.llamafile --server --host 127.0.0.1 --port 8080

Then open http://127.0.0.1:8080/. A port already used by another process should not be reclaimed; select a different unused port instead.

OpenAI-compatible API

With the loopback server running, the API base is http://127.0.0.1:8080/v1. For example:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"local","messages":[{"role":"user","content":"Hello"}],"temperature":0.6}'

Client compatibility depends on the llamafile 0.10.5 API surface; not every OpenAI API feature is guaranteed.

Portability

The single .llamafile contains the runtime, Q6_K weights, and defaults. It can be copied together with the small documentation files to a USB drive or portable SSD without installing Python or a model manager on the destination machine. The destination still needs a supported CPU/OS, enough RAM, permission to execute the file, and a filesystem/mount that supports large files and executable permissions. FAT32 cannot store this artifact because FAT32 has a 4 GiB per-file limit. APFS, exFAT, NTFS, and suitable Linux filesystems can store it, although execute permissions and mount behavior vary; copying to a local executable filesystem is the safest fallback.

Always verify the artifact SHA-256 after copying. Portable storage does not make an executable trustworthy and does not remove model-license obligations.

Platform and memory guidance

Llamafile is intended for portable local execution on supported macOS and Linux systems and CPU architectures covered by the bundled runtime. Hardware acceleration, instruction-set support, and performance vary by host. The artifact alone is about 7.367 GiB, and runtime memory use is higher than the file size because execution also needs model mappings, KV cache, runtime workspaces, and operating-system headroom.

A 16 GiB-class Mac is a practical starting point for the packaged 8,192-token context when other memory-heavy applications are closed, but it is not a guarantee; more memory is preferable. Larger contexts, longer batches, or concurrency need additional memory and may sharply reduce speed. If memory pressure is high, keep the packaged context or lower it rather than assuming the 262,144-token native maximum is practical on a given machine.

Text-only packaging and vision status

The source model is described by Empero AI as the text path of a VLM base. The fine-tune is text-only, and Empero AI says inherited vision was not evaluated. The direct GGUF repository contains no projector for this package. Accordingly, this llamafile includes no projector, does not enable a projector, is documented only for text generation, and should not be expected to accept or understand images. Runtime support for multimodal features in other packages does not change this artifact's contents.

Security

Bind servers to 127.0.0.1 unless remote access is deliberately required and separately protected. Binding to 0.0.0.0 or another non-loopback address can expose prompts, outputs, and model service access to the local network; a bare local server should not be assumed to provide authentication, TLS, rate limits, or tenant isolation. Use a properly configured authenticated reverse proxy and host firewall before any intentional network exposure.

Model-generated output is untrusted. Do not automatically execute generated commands, disclose secrets in prompts, or treat outputs as authoritative professional advice. The executable itself should be hash-verified before running.

Windows status

This artifact is above the greater-than-4-GiB direct-Windows executable limit applicable to this packaging format. Direct Windows .exe execution of this .llamafile is unsupported. Windows users should use a compatible standalone runtime (for example a supported standalone llamafile/llama.cpp build) with the external Qwen3.8-9B-Q6_K.gguf weights rather than renaming or directly executing this file. The external GGUF must be obtained and hash-verified separately.

Limitations

• No inference or execution check was performed by this build worker; archive identity and structure were verified without model loading.
• Publisher training, benchmark, capability, native-context, and vision claims were not independently validated.
• Q6_K is lossy relative to higher-precision weights; this packager did not measure quality loss.
• Local performance and memory requirements vary by hardware, runtime path, prompt length, context, and concurrency.
• The 8,192-token default is conservative; the 262,144-token publisher-described native context is not a promise that the maximum is practical in this package on every host.
• No projector is present, so this package is text-only despite VLM lineage.
• Hosted inference is disabled because Hugging Face hosted inference does not execute this standalone llamafile artifact.
• LLM outputs can be incorrect, biased, unsafe, or fabricated; evaluate them for the intended use.

License and attribution

The direct model repositories identify Apache License 2.0. The complete Apache-2.0 text is included as LICENSE. Review the upstream model repositories and the included license before redistribution or deployment. Packaging does not change the model's upstream license or erase third-party runtime/tool obligations.

• GGUF source: https://huggingface.co/empero-ai/Qwen3.8-9B-GGUF/tree/760121cd70bb4c36b2b5ec58eb765e0df5987efe
• model source: https://huggingface.co/empero-ai/Qwen3.8-9B/tree/0934f3d2327ff2df2197495278c4c46ae5a56bd9
• original base: https://huggingface.co/Qwen/Qwen3.5-9B
• llamafile project: https://github.com/Mozilla-Ocho/llamafile