APUS-Reflex-4B-GGUF

APUS Reflex 4B-GGUF

Branding note: This model family is now APUS Reflex. Some weight filenames, script/module names, served-model aliases, and the separate evaluation dataset ID retain their historical identifiers for download and runtime compatibility; these do not indicate different model versions.

English | 中文 · Source model · Collection · GGUF collection · MLX collection · MLX (Apple Silicon)

GGUF conversions of APUS-Reflex-4B (revision 65797c526c27) for Ollama, llama.cpp and LM Studio on Linux, Windows and macOS (Metal).

APUS Reflex is a decision model: each request supplies a state, an instruction and 2–16 candidates; the model answers with one candidate label (A–P) and the application reads the distribution over those labels. It is not a chat model.

Files and parity

Every file was scored on the Frozen80 panel with identical prompt tokens and compared with the HF BF16 release run by its own runtime (full depth, 66/80 · 82.50%).

Rows are llama-server b11118 on an NVIDIA RTX PRO 6000 (CUDA) unless marked Apple M5 (Metal). Q8_0 is the recommended default; Q4_K_M uses an importance matrix. Quantized scores are measured separately and do not inherit the BF16 result. Frozen80 is a reused development panel, not a blind benchmark. Per-file details: evaluation/. Per-question rows (candidate probabilities, choice, correctness; join with Frozen80 by panel_index): evaluation/per-question/.

Ollama

ollama run hf.co/apus-ailab/APUS-Reflex-4B-GGUF:Q8_0 --think=false

Always disable thinking. Ollama renders this architecture with its built-in Qwen3.5 renderer (the Modelfile TEMPLATE is not applied), and that renderer opens a thinking block unless told otherwise, which breaks the trained no-thinking contract. Use --think=false on the CLI, "think": false in /api/chat / /api/generate, or send the fully rendered prompt with "raw": true. params sets temperature 0 and num_predict 1; a local Modelfile is included.

To get candidate probabilities, use the bundled client (raw mode):

python examples/openjev_local.py --backend ollama --model hf.co/apus-ailab/APUS-Reflex-4B-GGUF:Q8_0

Ollama returns at most 20 top_logprobs and cannot report named tokens, so the distribution is exact only when every candidate label is in the top 20 (Q8_0: 64/80 Frozen80 prompts); otherwise use the selected label or llama-server.

Measured with Ollama 0.34.3 and the Q8_0 file on an Apple M5 Mac (24 GB, Metal):

Pulling hf.co/apus-ailab/APUS-Reflex-4B-GGUF:Q8_0 directly gave the same result with think: false: 80/80 labels, 66/80 correct.

llama.cpp (exact distribution)

llama-server -m APUS-OpenJev-v1-4B-Q8_0.gguf -c 9216 -ngl 999
python examples/openjev_local.py --backend llama-server --url http://127.0.0.1:8080

examples/openjev_local.py renders prompts with openjev_contracts.py, the same contract used in training.

Conversion

• llama.cpp b11118 (e6ab7c1a4), convert_hf_to_gguf.py --no-mtp (the merged release has no MTP weights).
• Q4_K_M importance matrix: 448 training-course decisions, 64 per source, disjoint from Frozen80 (details, imatrix.gguf).
• All 1-D tensors, including GDN A_log / dt_bias and norms, stay F32 in every file (check).
• Scope: full depth only (the 16/20-layer low exit is not available), text only (no vision tower), probabilities are not calibrated.

License

Apache-2.0, inherited from the source model; see LICENSE. Base model: Qwen/Qwen3.5-4B.

Authors: gumpcheng (xDAN2099), zhangxu, APUS AI-LAB.