Nex-N2.5-mini-GGUF

Nex-N2.5-mini GGUF

Community GGUF quantizations of nex-agi/Nex-N2.5-mini.

<div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;"> ☕ If this GGUF made your day easier, a coffee would make mine.<br> <a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br> I build and test these releases myself. Your coffee helps keep me going.<br> Thank you for supporting this work. </div>

About Nex-N2.5-mini

Nex-N2.5-mini is Nex-AGI's multimodal, agent-oriented model for long-horizon tasks. The upstream card describes the Nex-N2.5 family as supporting computer use, web browsing, visual grounding, coding, reasoning, and tool calling. It also documents image and video inputs through the official multimodal processor and chat template.

The upstream configuration identifies a Qwen3.5 Mixture-of-Experts model with 256 experts and 8 active experts per token, 40 text layers, and a 262,144-token text context configuration. The upstream repository presents the mini variant as a 35B-parameter BF16 model. The upstream deployment and benchmark details are available in the official model card.

This release contains text GGUF files plus a separate mmproj-Nex-N2.5-mini-F16.gguf vision projector. The local validation below uses a 4,096-token context and does not claim that the full configured context has been validated by this package.

Upstream Nex-N2.5 benchmark overview; the image and scores belong to the official model card.

This is a quantization-only release. No training, fine-tuning, merging, or weight modification other than GGUF conversion and quantization was performed. The Q8_0 file was quantized directly from the converted BF16 GGUF; the other published files used the same BF16 source and a model-specific importance matrix. No GGUF file was used as the source for another quantization.

Fidelity measurements

The table below compares every published text GGUF with the BF16 reference on a held-out WikiText pilot: eight chunks from wiki.test.raw and eight chunks from wiki.valid.raw, with a 4,096-token context, 512 batch/ubatch, 64 CPU threads, and the same Qwen3.5-compatible llama.cpp runtime. Values are averaged across the two splits. Lower Mean KLD, ΔPPL, and RMS Δp, and higher Top-1 agreement, indicate closer next-token behavior to BF16. The BF16 reference mean PPL was 6.684743 in this pilot.

For a general local profile, Q4_K_M is the practical starting point in this pilot. IQ4_NL and IQ4_XS are compact Q4-region alternatives. Q5_K_M and Q6_K are stronger quality/size choices, while Q8_0 is the highest-bit option. Q3_K_L, Q3_K_M, IQ3_M, and IQ3_S are lower-memory Q3-region compromises. Q2_K, Q2_K_S, IQ2_XS, IQ1_M, and Q1_0 are memory-constrained experimental profiles and should be checked against the intended workload.

These measurements describe next-token fidelity relative to BF16; they are not a direct percentage of capabilities retained. Instruction following, reasoning, multilingual behavior, formatting, vision, and tool-calling quality can vary by workload and should be evaluated separately when they matter.

The compact machine-readable results are available in reproducibility/quality-summary.tsv. Corpus hashes, conversion details, evaluation settings, runtime provenance, and artifact hashes are recorded in reproducibility/manifest.md.

Quick start

./llama-cli \
  -m Nex-N2.5-mini-Q4_K_M.gguf \
  --chat-template-file chat_template.jinja \
  --jinja \
  --reasoning off \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99 --cpu-moe --fit on --fit-target 1024

--cpu-moe keeps the MoE weights on the CPU and is useful when the available GPU memory is smaller than the model working set. Omit it when the target machine has enough memory and the runtime configuration has been tested for that setup.

For the multimodal path, keep the text GGUF and the separate projector beside the executable:

./llama-mtmd-cli \
  -m Nex-N2.5-mini-Q4_K_M.gguf \
  --mmproj mmproj-Nex-N2.5-mini-F16.gguf \
  --image path/to/image.jpg \
  --jinja \
  -p 'Answer briefly in English: What is the main subject of this image?' \
  -n 64 -c 4096 -ngl 99 --cpu-moe --fit on --fit-target 1024

Reproducibility and validation

The source was locked to upstream revision 87420286149d9cce9bd46cd335ef9bda33c37c1b and converted directly from the upstream BF16 safetensors. The text converter used --no-mtp because this revision advertises MTP configuration but does not contain MTP tensors. The vision projector was converted separately to F16.

All fifteen published text GGUF files passed llama.cpp tensor checks, load, and English generation smoke tests. The BF16 reference also passed the same text smoke profile. The Q4_K_M text file and the included F16 projector passed a multimodal image smoke test. Runtime throughput is supplementary and is available as the compact reproducibility/runtime-summary.tsv; it is not a quality score or a replacement for the fidelity table.

Raw conversion, calibration, quantization, smoke-test, fidelity, and benchmark logs remain local and are intentionally not uploaded. Published artifact checksums are in SHA256SUMS.txt.

License and attribution

The upstream model metadata specifies Apache License 2.0. Preserve the upstream attribution and license when redistributing these derivative GGUF artifacts. These are community GGUF quantizations, not an official nex-agi/Nex-N2.5-mini release or endorsement.