Qwythos-9B-v2-MLX-bf16

Qwythos-9B-v2-MLX-bf16

This repository, xunkutech-ai/Qwythos-9B-v2-MLX-bf16, contains a bfloat16 MLX conversion of empero-ai/Qwythos-9B-v2 for Apple Silicon inference with MLX-VLM and applications that use compatible MLX backends.

No additional fine-tuning was performed for this repository. The weights were converted from the new upstream v2 checkpoint to MLX-compatible safetensors while preserving the upstream Apache-2.0 license and model behavior.

v2 Upstream Notice

This conversion is built from the newer Qwythos-9B-v2 release, not from the earlier Qwythos-9B-Claude-Mythos-5-1M checkpoint.

According to the upstream model card, v2 is a robustness-focused update that:

• eliminates the repetition and degeneration observed under greedy or low-temperature decoding;
• preserves the reasoning capability of the earlier Qwythos release;
• restores the native Qwen3.5 Multi-Token Prediction (MTP) head in the upstream checkpoint;
• removes unsolicited identity preambles;
• retains the 1,048,576-token YaRN configuration and the Qwen3.5 multimodal-capable architecture;
• remains intentionally less refusal-oriented for research, cybersecurity, red-teaming, biology, chemistry, pharmacology, and clinical topics.

The upstream developers describe v2 as a hygiene and robustness release rather than a new capability tier.

Model Summary

• Repository: xunkutech-ai/Qwythos-9B-v2-MLX-bf16
• Format: MLX safetensors
• Precision: bfloat16
• Parameters: about 9B
• Architecture: Qwen3.5 hybrid vision-language model
• Text layers: 32
• Vision layers: 27
• Configured context length: 1,048,576 tokens with YaRN factor 4
• Original YaRN window: 262,144 tokens
• Primary use: local text-first reasoning and visual-language inference on Apple Silicon
• Upstream model: empero-ai/Qwythos-9B-v2
• Upstream revision converted: 2178f73a9b5ea28ccd8f6096ef6bac5cd59c9d8b
• MTP in this repository: excluded from the main model; use the standalone MTP drafter described below

The model retains the Qwen3.5 visual tower and image/video special tokens. However, the upstream card primarily reports text reasoning evaluations, so users should evaluate visual performance for their own workloads.

About the Upstream Model

Qwythos-9B-v2 is an Empero AI reasoning model built on Qwen3.5-9B. The v2 release applies FTPO (Final-Token Preference Optimization) to the earlier Qwythos checkpoint. Instead of broadly changing the output distribution, FTPO targets the token positions that begin repetition loops and trains the model to prefer coherent alternatives.

The upstream card reports approximately 2,000 automatically mined preference tuples, LoRA rank 256, alpha 128, a learning rate of 1.5e-5, and one training epoch with early stopping. The MTP head was restored from the Qwen3.5-9B base; the upstream card notes that it was not co-trained with the FTPO update.

Upstream-Reported Evaluation Results

The following results are copied in summary form from the upstream model card. They were measured with Empero AI's internal generative chain-of-thought harness and should not be treated as independently reproduced by this conversion.

See the upstream model card for its complete evaluation methodology, comparisons, sample generations, training details, and limitations.

Install

pip install -U mlx-vlm

or:

uv pip install -U mlx-vlm

This is a Qwen3.5 vision-language checkpoint and should be loaded with MLX-VLM. This card does not claim standalone MLX-LM compatibility.

Quick Start: Text-Only Generation

MLX-VLM supports text-only prompts by omitting the image argument:

mlx_vlm.generate \
  --model xunkutech-ai/Qwythos-9B-v2-MLX-bf16 \
  --prompt "Prove that there are infinitely many primes." \
  --max-tokens 2048 \
  --temperature 0.6 \
  --repetition-penalty 1.05 \
  --gen-kwargs '{"top_p": 0.95, "top_k": 20}'

For difficult reasoning, coding, tool-use, or long-context tasks, increase the generation budget.

Image Understanding

The converted checkpoint retains the upstream Qwen3.5 vision tower:

mlx_vlm.generate \
  --model xunkutech-ai/Qwythos-9B-v2-MLX-bf16 \
  --image /path/to/image.jpg \
  --prompt "Describe this image and explain the important details." \
  --max-tokens 1024 \
  --temperature 0.6

Multiple image paths can be passed after the image argument when supported by the installed MLX-VLM release.

Python Example

from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model_path = "xunkutech-ai/Qwythos-9B-v2-MLX-bf16"

# Load the MLX model and multimodal processor.
model, processor = load(model_path)
config = load_config(model_path)

# Use a local path, URL, or PIL image supported by MLX-VLM.
images = ["/path/to/image.jpg"]
prompt = "Analyze this image step by step."

# Format the prompt with the correct number of visual inputs.
formatted_prompt = apply_chat_template(
    processor,
    config,
    prompt,
    num_images=len(images),
)

result = generate(
    model,
    processor,
    formatted_prompt,
    images,
    max_tokens=1024,
    temperature=0.6,
    verbose=False,
)

print(result.text)

Recommended Sampling

The upstream model is designed to reason before answering. Its recommended starting point is:

generation_kwargs = {
    "temperature": 0.6,
    "top_p": 0.95,
    "top_k": 20,
    "repetition_penalty": 1.05,
    "max_tokens": 16384,
}

The upstream v2 card reports that repetition penalty is optional rather than load-bearing because the looping behavior was trained out. A smaller max_tokens value is usually sufficient for simple prompts; difficult reasoning tasks benefit from a larger budget.

Long Context

The model config contains a 1,048,576-token maximum position setting with static YaRN factor 4 over an original 262,144-token window.

Practical usable context depends on unified memory, prompt modality, KV-cache configuration, runtime version, and generation length. Static YaRN can also carry a short-context quality trade-off. Test the intended context length on the target machine rather than assuming the configured maximum is inexpensive or equally reliable for every workload.

Standalone MTP Drafter

The upstream v2 checkpoint restores 15 native MTP tensors. They are intentionally excluded from this main repository, which contains 760 main-model tensors. A companion MLX drafter can be stored separately as:

xunkutech-ai/Qwythos-9B-v2-MLX-bf16-mtp-draft

The standalone drafter uses model_type qwen3_5_mtp and contains only the sanitized MTP weights plus tokenizer files. With an MLX-VLM release that supports Qwen3.5 MTP speculative decoding, load it with:

mlx_vlm.generate \
  --model xunkutech-ai/Qwythos-9B-v2-MLX-bf16 \
  --draft-model xunkutech-ai/Qwythos-9B-v2-MLX-bf16-mtp-draft \
  --draft-kind mtp \
  --draft-block-size 4 \
  --prompt "Explain speculative decoding with MTP." \
  --max-tokens 2048 \
  --temperature 0.6

The companion identifier can also be replaced with a local drafter directory. Speculative-decoding speedup is workload- and hardware-dependent. The upstream card cautions that acceptance may be modest because the restored MTP head was not co-trained with the FTPO-updated main weights.

Conversion Notes

Conversion Provenance

• Source: empero-ai/Qwythos-9B-v2
• Source revision: 2178f73a9b5ea28ccd8f6096ef6bac5cd59c9d8b
• Source weights: one 19,306,305,296-byte safetensors file
• Output dtype: BF16
• Output tensor count: 760
• Output weight data: 18,819,627,488 bytes
• Output shards: 4
• Maximum shard tensor data: 5 GiB
• Additional training: none

The source checkpoint was converted with a streaming, GPU-independent safetensors converter. Unchanged BF16 tensors were copied without a full-model BF16-to-FP32 round trip. Only framework-required tensor transformations were applied.

Tensor Transformations

1. Hugging Face tensor names were remapped to MLX-VLM's Qwen3.5 namespace.
2. Qwen3.5 RMSNorm weights received the MLX-required +1 offset.
3. Linear-attention Conv1d weights were moved from [out, 1, kernel] to [out, kernel, 1].
4. The visual patch-embedding Conv3d weight was transposed from [1152, 3, 2, 16, 16] to MLX channels-last layout [1152, 2, 16, 16, 3].
5. The 15 upstream MTP tensors were omitted from this main-model variant and extracted separately for the companion drafter.

Compatibility Fixes Applied

1. partial_rotary_factor Location

The upstream config stores partial_rotary_factor at the text_config top level, while MLX-VLM Qwen3.5 code paths consume it from rope_parameters.

The converted config moves the value into:

{
  "text_config": {
    "rope_parameters": {
      "rope_type": "yarn",
      "factor": 4.0,
      "original_max_position_embeddings": 262144,
      "mrope_section": [11, 11, 10],
      "rope_theta": 10000000,
      "partial_rotary_factor": 0.25
    }
  }
}

This avoids errors reporting that rope_parameters is missing partial_rotary_factor.

2. vision_config.model_type

The upstream value qwen3_5_vision was normalized to qwen3_5 for compatibility with MLX-VLM versions and bundled MLX backends that use a narrower model-type whitelist.

3. Vision Conv3d Weight Layout

Hugging Face stores the visual patch projection in channels-first Conv3d layout:

[out, in, depth, height, width] = [1152, 3, 2, 16, 16]

MLX expects channels-last kernel layout:

[out, depth, height, width, in] = [1152, 2, 16, 16, 3]

Both the safetensors shape and the underlying BF16 data were transposed. This fix is required for model loading and downstream sensitivity measurement or quantization; changing only the metadata shape would be incorrect.

Verification

The conversion was independently checked without materializing the full model in memory:

• all 760 tensors are unique and BF16;
• no MTP tensor is present in this main-model repository;
• the safetensors index exactly matches the shard contents;
• each shard contains at most 5 GiB of tensor data;
• partial_rotary_factor is present in rope_parameters;
• vision_config.model_type is qwen3_5;
• every BF16 element of the converted visual patch projection matches the expected source transpose.

Files

Function Calling

The upstream Qwythos model uses the Qwen3.5 chat template and is intended for tool-augmented and agentic workflows. When a runtime accepts tool definitions, pass them through the tokenizer or processor chat-template interface and parse the emitted tool-call blocks in the application.

Tool execution, argument validation, permissions, and safety policy must be enforced outside the model.

Limitations and Safety

• This is a format conversion, not a new training run. It does not improve or independently validate the upstream model's capabilities.
• The benchmark values above are upstream-reported internal-harness results, not measurements reproduced by this conversion.
• The model may hallucinate facts, citations, identifiers, code behavior, or tool arguments. Use retrieval, execution checks, and human review when correctness matters.
• The upstream release is intentionally less refusal-oriented and may engage with sensitive medical, biological, chemical, or security topics. Add application-level safeguards and comply with applicable law.
• Do not use the model as the sole authority for medical, legal, financial, security-critical, or other high-stakes decisions.
• The configured one-million-token context can require substantial unified memory and has not been validated for every modality or deployment.
• The upstream card reports HumanEval below the raw Qwen3.5-9B base and notes that MTP acceptance may be modest.
• Visual-language quality was not independently benchmarked as part of this conversion.

License

This MLX conversion is released under the same Apache-2.0 license as the upstream model.

Acknowledgements

• Original model: empero-ai/Qwythos-9B-v2
• Original developer: Empero AI
• Base family: Qwen3.5-9B by Alibaba/Qwen
• MLX ecosystem: Apple MLX and MLX-VLM
• MLX conversion repository: xunkutech-ai/Qwythos-9B-v2-MLX-bf16