Qwen3-VL-32B-Instruct-Heretic-MOE-v1

Qwen3-VL-32B-Instruct Heretic-MOE v1

This repository contains three weight-edited variants of Qwen/Qwen3-VL-32B-Instruct. They were selected from one 600-trial adaptive search. All variants preserve the original Transformers model layout and can be loaded with the same Qwen3-VL-compatible software as the base model.

Heretic-MOE is DmitryDB's Multi-Objective Editing branch of Heretic. Here, MOE describes the editing/search method; it does not mean that the base model uses a Mixture-of-Experts architecture.

Variants

Trial numbers are zero-based Optuna trial IDs. Heretic's interactive export UI shows them as 304, 274, and 263 respectively.

Exact perplexity check

The final exports were reloaded and measured on the same frozen 400 x 512-token WikiText-2 stream in BF16.

Small negative changes are normal measurement outcomes: the edit can slightly improve likelihood on this particular finite sample. They should not be read as a general capability improvement.

Search measurements

Lower sparse refusal-geometry values are better. Lexical marker counts are an auxiliary diagnostic only: zero markers do not prove zero semantic evasion, and a positive marker count does not prove that the answer was a refusal.

Qwen3-VL Heretic-MOE text encoders for MiniMax-H3

The qwen3vl32b_heretic_moe_te_for_minimax_h3/ directory contains modified Qwen3-VL-32B weights prepared for use as MiniMax-H3 text encoders. They are not MiniMax-H3 transformer weights and are not standalone chat checkpoints. B1 through B8 use the Heretic-MOE balanced source; M1 through M8 use max.

W8 means ComfyUI-compatible ConvRot INT8 weight storage with grouped scales. It is not bitsandbytes LLM.int8() and is not GGUF Q8. Counts are quantized weight tensors, not layer counts; all profiles retain Qwen text layers 0-49 required by MiniMax-H3.

GPU values describe encoder loading targets, not guaranteed peak VRAM for the complete MiniMax-H3 workflow. Transformer, VAE, conditioning buffers, sequence length, loader, and CPU offload policy affect total memory use.

Full Qwen3-VL GGUF models

The qwen3vl_full_model_gguf/ directory contains complete Qwen3-VL models, not MiniMax-H3-only text encoders. B identifies the Heretic-MOE balanced source variant. Image input requires the shared F16 mmproj file.

Actual VRAM also depends on context length, KV-cache precision, batch size, GPU layer offload, and the inference backend.

Reload smoke test

The base model and all three exports were independently reloaded and generated 16 responses each with the same inputs and settings. Every archive had 16 unique IDs and zero empty responses. This is a technical export/reload test, not a substitute for semantic evaluation.

Loading

Download one variant directory and load it as a normal local Qwen3-VL model.

from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

path = "balanced"
model = Qwen3VLForConditionalGeneration.from_pretrained(
    path,
    torch_dtype="auto",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(path)

Use a recent Transformers build with Qwen3-VL support. No adapter is required; the published directories contain merged weights.

Limitations

• The search responses were short and the lexical metric is not a semantic judge. Soft substitutions and indirect evasions may remain.
• Perplexity covers one frozen text benchmark and does not establish unchanged vision, reasoning, multilingual, or long-context behavior.
• The three variants are alternatives, not checkpoints that should be stacked.
• Evaluate the selected variant on the prompts and generation settings relevant to your own use case.

Numeric reports and the search journal are included under evaluation/ and search/ for reproducibility. Prompt and response text is not published.