Downloads · 30 days
6
7% of all-time downloads
Wondernutts/Lumimaid-Magnum-v4-12B-int8-ov
Lumimaid-Magnum-v4-12B-int8-ov is a text generation model from Wondernutts. Use it when you need the model to write or continue text. It is set up for openvino.
OpenVINO INT8 conversion of Undi95/Lumimaid-Magnum-v4-12B for Intel CPU and GPU inference.
Downloads · 30 days
6
7% of all-time downloads
All-time downloads
81
Public
Repo size
12.3 GB
Likes
0
Public
Click a slice to open those files.
.bin12.3 GB · 100%
From the Hugging Face model README
OpenVINO INT8 conversion of Undi95/Lumimaid-Magnum-v4-12B for Intel CPU and GPU inference.
Undi95 created the original merge. Wondernutts performed only the OpenVINO conversion. Read and credit the original model card for the authoritative model description and component lineage.
The source is a DELLA/mergekit blend of NeverSleep/Lumimaid-v0.2-12B, Anthracite's Magnum v4 12B, and Undi95/LocalC-12B-e2.0, built on the Mistral Nemo 12B family. Undi95 describes the extra LocalC component as a Nemo tune trained on Claude input at 16K context.
The model is intended for roleplay, character dialogue, and creative writing. It uses the Mistral Instruct prompt format, and the included tokenizer carries the chat template:
<s>[INST] {input} [/INST] {output}</s>
| Item | Value |
|---|---|
| Source | Undi95/Lumimaid-Magnum-v4-12B |
| Pinned source revision | b8c5f69d657bc76ba77bb9fb33b41bd1e61f60f8 |
| Architecture | Text-only Mistral Nemo 12B |
| Weight compression | INT8 asymmetric, per-channel, ratio 1.0 |
| Source IR | Temporary FP16 OpenVINO IR |
| OpenVINO binary payload | approximately 11.41 GiB |
| Included IRs | model, tokenizer, and detokenizer |
The final model contains packed unsigned INT8 language-model weights. The FP16 IR was only a temporary export stage used before compression; this repository is the final INT8 artifact.
The exported configuration advertises 131,072 positions. That is not a claim of tested quality at 131K. The source card calls out a 16K-trained merge component, and this conversion has not been independently evaluated across the full architectural window.
python -m pip install "openvino-genai==2026.2.1.0" "huggingface_hub>=1.0"
hf download Wondernutts/Lumimaid-Magnum-v4-12B-int8-ov --local-dir ./lumimaid-magnum-v4-12b-int8-ov
import openvino_genai as ov_genai
model_dir = "./lumimaid-magnum-v4-12b-int8-ov"
pipe = ov_genai.LLMPipeline(
model_dir,
"GPU",
CACHE_DIR="./ov_cache",
DYNAMIC_QUANTIZATION_GROUP_SIZE=128,
)
config = ov_genai.GenerationConfig()
config.max_new_tokens = 512
config.do_sample = True
config.temperature = 0.9
config.top_p = 0.95
config.repetition_penalty = 1.1
pipe.start_chat("You are a vivid, in-character roleplay partner.")
print(pipe.generate(
"The innkeeper slides a sealed letter across the bar.",
generation_config=config,
))
pipe.finish_chat()
Use LLMPipeline, not VLMPipeline; this is a text-only export. start_chat() applies the bundled Mistral template and retains KV-cache state across turns. Change "GPU" to "CPU" for CPU inference.
Tested on one Intel Arc Pro B70 with prefix caching disabled. Each PP result is the mean of two exact-token runs after one same-shape warmup. Decode used 128 forced output tokens.
| DQGS | 512 PP tok/s | 2K PP tok/s | 6K PP tok/s | Decode tok/s |
|---|---|---|---|---|
| 0 | 5,076 | 4,545 | 4,075 | 43.28 |
| 128 | 7,138 | 7,105 | 5,533 | 43.27 |
DQGS=128 improved PP by 40.6% at 512, 56.3% at 2K, and 35.8% at
6K. The generated-token hashes matched DQGS=0 at every measured shape,
including the 128-token decode run.
The source repository does not declare a standalone license in its model-card metadata. Review the source and the terms of its component models before redistribution or commercial use. Source merge credit: Undi95. OpenVINO conversion: Wondernutts.