Downloads · 30 days
3
6% of all-time downloads
Echo9Zulu/Muse-12B-int4_asym-ov
Muse-12B-int4_asym-ov is a machine learning model from Echo9Zulu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Downloads · 30 days
3
6% of all-time downloads
All-time downloads
49
Public
Repo size
7 GB
Likes
0
Public
Click a slice to open those files.
.bin7 GB · 100%
From the Hugging Face model README
To run this model check out OpenArc.
This model was converted to OpenVINO IR using weight only compression to int4_asym.
Muse-12B has been exceptional so far.
It doesn't shy away from, um, intense topics and refusals aren't a problem. At this time we don't have access to all the samplers reccomended by LatitudeGames but I haven't seen massive degredation. Long context performance remains strong, and with some scaffolding could be a reliable workhorse, though sometimes a bit verbose.
Another interesting usecase has been tinkering inside the talk_to_llm.py. Using this demo hooks up Muse-12B with whisper and kokoro using the OpenArc server.
Very interesting way to expereince a text adventure.
Results were captured using openarc bench.
Very nice.
openarc bench selects input tokens by sampling the entire vocabulary using a similar approach to llama-bench.
input tokens: [512]
max tokens: [128]
runs: 5
benching... (5/5)
Muse-12B-int4_asym-ov
┏━━━━━┳━━━━━┳━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ run ┃ p ┃ n ┃ ttft(s) ┃ tpot(ms) ┃ prefill(t/s) ┃ decode(t/s) ┃ duration(s) ┃
┡━━━━━╇━━━━━╇━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ 1 │ 512 │ 128 │ 0.20 │ 28.92 │ 2623.5 │ 34.6 │ 3.87 │
│ 2 │ 512 │ 128 │ 0.16 │ 28.92 │ 3134.4 │ 34.6 │ 3.84 │
│ 3 │ 512 │ 128 │ 0.17 │ 28.89 │ 3007.7 │ 34.6 │ 3.84 │
│ 4 │ 512 │ 128 │ 0.17 │ 28.88 │ 3045.6 │ 34.6 │ 3.84 │
│ 5 │ 512 │ 128 │ 0.17 │ 28.91 │ 2998.3 │ 34.6 │ 3.84 │
└─────┴─────┴─────┴─────────┴──────────┴──────────────┴─────────────┴─────────────┘
Total: 5 runs
System:
Xeon W2255
128GB DDR4-ECC
Asrock A770
Ubuntu 24.04: 6.14.4-061404-generic
openvino 2025.3.0
openvino-genai 2025.3.0.0