Downloads · 30 days
43
1% of all-time downloads
Goraint/Qwen3-8b-192k-Context-6X-Josiefied-Uncensored-MLX-AWQ-4bit
Qwen3-8b-192k-Context-6X-Josiefied-Uncensored-MLX-AWQ-4bit is a text generation model from Goraint. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
A 4-bit AWQ quantized version of Qwen3-8B optimized for efficient inference using the MLX library, designed to handle long-context tasks (192k tokens) with reduced resource usage. Retains core capabilities of Qwen3-8B…
Downloads · 30 days
43
1% of all-time downloads
All-time downloads
7.7K
Public
Parameters
8.2B
4.7 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors4.7 GB · 100%
How the weights are stored.
U328.2B · 100%
From the Hugging Face model README
A 4-bit AWQ quantized version of Qwen3-8B optimized for efficient inference using the MLX library, designed to handle long-context tasks (192k tokens) with reduced resource usage. Retains core capabilities of Qwen3-8B while enabling deployment on edge devices.
| Metric | Value |
|---|---|
| Model Size | ~4.38 GB (4-bit quantized) |
| Inference Speed | 30.58 tokens/sec (M1 MAX) |
| 112.80 tokens/sec (M3 ULTRA) | |
| gguf Q4_K_S | 8.14 tokens/sec (M1 MAX) |
| Context Support | 192,000 tokens |
You need to modify the prompt template to ensure compatibility with LM Studio's inference pipeline. Below is the required template structure:
{%- if tools %}
{{- '\/system\n' }}
{%- if messages[0].role == 'system' %}
{{- messages[0].content + '\n\n' }}
{%- endif %}
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
{%- for tool in tools %}
{{- "\n" }}
{{- tool | tojson }}
{%- endfor %}
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call>...</tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call>\n" }}
{%- else %}
{%- if messages[0].role == 'system' %}
{{- '\/system\n' + messages[0].content + '\/\n' }}
{%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- set tool_start = "⦅" %}
{%- set tool_start_length = tool_start|length %}
{%- set start_of_message = message.content[:tool_start_length] %}
{%- set tool_end = "⦆" %}
{%- set tool_end_length = tool_end|length %}
{%- set start_pos = (message.content|length) - tool_end_length %}
{%- if start_pos < 0 %}
{%- set start_pos = 0 %}
{%- endif %}
{%- set end_of_message = message.content[start_pos:] %}
{%- if ns.multi_step_tool and message.role == "user" and not(start_of_message == tool_start and end_of_message == tool_end) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endfor %}
{%- for message in messages %}
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
{{- '\/' + message.role + '\n' + message.content + '\/' + '\n' }}
{%- elif message.role == "assistant" %}
{%- set content = message.content %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
{%- set reasoning_content = message.reasoning_content %}
{%- else %}
{%- if '\/' in message.content %}
{%- set content = (message.content.split('\/')|last).lstrip('\n') %}
{%- set reasoning_content = (message.content.split('\/')|first).rstrip('\n') %}
{%- set reasoning_content = (reasoning_content.split('')|last).lstrip('\n') %}
{%- endif %}
{%- endif %}
{%- if loop.index0 > ns.last_query_index %}
{%- if loop.last or (not loop.last and reasoning_content) %}
{{- '\/' + message.role + '\n\n' + reasoning_content.strip('\n') + '\n\/\n' + content.lstrip('\n') }}
{%- else %}
{{- '\/' + message.role + '\n' + content }}
{%- endif %}
{%- else %}
{{- '\/' + message.role + '\n' + content }}
{%- endif %}
{%- if message.tool_calls %}
{%- for tool_call in message.tool_calls %}
{%- if (loop.first and content) or (not loop.first) %}
{{- '\n' }}
{%- endif %}
{%- if tool_call.function %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{{- '<tool_call>\n{"name": "' }}
{{- tool_call.name }}
{{- '", "arguments": ' }}
{%- if tool_call.arguments is string %}
{{- tool_call.arguments }}
{%- else %}
{{- tool_call.arguments | tojson }}
{%- endif %}
{{- '}\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '\/\n' }}
{%- elif message.role == "tool" %}
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
{{- '\/user' }}
{%- endif %}
{{- '\n⦅\n' }}
{{- message.content }}
{{- '\n⦆' }}
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
{{- '\/\n' }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '\/assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '
text-generationnot-for-all-audiences, conversational, mlx# Install MLX (Apple Silicon only)
pip install mlx
# Load model with Hugging Face Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Goraint/Qwen3-8b-192k-Context-6X-Josiefied-Uncensored-MLX-AWQ-4bit", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Goraint/Qwen3-8b-192k-Context-6X-Josiefied-Uncensored-MLX-AWQ-4bit")
prompt = "Explain quantum computing in simple terms."
inputs = tokenizer(prompt, return_tensors="pt").to("mps")
outputs = model.generate(**inputs, max_length=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Apache 2.0
Note: This model is a community contribution and may not be officially supported by Alibaba Cloud. Always validate outputs for accuracy and safety in production environments.