Downloads · 30 days
44
11% of all-time downloads
acul3/qwen3.5-2b-id-meeting-summarizer-mlx-q4
qwen3.5-2b-id-meeting-summarizer-mlx-q4 is a text generation model from acul3. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.
MLX-quantized port of acul3/qwen3.5-2b-id-meeting-summarizer for on-device inference on Apple Silicon (Mac, iPad, iPhone via mlx-swift-lm).
Downloads · 30 days
44
11% of all-time downloads
All-time downloads
407
Public
Parameters
1.9B
1.1 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors1.1 GB · 98%
How the weights are stored.
U321.9B · 100%
From the Hugging Face model README
MLX-quantized port of acul3/qwen3.5-2b-id-meeting-summarizer for on-device inference on Apple Silicon (Mac, iPad, iPhone via mlx-swift-lm).
The fine-tune supports three prompt-selected tasks. See the source README for verbatim training prompts.
| Task | Format pass | Notes |
|---|---|---|
paragraph | 100% | single-paragraph Indonesian summary |
title_generator | 100% | takes a summary as input, returns ≤7-word title |
rich_summary | 50% (raw) / 85–95% (with guards) | full markdown breakdown with Overview / Conclusion / Action Items |
repetition_penalty=1.1Q4 quantization shifts the next-token distribution just enough that pure greedy decoding (do_sample=False, temp=0) falls into local repetition loops after the first few sentences. The source model at bf16/fp16 doesn't have this problem.
A repetition penalty of 1.1 fully fixes the loop without degrading content quality. This is required, not optional.
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler, make_logits_processors
model, tokenizer = load("acul3/qwen3.5-2b-id-meeting-summarizer-mlx-q4")
PROMPT = """You are a helpful assistant expert in writing.
You answer only with the result without explanation or pretext.
Please follow the instructions word by word obediently.
<transcript>
Audio Transcript:
{transcript}
</transcript>
Analyze and generate a summary based on the audio transcript above written in 1 paragraph.
... [see source README for the full verbatim prompt]
"""
# Qwen3.5's chat template emits <think></think> by default — fine-tune was
# trained on non-thinking outputs.
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": PROMPT.format(transcript=my_transcript)}],
add_generation_prompt=True,
tokenize=False,
enable_thinking=False,
)
out = generate(
model, tokenizer,
prompt=prompt,
max_tokens=2048,
sampler=make_sampler(temp=0.0),
logits_processors=make_logits_processors(repetition_penalty=1.1), # required
)
print(out)
import MLXLLM
import MLXLMCommon
let config = ModelConfiguration(
id: "acul3/qwen3.5-2b-id-meeting-summarizer-mlx-q4",
defaultPrompt: "Ringkas teks berikut."
)
let container = try await #huggingFaceLoadModelContainer(configuration: config)
let session = ChatSession(
container,
generateParameters: .init(
maxTokens: 2048,
temperature: 0.0,
repetitionPenalty: 1.1, // required, see above
repetitionContextSize: 64
)
)
let summary = try await session.respond(to: prompt)
The source HF repo ships an Unsloth save artifact with two non-standard features that block mainline mlx-lm from converting directly:
model.language_model.language_model.language_model.X instead of model.X. Mlx-lm's loader can't map these.The conversion script remaps text keys, drops vision tensors, flattens the VL config wrapper into a flat qwen3_5 text config, sets tie_word_embeddings: true (no lm_head in source), then runs mlx_lm.convert:
# Step 1: remap (see tools/convert-summarizer/remap_weights.py in the transkrip repo)
python remap_weights.py \
--src ~/.cache/huggingface/hub/models--acul3--qwen3.5-2b-id-meeting-summarizer/snapshots/<hash> \
--dst ./qwen35-2b-id-text-only-bf16
# Step 2: quantize with mainline mlx-lm
mlx_lm.convert \
--hf-path ./qwen35-2b-id-text-only-bf16 \
--mlx-path ./qwen35-2b-id-meeting-mlx-q4 \
-q --q-bits 4 --q-group-size 64
apache-2.0 (inherited from source).
acul3/qwen3.5-2b-id-meeting-summarizerunsloth/Qwen3.5-2B → upstream Qwen/Qwen3.5-2Bmlx-lm