glm-edge-1.5b-chat-mlx-4bit

GLM-Edge-1.5B-Chat (4-bit MLX Quantized)

This repository contains Zhipu AI's GLM-Edge-1.5B-Chat quantized to 4-bit precision. It is compiled natively for Apple Silicon under the MLX framework.

4-bit quantization maximizes token generation speed and minimizes memory usage. It is the optimal setup for low-resource environments.

Performance Benchmarks

• Inference Speed: ~72.56 tokens per second (base M1 Apple Silicon)
• VRAM Footprint: ~0.87 GB
• Memory Efficiency: Runs comfortably on 8GB Unified Memory setups alongside heavy IDEs.

Installation

Install the MLX LM package.

pip install mlx-lm

Usage

Command Line Interface

Chat with the model in your terminal.

mlx_lm.chat --model SirSahOl/glm-edge-1.5b-chat-mlx-4bit

Python API

Load and generate text programmatically.

from mlx_lm import load, generate

model, tokenizer = load("SirSahOl/glm-edge-1.5b-chat-mlx-4bit")

messages = [{"role": "user", "content": "Explain quantum superposition."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

response = generate(model, tokenizer, prompt=prompt, verbose=True)

Multi-Quantization Comparison

Evaluate your hardware budget and choose the optimal precision:

Limitations

• 4-bit group-wise quantization introduces minor logical degradation. For complex code refactoring or multi-turn structured reasoning, consider the 8-bit or 16-bit variants.

LM Studio Configuration (Universal Preset Fix)

If you load this model in LM Studio, you must configure custom Stop Strings to prevent the model from entering an infinite self-dialogue loop.

Option A: Automatic Preset (Recommended)

You can create a custom prompt preset to configure all settings automatically. Create a JSON file named GLM-Edge.json inside your LM Studio config directory:

• macOS / Linux: ~/.lmstudio/config-presets/GLM-Edge.json
• Windows: %USERPROFILE%\.lmstudio\config-presets\GLM-Edge.json

Add the following JSON content:

{
  "name": "GLM-Edge",
  "inference_params": {
    "pre_prompt": "You are a helpful, direct, and honest assistant.",
    "input_prefix": "<|user|>\n",
    "input_suffix": "\n<|assistant|>\n",
    "pre_prompt_prefix": "<|system|>\n",
    "pre_prompt_suffix": "\n",
    "antiprompt": [
      "<|user|>",
      "<|observation|>",
      "<|endoftext|>"
    ],
    "stopStrings": [
      "<|user|>",
      "<|observation|>",
      "<|endoftext|>"
    ],
    "temperature": 0.7,
    "max_tokens": 2048
  }
}

Restart LM Studio, open a Chat session, and select "GLM-Edge" from the Prompt Template dropdown.

Option B: Manual Configuration

Alternatively, configure the settings manually in the Advanced Configuration sidebar:

1. Stop Strings (Antiprompts / stopStrings): Add <|user|>, <|observation|>, and <|endoftext|>
2. Prompt Formatting:
   • User Prefix: <|user|>\n
   • Assistant Suffix: \n<|assistant|>\n
   • System Prefix: <|system|>\n
   • System Suffix: \n