Zwen-Prime

Zwen-Prime

An elite Full-Stack Principal Engineer in a 7B parameter body. Built by Zwen AI Labs.

Zwen-Prime is a two-stage model: a DARE-TIES neural fusion of three Mistral-7B-class specialists, followed by a supervised fine-tune on a hand-crafted five-corpus mixture that hard-installs strict <thinking> reasoning, Big-O discipline, native JSON function-calling, and zero-filler full-stack output. It is engineered for local inference on Apple Silicon and served via the official zwen-cli (with Ollama / llama.cpp as alternatives).

Model Summary

The Brain — Five-Corpus Training Mixture

Zwen-Prime is fine-tuned on a deliberate mixture of five datasets, each installing a distinct cognitive faculty:

The custom 550-row dataset (zwen_prime_master_dataset.jsonl) is the enforcement spine: 50 Identity rows that set the persona and 500 Algorithmic/Logic rows that lock in the <thinking>...</thinking> → raw-code template across advanced TypeScript generics, React/Next.js App-Router architecture, Java concurrency primitives, and Python system design.

Capabilities

• Strict reasoning core: Every non-trivial answer opens a <thinking> block with step-by-step logic and a time/space Big-O breakdown, immediately followed by the raw deliverable — no filler, no preamble.
• Full-stack mastery:
  • Python: typed (PEP 484/604), async-aware, concurrency-correct algorithms.
  • TypeScript: advanced generics, conditional/mapped types, type-stateful builders, discriminated unions.
  • React / Next.js: App Router, RSC, Server Actions, route handlers, caching/revalidation, React 19 hooks, streaming.
  • Java: ReentrantLock, StampedLock, Semaphore, CompletableFuture, ForkJoinPool, virtual threads, happens-before reasoning.
• Native function calling: Emits [TOOL_CALLS] raw JSON matching the provided tool schema; no markdown, no prose around the call.
• Mathematical reasoning: Derives quantitative claims in the scratchpad before asserting them; sanity-checks units, magnitudes, and boundaries.
• Long-context discipline: Anchors claims to source passages; refuses to confabulate when evidence is absent.
• Zero conversational filler: No greetings, sign-offs, apologies, or compliance narration.

Merge Details

Merge Method

The base was produced with the DARE TIES merge method via mergekit, using Mistral-7B-Instruct-v0.3 as the density-0 reference base.

Models Merged

• dphn/dolphin-2.9.3-mistral-7B-32k — reasoning specialist
• theprint/ReWiz-7B — fine-tune specialist
• mistralai/Mistral-7B-Instruct-v0.3 — base / reference (density 0, weight 0)

Merge Configuration

base_model: mistralai/Mistral-7B-Instruct-v0.3
merge_method: dare_ties
dtype: bfloat16
out_shard_size: 1.2B
parameters:
  density: 0.5
  weight: 1.0
  normalize: true
  int8_mask: true
  rescale: true
  lambda: 1.0
models:
  - model: dphn/dolphin-2.9.3-mistral-7B-32k
    parameters:
      density: 0.5
      weight: 1.0
  - model: theprint/ReWiz-7B
    parameters:
      density: 0.5
      weight: 1.0
  - model: mistralai/Mistral-7B-Instruct-v0.3
    parameters:
      density: 0.0
      weight: 0.0

Fine-Tune Stage

The merged base was then LoRA fine-tuned on the five-corpus mixture and the adapters permanently fused into the base weights (peft.merge_and_unload). The fused model is exported to GGUF for local serving.

DARE-TIES merge → LoRA SFT (5-corpus mixture) → merge_and_unload → GGUF → Ollama

ChatML-Native Format

Although the GGUF embeds the Mistral v0.3 chat template, Zwen-Prime's fusion corpus (Dolphin / ReWiz) is ChatML-formatted, so the model's native turn format is <|im_start|>system…<|im_end|><|im_start|>user…. Under Mistral [INST] framing it ignores the user prompt and runs on; under ChatML it reasons cleanly in <thinking> and stops on turn boundaries. Both official runtimes therefore pin the ChatML wrapper:

• zwen-cli pins ChatMLChatWrapper automatically — no configuration needed.
• The Ollama Modelfile injects the strict Zwen-Prime system prompt and the ChatML template via its TEMPLATE/SYSTEM directives.

The base merge is dolphin-2.9.3-mistral-7B-32k, and its 32k RoPE scaling is preserved intact — so Zwen-Prime officially supports a 32,000-token context window, enough to ingest a large enterprise codebase in a single pass.

Serving

Zwen-Prime is pre-quantized to Q4_K_M to run blisteringly fast on Apple Silicon and consumer GPUs (< 5 GB RAM required). The published quant is Zwen-Prime-Final.Q4_K_M.gguf (~4.2 GB).

Official CLI (recommended) — installs globally and auto-downloads the GGUF into ~/.zwen/models/ on first run, with a 32k context out of the box:

npm install -g @zwenailabs13/zwen-cli
zwen run zwen-prime
zwen chat          # interactive REPL
zwen list          # list cached models in ~/.zwen/models/

Ollama (alternative) — the Modelfile automatically injects the strict Zwen-Prime system prompt and handles the ChatML formatting:

ollama run zwenailabs/zwen-prime

llama.cpp (manual) — point any ChatML-aware runner at Zwen-Prime-Final.Q4_K_M.gguf and supply the Zwen-Prime system prompt.

System Prompt

Zwen-Prime ships with a dedicated system prompt that sets the Principal-Engineer persona, the <thinking> mandate, the zero-filler output protocol, full-stack mastery expectations, native JSON tool-calling rules, long-context discipline, and the hard constraints. It is embedded in the Modelfile's SYSTEM directive.

Intended Use

Zwen-Prime is intended as a local, autonomous engineering copilot: designing and shipping production code across Python, TypeScript, React/Next.js, and Java; reasoning through math and algorithms with verifiable steps; and calling tools via raw JSON when integrated into an agent runtime.

Limitations

• 7B scale: Strong on focused engineering tasks; not a frontier-class generalist.
• Wall-clock-sensitive: Derived math is verified symbolically in-context but not executed; verify numerically when stakes are high.
• Function calling: Follows the Mistral v0.3 / XLAM JSON convention and expects a tool-aware runtime to dispatch [TOOL_CALLS].
• Behavioral design: Fine-tuned toward zero-filler directness; users expecting conversational preamble will not get it.

License & Commercial Use

The weights and architecture of Zwen-Prime are released under the CC BY-NC 4.0 (Creative Commons Non-Commercial) license.

• Free for Developers: Individual developers, researchers, and hobbyists are fully encouraged to download, run, and modify Zwen-Prime locally for free.
• Commercial Restrictions: Cloud hosting, API provisioning, enterprise internal deployments, and any form of commercial reselling are strictly prohibited under this license.
• Enterprise Licensing: For commercial deployment, managed hosting, or enterprise use, you must acquire a commercial license through Axora.

Visit: https://axoraio.in Email: [email protected]