Qwen3-4B-NURL

Qwen3-4B-NURL

Qwen3-4B finetuned to write NURL — a compiled systems programming language with terse prefix notation, compile-time ownership with automatic drops, and a self-hosted compiler.

The base model knows nothing about NURL (it invents Rust-like syntax); this model writes idiomatic NURL:

@ sum_vec ( Vec i ) v → i {
    : i n ( vec_len [i] v )
    : ~ i total 0
    : ~ i k 0
    ~ < k n {
        : i val ( vec_get [i] v k )
        = total + total val
        = k + k 1
    }
    ^ total
}

Training

Trained entirely with nurllama — the pure-NURL LLM engine — on a single RTX 4090 (24 GB):

nurllama finetune Qwen3-4B-Q8_0.gguf train.txt \
    --steps 4000 --seq 56 --rank 16 --alpha 32 --lr 2e-4 \
    --mixed --stream --window-stride 0 \
    --checkpoint ckpt.st --save-every 200 --resume

• LoRA rank 16, α 32 on q/k/v/o + gate/up/down of all 36 layers, trained over a grad-tape capture replayed on the device in mixed precision (f32 storage, f64 accumulation).
• Data: a 7.5M-token corpus generated from the NURL repository — chat samples (function completion, doc Q&A, spec excerpts) plus raw source, all derived from ground truth (compiler test goldens, hand-written docs); nothing model-generated. --window-stride 0 samples the whole corpus evenly with a golden-ratio window permutation.
• Merged with nurllama finetune --merge-only: base + (α/r)·A·B written as this repo's sharded model-*.safetensors (F32, true HF lane order — loads in transformers unchanged via model.safetensors.index.json).
• nurl-lora-adapters.safetensors carries the raw LoRA pairs in nurllama's own format (blk.<L>.<proj>.lora_a/b), usable with nurllama run <base.gguf> --weights after a --merge-only, or re-merged against the base at any α.

Usage

With transformers (chat template included):

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("nurl-lang/Qwen3-4B-NURL")
model = AutoModelForCausalLM.from_pretrained("nurl-lang/Qwen3-4B-NURL")
msgs = [{"role": "user", "content": "Write a NURL function that adds two integers."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
print(tok.decode(model.generate(ids, max_new_tokens=80)[0]))

With nurllama (the engine it was trained in) — download the 132 MB adapters instead of the 16 GB merged model, and merge locally against the base GGUF (identical result, byte for byte):

nurllama pull hf.co/Qwen/Qwen3-4B-GGUF/Qwen3-4B-Q8_0.gguf --name qwen3-4b
curl -LO https://huggingface.co/nurl-lang/Qwen3-4B-NURL/resolve/main/nurl-lora-adapters.safetensors
nurllama finetune ~/.nurllama/blobs/<qwen3-4b-blob> /dev/null \
    --out nurl-lora-adapters.safetensors --merged qwen3-4b-nurl.st \
    --alpha 32 --stream --merge-only

nurllama run qwen3-4b "<prompt>" --weights qwen3-4b-nurl.st          # one-shot
nurllama serve --weights qwen3-4b-nurl.st                            # ollama-compatible API

Lineage

Base model: Qwen/Qwen3-4B (Apache-2.0). Tokenizer and configuration are the base model's, unchanged (tied embeddings; lm_head is the embedding table, as in the base).