nl2sh-qwen25-coder-1.5b-GGUF

nl2sh-qwen25-coder-1.5b

Natural-language → shell command generator, drop-in alternative to whatisit-nl2sh's nl2sh-1.5b (same base model family: Qwen2.5-Coder-1.5B-Instruct), trained on a broader, hardened data mix.

Measured on InterCode-ALFA (300 tasks, temp 0, max_tokens 64, unmodified upstream scorer with embedding heuristic at threshold 0.75):

Same-size Q4_K_M beats the upstream default by +3.7 points over its published score and +6.7 points measured head-to-head on identical hardware/protocol. (Quant-to-quant differences of 1–2 pts are within the ±5.5 pt binomial noise at n=300 — treat all variants as one ~0.65 blob; Q4_K_M is recommended as it holds the top as-measured score at the smallest size.)

Files

Perplexity ladder (held-out NL2SH pairs): f16 4.649 · Q8 4.655 · Q6 4.665 · Q4+imx 4.720 · Q4 4.789.

Robustness

Trained with organic chatter pairs so arbitrary input always yields a sane, harmless command instead of garbage:

> hello          → echo hello
> thanks         → echo "you're welcome!"
> who am i       → whoami
> qwerty         → pwd
> what time is it→ date

Training setup

Final train loss 0.904 avg, token accuracy 82.7%.

Note for RDNA4/ROCm trainers: requires PYTORCH_HIP_ALLOC_CONF=roundup_power2_divisions:16 and hipBLASLt disabled (TORCH_BLAS_PREFER_HIPBLASLT=0 ROCBLAS_USE_HIPBLASLT=0 DISABLE_ADDMM_HIP_LT=1) to avoid the gfx1201 page-fault bug (ROCm#6600).

Usage

llama-server -m qcoder-nl2sh-q4_k_m.gguf --temp 0

whatisit setup --model ./qcoder-nl2sh-q4_k_m.gguf
whatisit doctor

Greedy decoding, output ≤64 tokens — answers with a single command. Non-command input still produces a harmless command (see robustness above).

Limitations

• Single-turn, no shell state; English only.
• One command per request, not scripts.
• Quant-to-quant benchmark differences are within measurement noise.