pseng-14b-preview

PSEng-14B-preview

A LoRA adapter that teaches Phi-4 to write PowerShell conforming to the PowerShell Engineer Standard without the Standard in the context window. Trained locally on Apple Silicon with MLX. No hosted API touched the data at any stage.

This is a preview from an in-progress run, not a finished model. Read the limitations before quoting anything from it.

⚠️ This is an MLX adapter, not a PEFT adapter. PeftModel.from_pretrained() will not load it. The adapter_config.json here uses MLX's schema (lora_parameters, num_layers, keys), not PEFT's (peft_type, r, target_modules). Use mlx-lm, as shown below. Apple Silicon required.

Use

pip install mlx-lm

The adapter was trained against a 4-bit quantized Phi-4. Build that exact base first , it is a local conversion, not a Hub repo:

python -m mlx_lm convert \
  --hf-path mlx-community/phi-4-bf16 \
  -q --q-bits 4 --q-group-size 64 \
  --mlx-path phi-4-4bit

Then generate:

python -m mlx_lm generate \
  --model phi-4-4bit \
  --adapter-path <path to this adapter> \
  --max-tokens 1600 --temp 0.0 \
  --prompt "Write a function that disables Active Directory accounts inactive for more than a given number of days."

Standard-conforming functions are long: comment-based help with three examples, validated parameters, ShouldProcess where state changes. Give it 1,600 tokens. Greedy decoding (--temp 0.0) is what it was evaluated under.

Applying the adapter to a differently-quantized base may not reproduce the results below.

What it scores, and what that does and does not mean

Measured on a 48-task development split. This is not the project's frozen benchmark, which is reserved for a pre-registered evaluation this model has not yet undergone.

Scores are 0–100 under a static rubric: PowerShell's own parser, an AST walk for Standard structure, PSScriptAnalyzer under pinned settings, and a validator that rejects invented cmdlets and parameters. Any parse failure or invented surface scores the task zero.

Putting the entire Standard in Phi-4's context bought about one point. Training on it bought forty-two. That gap is a caution about prompt-based delivery generally, not a claim that this model is good.

These numbers are not comparable to the PSEng-8B flagship's published benchmark scores. Different task set, different base model. A like-for-like number will exist only after the pre-registered evaluation.

Limitations

• Preview. Trained on 517 pairs, stopped at iteration 500 of a planned 1,200 on a shallow, noisy validation minimum. A fuller corpus and a proper hyperparameter sweep are the next steps and are expected to supersede this artifact.
• Invented surface is the dominant failure. 17 of 48 dev tasks scored zero, 12 of them for cmdlets or parameters that do not exist. On tasks that clear the gates it averages 91.8, so the distribution is bimodal. Verify any command it emits against Get-Help before running it.
• Conformance is not correctness. Nothing in the evaluation executes the generated code. It measures whether the code follows the Standard, not whether it does the right thing.
• It will not beat a frontier model. It is small, private, offline, free, and good at one language.
• Windows-only surface (Active Directory, Exchange) is validated against captured manifests rather than a live system.

Training

• Base: Phi-4 (14B, MIT), mlx-community/phi-4-bf16 revision e9ebdf8d, converted locally to 4-bit (group size 64, affine).
• Method: QLoRA, rank 32, 20 of 40 layers, targeting Phi's fused self_attn.qkv_proj and mlp.gate_up_proj plus self_attn.o_proj and mlp.down_proj. lr 1e-5, batch 1, sequence 3,072, completion-only loss, seed 20260805.
• Corpus: 517 train / 43 valid pairs, every accepted pair scoring 100/100 under the frozen rubric. Teacher: Qwen3-Coder-30B-A3B-Instruct, run locally. Task prompts carry no Standard text , the model must have internalised it.
• Data hygiene: development and benchmark tasks are excluded from training by fatal invariant checks; completions that merely echo the Standard's own reference implementation are filtered out.

Licence and attribution

MIT, inherited from Phi-4 (Copyright (c) Microsoft Corporation). Its notice travels with this adapter in NOTICE.

What is in this download: weight deltas trained on top of Phi-4, and nothing else. No third-party model weights are included, and no model other than Phi-4 is needed to run it.

The training corpus was generated during dataset construction by Qwen3-Coder-30B-A3B-Instruct (Apache 2.0), running locally. That model is not in the inference path and none of its weights are distributed here. Task material derived from MicrosoftDocs repositories is CC BY 4.0. Full attribution for all three is in NOTICE.

PowerShell is a trademark of Microsoft Corporation. This project is not affiliated with, endorsed by, or sponsored by Microsoft.