Downloads · 30 days
13
72% of all-time downloads
mlboydaisuke/Phi-4-mini-instruct-ExecuTorch
Phi-4-mini-instruct-ExecuTorch is a text generation model from mlboydaisuke. Use it when you need the model to write or continue text. The card lists the license as mit.
Downloads · 30 days
13
72% of all-time downloads
All-time downloads
18
Public
Repo size
2.6 GB
Likes
0
Public
Click a slice to open those files.
.pte2.6 GB · 100%
From the Hugging Face model README
phi_4_mini_xnnpack_8da4w_e8.pte (2597.9 MB)
embedding_quantize: "8,0")export_llm, static shape (seq_len=1), max_seq_length 2048,
XNNPACK extended_opsllm_params/phi_4_mini_xnnpack_8da4w_e8.yamlllm_params/gen_static.py, token-by-token prefill then greedy decode, a fresh process per
prompt so no answer is read through the previous one's cache:
| prompt | answer | decode |
|---|---|---|
| capital of France? | "The capital of France is Paris." then keeps going | 49.3 tok/s |
| 17 times 4? | "17 times 4 is equal to 68." and stops | 49.7 tok/s |
Stopping is the part worth looking at. Chat template is <|user|>...<|end|><|assistant|>,
bos 199999, eos [200020, 199999] — the model's own
generation_config.json. The upstream example config
(examples/models/phi_4_mini/config/phi_4_mini_xnnpack.yaml) carries 151643 for both, which
is Qwen's, and a model that never sees its end token does not stop.
Not measured on a phone. Worth noting against the rest of the shelf: this decodes at 49
tok/s where Qwen3.5-4B manages
8.6 at a comparable 2.8 GB. The Qwen3.5 exports run with use_sdpa_with_kv_cache: False
and this one with it on.
python convert/export_phi_4_mini.py
Not export_llm directly, because its download path cannot reach this model:
load_checkpoint_from_pytorch_model reads pytorch_model.bin and this repository ships
safetensors only, so the code falls back to torchtune's checkpointer — a dependency with
its own torch pin, for a key rename and two splits. The script reads the safetensors, calls
upstream's own phi_4_hf_to_meta, and ties the output embedding back on: input and output
embeddings are tied here, so there is no lm_head.weight to map to output.weight, and the
model refuses a checkpoint without it. The torchtune converter beside it ties them; the
safetensors one does not.
python llm_params/gen_static.py \
--pte phi_4_mini_xnnpack_8da4w_e8.pte \
--tokenizer tokenizer.json \
--prompt '<|user|>What is the capital of France?<|end|><|assistant|>' \
--eos_ids "[200020,199999]"
The 8-bit embedding needs from executorch.kernels import quantized before the program is
loaded. Without it the method will not even load — kernel 'quantized_decomposed::embedding_byte.dtype_out' not found — which reads like a broken
export rather than a runtime missing its kernels.
(conversion scripts: executorch-models · iOS sample: executorch-samples)