Sonoma-1.2B-0901

Sonoma-1.2B

Sonoma-1.2B is the general-availability continuation of Sonoma-1.2B Preview, derived from LiquidAI/LFM2.5-1.2B-Thinking. It was post-trained on an Apple M5 Mac with assistant-only rank-32 LoRA across all 16 model blocks, then fused into an MLX model.

Evaluation

On this fixed local held-out split, the GA continuation reduces loss by 24.3% and perplexity by 20.3% versus the Preview checkpoint. This is a narrow post-training evaluation, not a general benchmark or a comparison with other models.

Deterministic behavior smoke test

The same eight-task, temperature-0 test was run with seed 20260901 and a 384-token cap on both fused MLX checkpoints:

This small regression suite is a directional signal only. Full per-task responses are in eval/ga-behavior-0901.json and eval/preview-behavior-0901.json.

Q4_K_M throughput

llama-bench build 8210 (2cd20b72e) on Apple M5, Metal, 99 GPU layers, three repetitions, 512 prompt tokens, and 128 generated tokens measured 596.8 prompt tokens/sec and 53.0 generated tokens/sec on the Q4_K_M file. The raw result is in eval/ga-q4-llama-bench.json; throughput varies with context length, build, and hardware.

Training data

The reproducible selected mixture contains 7,998 deduplicated records, with assistant-token masking and a 1,024-token training limit. It draws evenly from Fable 5 Premium, OpenThoughts-114k, Glaive Code Assistant v3, and Dojo Synthetic SFT. Review each source dataset's license and terms before redistribution or use.

Files

• model.safetensors: fused MLX weights
• adapter/: the final rank-32 LoRA adapter
• gguf/Sonoma-1.2B-GA-Q4_K_M.gguf: Q4_K_M llama.cpp export
• eval/ga-eval.log: independent final held-out evaluation
• eval/train-ga.log: training and final test log
• SHA256SUMS: SHA-256 hashes of the release weights
• training/: selected-data manifest and training/evaluation configs
• src/ and scripts/: the small reproducibility and benchmark utilities

The root weights use MLX's fused LFM2 convolution layout and are intended for MLX. For llama.cpp, use gguf/Sonoma-1.2B-GA-Q4_K_M.gguf from this bundle or the separate Mincofficial/Sonoma-1.2B-0901-GGUF Q4_K_M release.

License

Sonoma-1.2B is a derivative of LFM2.5-1.2B-Thinking and remains subject to the LFM Open License v1.0. Review the license before use or redistribution.