LFM2.5-1.2B-Thinking-Q4_K_M-BE

LFM2.5-1.2B-Thinking-Q4_K_M-BE

LiquidAI's LFM2.5-1.2B-Thinking model quantized to Q4_K_M and converted to big-endian (BE) for IBM AIX POWER9.

Model Details

• Base Model: LFM2.5-1.2B-Thinking
• Quantization: Q4_K_M (4-bit)
• Size: 731 MB
• Endianness: Big-endian (for AIX/POWER9)
• Architecture: lfm2 hybrid (10 shortconv + 6 GQA attention layers)
• Context Window: 128,000 tokens

Performance (AIX POWER9)

• Generation: 19.25 tok/s (16 threads, SMT-2)
• Hardware: IBM POWER9 @ 2.75 GHz
• OS: AIX 7.3 TL4
• Compiler: GCC 13.3.0

Usage

./build/bin/llama-simple -t 16 -n 256 \
  -m models/LFM2.5-1.2B-Thinking-Q4_K_M-be.gguf \
  -p "What is 3+3? Think step by step."

Characteristics

• Thinking Traces: Generates verbose reasoning output
• Use Case: Complex reasoning, analysis, problem-solving
• Trade-off: Slower than Instruct (19.25 vs 26.9 tok/s) due to thinking content
• Quality: Better reasoning quality through explicit thinking process

Testing Results

Tested on AIX 7.3 TL4 with:

• GCC 13.3.0
• 16 threads, SMT-2 mode
• Prompt: "What is 3+3? Think step by step."
• Output: Complete thinking traces + final answer

References

• LibrePower: https://librepower.org
• llama-aix Project: https://gitlab.com/librepower/llama-aix
• Original Model: https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking-GGUF
• Quantization Tool: https://github.com/ggerganov/llama.cpp

License

Same as original LFM2.5-1.2B-Thinking model.