microsoft_phi-4-ingest-best-gptq

Positron AI Quantized Build

This repository contains a Positron AI quantized build of microsoft/phi-4 for tron inference on Positron FPGA-serving infrastructure.

Recommended Use

Use this artifact when you need a GPTQ 4-bit build of microsoft/phi-4 optimized for Positron's tron runtime. This build targets Positron's ingest-serving path, where runtime fidelity and FPGA deployability are prioritized over general-purpose GPU portability.

For general-purpose GPU inference, compare against the original model and other quantized formats before deployment.

Artifact Summary

Quantization Details

Validation Results

KL-divergence measures token-distribution drift between this quantized artifact and the BF16 reference model; lower values indicate closer agreement. It was computed on Positron's tron FPGA-serving path, so treat it as a runtime-specific drift measurement rather than a GPU benchmark.

Evaluation Methodology

Known Limitations

• KL-divergence was measured on a 12-prompt Positron validation suite; treat it as a runtime validation signal, not a broad benchmark.
• Results are specific to the Positron tron FPGA-serving path and may differ from GPU-native inference.
• MMLU evaluation is pending; results will be added when available.

Provenance

This artifact was produced by Positron AI from microsoft/phi-4. The original model license and usage restrictions continue to apply.