pra-qwen3-8b-mlx-4bit

EInnovator/pra-qwen3-8b-mlx-4bit

pra-qwen3-8b-mlx-4bit is a machine learning model from EInnovator. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pra. The card lists the license as apache-2.0.

This repository packages the model-specific Progressive Retrieval Attention (PRA) structural mapping, runtime profiles, optional learned components, compatibility metadata, and measured qualification evidence. It does…

Downloads · 30 days

Access

Public

Updated Sep 3, 2026

Repo size

—

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.json280 KB · 87%

At a glance

Library: pra
License: apache-2.0
Access: Public
Created: Sep 2, 2026
Updated: Sep 3, 2026
SHA: af21d02c

Base models

mlx-community/Qwen3-8B-4bit

Datasets

Model card

From the Hugging Face model README

PRA Runtime Bundle for mlx-community/Qwen3-8B-4bit · MLX / 4bit

What this PRA Runtime Bundle is

Base model: mlx-community/Qwen3-8B-4bit
Immutable revision: 545dc4251c05440727734bcd94334791f6ab0192
Architecture: Qwen3ForCausalLM
Parameters: 8B
Tokenizer revision: 545dc4251c05440727734bcd94334791f6ab0192
Serving precision: INT4 / MLX-4bit
Post-training: pretrained and post-trained

Recommended configuration

Engine: mlx
Recommended PRA mode: Native Memory
Recommended profile: BALANCED
Bundle evidence tier: ENGINE_QUALIFIED
Native Memory status: QUALIFIED

Availability, qualification, and recommendation are separate. A mode may be implemented without being qualified or recommended for this identity.

Precision qualification

Precision evidence is scoped to the exact model conversion, engine, mode, and profile. Qualification does not transfer automatically between BF16, INT8, INT4, or encoding-specific formats.

Family	Encoding	Serving	Feature extraction	Adaptor parameters	Engine	Mode	Profile	Evidence	Datasets
INT4	MLX-4bit	INT4	NEEDS_RUN	NO_QUALIFIED_ADAPTER	mlx	Native Memory	BALANCED	ENGINE_QUALIFIED	NOT_MEASURED

Headline results

Workload	Selected Context quality	Native Memory quality	Delta NM vs SC	Visible-context delta NM vs SC	TTFT delta NM vs SC	Completion delta NM vs SC	Paired parity	Evidence
combined (n=15)	token_f1=0.2365	token_f1=0.2365	+0.0000	-89.1%	-0.6%	+3.0%	15/15	ENGINE_QUALIFIED

All headline rows freeze the PRA-selected evidence. Deltas are Native Memory minus Selected Context; negative latency and visible-context deltas are reductions. These rows contain no ordinary No-PRA arm.

Evidence receipt: mlx-lm 0.31.3; Apple M4 Pro (Mac16,7), 48 GB; selector-frozen natural QA (n=15); 2026-09-01; PRA commit 4b4486a66c80d09aa7982be29812d4027c57a4e3; artifact qualification/qwen3_8b_mlx_profiles.json; SHA-256 43a316c4ecca24420d5d5bc839d2b7a94d9a1074cbe5449092f23a6ba0acf47f.

Evidence by engine, mode, and profile

Each row identifies the exact runtime surface for which metrics are available. MEASURED counts scalar metrics with real observations; missing profile/mode combinations are not inferred from another row.

Engine	Mode	Profile	No PRA	Mode / no adaptor	Same mode / bundle	Measured metric groups
mlx	Native Memory	QUALITY	CALIBRATION_PENDING	Native Memory: CALIBRATION_PENDING	Native Memory + Bundle: CALIBRATION_PENDING	CALIBRATION_PENDING
mlx	Native Memory	BALANCED	NEEDS_RUN	Native Memory: MEASURED (16)	Native Memory + Bundle: NO_QUALIFIED_ADAPTER	context, quality, resources, serving
mlx	Native Memory	ECONOMY	CALIBRATION_PENDING	Native Memory: CALIBRATION_PENDING	Native Memory + Bundle: CALIBRATION_PENDING	CALIBRATION_PENDING

Canonical staged evidence

Each table holds task, hardware, engine, model, precision, and profile fixed. Every delta names its source and target; bundle use is held orthogonal to execution depth.

combined / mlx-lm / balanced

Exact identity: mlx-community/Qwen3-8B-4bit at 545dc4251c05440727734bcd94334791f6ab0192 on Apple M4 Pro (Mac16,7), 48 GB; precision INT4 / MLX-4bit.

Quality

Metric	Unit	Direction	Selected Context	Native Memory	Native Memory + Bundle	Delta NM vs SC	Delta Bundle vs NM
Token F1	fraction	higher_is_better	0.236455	0.236455	NO_QUALIFIED_ADAPTER	+0 (+0.00%)	NO_QUALIFIED_ADAPTER
Exact Match	fraction	higher_is_better	0	0	NO_QUALIFIED_ADAPTER	+0	NO_QUALIFIED_ADAPTER
Gold Answer Log Probability	log_probability	higher_is_better	-12.2676	-12.2676	NO_QUALIFIED_ADAPTER	+0 (-0.00%)	NO_QUALIFIED_ADAPTER

Context

Metric	Unit	Direction	Selected Context	Native Memory	Native Memory + Bundle	Delta NM vs SC	Delta Bundle vs NM
Visible Tokens	token	lower_is_better	315.533	34.2667	NO_QUALIFIED_ADAPTER	-281.267 (-89.14%)	NO_QUALIFIED_ADAPTER
Selected Native K/V Tokens	token	neutral	0	10125.6	NO_QUALIFIED_ADAPTER	+10125.6	NO_QUALIFIED_ADAPTER

Serving

Metric	Unit	Direction	Selected Context	Native Memory	Native Memory + Bundle	Delta NM vs SC	Delta Bundle vs NM
TTFT p50 (ms)	ms	lower_is_better	115.775	115.061	NO_QUALIFIED_ADAPTER	-0.713958 (-0.62%)	NO_QUALIFIED_ADAPTER
TTFT p95 (ms)	ms	lower_is_better	183.02	231.031	NO_QUALIFIED_ADAPTER	+48.0105 (+26.23%)	NO_QUALIFIED_ADAPTER
TTFT p99 (ms)	ms	lower_is_better	183.02	231.031	NO_QUALIFIED_ADAPTER	+48.0105 (+26.23%)	NO_QUALIFIED_ADAPTER
ITL p50 (ms)	ms	lower_is_better	18.787	19.4877	NO_QUALIFIED_ADAPTER	+0.700714 (+3.73%)	NO_QUALIFIED_ADAPTER
ITL p95 (ms)	ms	lower_is_better	18.9825	20.5185	NO_QUALIFIED_ADAPTER	+1.53595 (+8.09%)	NO_QUALIFIED_ADAPTER
ITL p99 (ms)	ms	lower_is_better	18.9825	20.5185	NO_QUALIFIED_ADAPTER	+1.53595 (+8.09%)	NO_QUALIFIED_ADAPTER
Output Tokens Per Second	output_token/s	higher_is_better	53.3045	51.1062	NO_QUALIFIED_ADAPTER	-2.19834 (-4.12%)	NO_QUALIFIED_ADAPTER
Completion Latency Mean (ms)	ms	lower_is_better	275.946	284.088	NO_QUALIFIED_ADAPTER	+8.14217 (+2.95%)	NO_QUALIFIED_ADAPTER

Resources

Metric	Unit	Direction	Selected Context	Native Memory	Native Memory + Bundle	Delta NM vs SC	Delta Bundle vs NM
Active Detail Bytes	byte	lower_is_better	0	4.14745e+07	NO_QUALIFIED_ADAPTER	+4.14745e+07	NO_QUALIFIED_ADAPTER
Retained Detail Bytes	byte	lower_is_better	0	4.14745e+07	NO_QUALIFIED_ADAPTER	+4.14745e+07	NO_QUALIFIED_ADAPTER
Peak Memory Bytes	byte	lower_is_better	5.18009e+09	5.13407e+09	NO_QUALIFIED_ADAPTER	-4.60226e+07 (-0.89%)	NO_QUALIFIED_ADAPTER

Installation

pip install 'pra-hf[hf-hub,hf-runtime]'
pra doctor

Quickstart

pra inspect mlx-community/Qwen3-8B-4bit -e mlx -a EInnovator/pra-qwen3-8b-mlx-4bit
pra evaluate mlx-community/Qwen3-8B-4bit -e mlx -D qasper -a EInnovator/pra-qwen3-8b-mlx-4bit
pra recommend .pra/runs/latest
pra serve mlx-community/Qwen3-8B-4bit -e mlx -a EInnovator/pra-qwen3-8b-mlx-4bit -p balanced

Profiles

Profile	Purpose	Routing	Consumer layers	Status	Recommendation
QUALITY	Candidate maximum-quality profile; held-out calibration is incomplete	generic cosine	all eligible	CALIBRATION_PENDING	Not promoted
BALANCED	Qualified default preserving the all-eligible consumer geometry	generic cosine	all eligible	QUALIFIED	Default
ECONOMY	Reduced-consumer candidate; the held-out quality gate has not passed	generic cosine	CALIBRATION_PENDING	CALIBRATION_PENDING	Not promoted

Engine compatibility

Engine	Selected Context	Native Memory	Native Serving	Recommended today
mlx	validated	QUALIFIED	NOT_APPLICABLE	Native Memory with BALANCED

End-to-end qualification

Workload	Mode	Quality	Visible tokens	TTFT p50	Completion mean	Hardware	Evidence
2wikimultihopqa (n=5)	Selected Context	token_f1=0.3578	351.2	114.2 ms	271.9 ms	Apple M4 Pro (Mac16,7), 48 GB	ENGINE_QUALIFIED
2wikimultihopqa (n=5)	Native Memory	token_f1=0.3578	31.6	113.8 ms	278.3 ms	Apple M4 Pro (Mac16,7), 48 GB	ENGINE_QUALIFIED
hotpotqa (n=5)	Selected Context	token_f1=0.3016	262	180.3 ms	311.6 ms	Apple M4 Pro (Mac16,7), 48 GB	ENGINE_QUALIFIED
hotpotqa (n=5)	Native Memory	token_f1=0.3016	43.4	179 ms	325 ms	Apple M4 Pro (Mac16,7), 48 GB	ENGINE_QUALIFIED
qasper (n=5)	Selected Context	token_f1=0.05	333.4	114.1 ms	244.4 ms	Apple M4 Pro (Mac16,7), 48 GB	ENGINE_QUALIFIED
qasper (n=5)	Native Memory	token_f1=0.05	27.8	112.6 ms	249 ms	Apple M4 Pro (Mac16,7), 48 GB	ENGINE_QUALIFIED
combined (n=15)	Selected Context	token_f1=0.2365	315.5	115.8 ms	275.9 ms	Apple M4 Pro (Mac16,7), 48 GB	ENGINE_QUALIFIED
combined (n=15)	Native Memory	token_f1=0.2365	34.27	115.1 ms	284.1 ms	Apple M4 Pro (Mac16,7), 48 GB	ENGINE_QUALIFIED

Native Memory qualification

Native Memory uses the same selector output as Selected Context. It is recommended only where the profile and engine tables say so.

Workload	Selected native K/V tokens	Active detail	Peak memory	Completion cost vs Selected Context
2wikimultihopqa	1.151e+04	44.94 MiB	4.77 GiB	1.023x
hotpotqa	7870	30.74 MiB	4.74 GiB	1.043x
qasper	1.1e+04	42.98 MiB	4.78 GiB	1.019x
combined	1.013e+04	39.55 MiB	4.78 GiB	1.03x

Research diagnostics

No separate routing diagnostic is packaged for this bundle.

These are qualification measurements, not guaranteed production performance. Run pra evaluate on your hardware and workload. Engine version, profile, cohort, evidence tier, date, and artifact provenance remain recorded in qualification/ and bundle.yaml.

How to evaluate locally

pra evaluate mlx-community/Qwen3-8B-4bit -e mlx -a EInnovator/pra-qwen3-8b-mlx-4bit -D qasper -o .pra/runs/qasper
pra recommend .pra/runs/qasper
pra report .pra/runs/qasper --format html

Known limitations

No learned router is bundled for this exact quantized identity; routing-adapter transfer from another quantization is intentionally disallowed.
Paired natural-QA evidence contains 5 examples per dataset and supports engine qualification, not production qualification.
Reduced consumer-layer configurations failed the held-out quality gate; BALANCED therefore retains all eligible layers.
The qualification identity is the exact 4bit MLX model and revision; it does not transfer automatically to another checkpoint, engine, or quantization.
The selector-frozen natural-QA run qualifies the generic Native Memory path; an exact learned-adaptor arm still requires a separate run.
Base-model and dataset licenses apply separately to the router artifact.

Training/creation

The structural adapter is training-free. Learned-component training metadata is stored beside each component and summarized in bundle.yaml.

Reproducibility

PRA commit: 81f42d69936bf50eb6fe11a0f7477b415bbf250d
Bundle build commit: 81f42d69936bf50eb6fe11a0f7477b415bbf250d
Bundle schema: 2
PRA package: 0.2.0rc1
Component fingerprints and file checksums are recorded in bundle.yaml.

Community/support

Library: pra
License: apache-2.0
Created: Sep 2, 2026
Updated: Sep 3, 2026