
Ornith-1.0-35B GPTQ-Pro FOEM 4-bit g128 ns256
<p style="margin: 10px 0 6px 0; color: #475569; font-size: 14px; line-height: 1.5;">
These models are built and maintained on rented GPU compute. If you want to show some appreciation, a follow on X or a coffee helps keep the releases coming.
</p>
<div style="display: flex; flex-wrap: wrap; align-items: center; gap: 10px; margin: 8px 0 18px 0;">
<a href="https://x.com/xreyrobert" target="_blank" rel="noopener noreferrer" class="follow-link" style="display: inline-flex; align-items: center; justify-content: center; gap: 8px; min-height: 36px; box-sizing: border-box; padding: 9px 14px; border: 1px solid #111827; border-radius: 999px; background: #111827; color: #ffffff; text-decoration: none; font-weight: 700; font-size: 14px; line-height: 1;">
<svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true" style="width: 16px; height: 16px; flex: 0 0 auto;"><path d="M18.244 2.25h3.308l-7.227 8.26 8.502 11.24H16.17l-5.214-6.817L4.99 21.75H1.68l7.73-8.835L1.254 2.25H8.08l4.713 6.231zm-1.161 17.52h1.833L7.084 4.126H5.117z"></path></svg>
Follow @xreyrobert
</a>
<a href="https://donate.stripe.com/bJe5kDbkwaYn3Jq9uL18c00" target="_blank" rel="noopener noreferrer" class="support-link" style="display: inline-flex; align-items: center; justify-content: center; gap: 8px; min-height: 36px; box-sizing: border-box; padding: 9px 14px; border: 1px solid #f59e0b; border-radius: 999px; background: #fbbf24; color: #111827; text-decoration: none; font-weight: 700; font-size: 14px; line-height: 1;">
Support this work
</a>
</div>
This is a GPTQ-Pro 4-bit quantization of
deepreinforce-ai/Ornith-1.0-35B.
It is a deployment artifact, not a new fine-tune. The goal is to make the
Ornith 35B MoE checkpoint practical for GPTQ-compatible local and homelab
serving while preserving the coding-agent oriented behavior of the source
model.
The headline validation target for this release is text-generation and
agentic coding workload serving with vLLM + GPTQ-Marlin. Vision paths and
speculative decoding are not positioned as validated features for this release.
Source And Credits
Source model:
Quantization tooling and reference recipe:
Thanks to deepreinforce-ai for the Ornith release, to modelcloud for GPTQModel,
and to groxaxo for GPTQ-Pro and the Qwen3.6 GPTQ-Pro recipe family this run was
aligned with.
Artifact Summary
| Field | Value |
|---|
| Source model | deepreinforce-ai/Ornith-1.0-35B |
| Architecture | Qwen3_5MoeForConditionalGeneration |
| Model type | qwen3_5_moe |
| Hidden layers | 40 |
| Hidden size | 2048 |
| Experts | 256 |
| Experts per token | 8 |
| Max position embeddings | 262144 |
| Tensor files | 5 |
| Indexed tensors | 124576 |
| Indexed total size | 20,652,683,992 bytes |
mtp.* tensors in index | 0 |
| Tokenizer class | Qwen2TokenizerFast |
The source config contains mtp_num_hidden_layers=1, but the uploaded weight
index contains no mtp.* tensors. Treat this checkpoint as non-MTP for
speculative decoding unless a follow-up artifact restores and validates real
MTP tensors.
Vision configuration is present from the source architecture, but multimodal
serving has not been validated for this quantized release. The intended use
case is text generation and coding-agent evaluation.
Quantization Recipe
| Setting | Value |
|---|
| Method | GPTQ-Pro / GPTQModel |
| Quantizer | gptqmodel:6.1.0-dev |
| Bits | 4 |
| Group size | 128 |
| Symmetric quantization | true |
| Desc act | false |
| True sequential | true |
| Checkpoint format | gptq |
| Pack dtype | int32 |
| Pack implementation | cpu |
| MSE | 2.0 |
| Damp percent | 0.05 |
| Damp auto increment | 0.01 |
| FOEM alpha | 0.25 |
| FOEM beta | 0.2 |
| Calibration samples | 256 |
| Calibration sequence length | 2048 |
| Calibration mix | code-oriented local JSONL mix |
| MoE routing during quantization | ExpertsRoutingBypass |
| MoE routing batch size | 64 |
Excluded from quantization by dynamic rules:
- embeddings
lm_head
- MTP modules
- norms
- vision / visual modules
The calibration data itself is not included in this model repository.
Intended Serving Shape
This checkpoint is intended for advanced users testing text-only vLLM or
GPTQ-compatible serving for Qwen/Ornith MoE checkpoints.
A starting vLLM shape for long-context text serving:
vllm serve XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256 \
--served-model-name ornith-1.0-35b-gptq-pro-foem-4bit-g128-ns256-ctx262k \
--language-model-only \
--dtype float16 \
--quantization gptq_marlin \
--tensor-parallel-size 1 \
--max-model-len 262144 \
--max-num-seqs 1 \
--kv-cache-dtype fp8_e5m2 \
--reasoning-parser qwen3 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_xml \
--enable-prefix-caching \
--gpu-memory-utilization 0.95 \
--trust-remote-code
Serving context for the published Smoke24/vLLM measurements:
The Smoke24/vLLM numbers were collected on an internal llm-residency vLLM
deployment. The custom image recipe is not published yet, so this card does not
present that image as a public reproduction target. The stable serving knobs
captured from the run are listed for context.
| Field | Value |
|---|
| Nomad job profile | vllm-ornith-35b-gptq-pro-262k |
| Served model name | ornith-1.0-35b-gptq-pro-foem-4bit-g128-ns256-ctx262k |
| Critical flags | --dtype float16, --quantization gptq_marlin, --kv-cache-dtype fp8_e5m2, --reasoning-parser qwen3, --tool-call-parser qwen3_xml, --max-model-len 262144, --max-num-batched-tokens 2096 |
Treat this as a serving starting point, not a compatibility guarantee for every
vLLM release. GPTQ-Marlin, Qwen3.5 MoE handling, FP8 KV cache, and tool parser
behavior are loader-version sensitive.
Public vLLM Reproducibility
This artifact has a public reproducibility path on the unmodified upstream vLLM OpenAI image:
- image:
docker.io/vllm/vllm-openai:nightly-7a1eb8ac2ec4ea69338c51dc7afd4b15010abfa8
- vLLM version observed in validation:
0.20.1rc1.dev16+g7a1eb8ac2
- GPU class: single RTX 3090 24 GB / Ampere
--enforce-eager was not used
- no local sleep/wake patch or
localhost/*sleepwake* image is required for the validation below
Validated serving shape:
- context:
--max-model-len 262144 with --gpu-memory-utilization 0.96
--language-model-only, --dtype float16, --quantization gptq_marlin
--kv-cache-dtype fp8_e5m2, --enable-prefix-caching, --max-num-seqs 1
--max-num-batched-tokens 2096, --max-cudagraph-capture-size 32
--reasoning-parser qwen3, --tool-call-parser qwen3_xml
The 262k profile is tight on 24 GB GPUs; gpu_memory_utilization=0.95 was short on KV cache in validation, while 0.96 passed.
vLLM RTX 3090 / Ampere Workload Snapshot
The image above reports observed vLLM / Prometheus metrics for the served model
label ornith-1.0-35b-gptq-pro-foem-4bit-g128-ns256-ctx262k over the recent
workload window that included validation traffic:
| Metric | Value |
|---|
| Context budget | 262k |
| Decode-only throughput from TPOT | ~151 tok/s |
| Prefill throughput | ~4,441 tok/s |
| Prefix cache hit ratio | 89.5% |
| Average TTFT | 0.63s |
| Average E2E latency | 10.20s |
These are serving metrics, not a standalone quality benchmark. They describe
the observed vLLM GPTQ-Marlin runtime behavior for this deployment shape.
MMLU-Pro 350 Selected Subset
<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; border: 1px solid #cbd5e1; border-radius: 16px; box-shadow: 0 10px 15px -3px rgba(0, 0, 0, 0.05), 0 4px 6px -2px rgba(0, 0, 0, 0.05); overflow: hidden; background: #ffffff; margin-bottom: 30px;">
<div style="background: linear-gradient(135deg, #7c3aed 0%, #4f46e5 100%); padding: 20px; color: white;">
<h3 style="margin: 0; font-size: 20px; font-weight: 700; display: flex; align-items: center; gap: 8px; color: white; border: none;">📊 Evaluation & Performance Metrics</h3>
<p style="margin: 4px 0 0 0; font-size: 13px; color: #ddd6fe;">2 July update: single-pass MMLU-Pro selected subset quality check for the XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256 deployment, using the same 350-question subset and evaluation harness used for the XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1 selected-subset comparison.</p>
</div>
<div style="padding: 24px; display: flex; flex-direction: column; gap: 24px;">
<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(180px, 1fr)); gap: 16px;">
<div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);">
<span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">📚 MMLU-Pro Subset</span>
<span style="font-size: 24px; font-weight: 800; color: #1e293b; display: block;">90.86%</span>
<span style="font-size: 11px; color: #64748b; font-weight: 500;">318 / 350 single-pass run</span>
</div>
<div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);">
<span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">🧠 vs XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1</span>
<span style="font-size: 24px; font-weight: 800; color: #10b981; display: block;">+0.86 pp</span>
<span style="font-size: 11px; color: #64748b; font-weight: 500;">315 / 350 selected-subset run</span>
</div>
<div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);">
<span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">⚖️ vs Qwopus CSV</span>
<span style="font-size: 24px; font-weight: 800; color: #10b981; display: block;">+3.43 pp</span>
<span style="font-size: 11px; color: #64748b; font-weight: 500;">306 / 350 reference CSV</span>
</div>
<div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);">
<span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">⚡ vLLM Decode</span>
<span style="font-size: 24px; font-weight: 800; color: #1e293b; display: block;">150.68</span>
<span style="font-size: 11px; color: #64748b; font-weight: 500;">completion tok/s, request wall-time</span>
</div>
</div>
<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; box-shadow: 0 2px 4px rgba(0,0,0,0.02);">
<div style="background: linear-gradient(135deg, #f8fafc 0%, #f1f5f9 100%); padding: 12px 16px; border-bottom: 1px solid #cbd5e1; font-weight: 700; font-size: 14px; color: #1e293b; display: flex; align-items: center; gap: 8px;">
<span>📚</span> MMLU-Pro Selected Subset - 2 July update
</div>
<div style="padding: 16px; display: flex; flex-direction: column; gap: 16px;">
<div style="background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 8px; padding: 12px 14px; font-size: 13px; color: #334155; line-height: 1.6;">
<b>Evaluation format:</b> This uses the same 350-question MMLU-Pro subset published in the <code>test_data</code> directory of <code>Jackrong/Qwopus3.6-27B-v2</code>: 7 categories, 50 questions per category. This is not a full MMLU-Pro leaderboard run.
</div>
<div style="background: #eff6ff; border: 1px solid #bfdbfe; border-radius: 8px; padding: 12px 14px; font-size: 13px; color: #1e3a8a; line-height: 1.6;">
<b>Protocol note:</b> The headline <b>318 / 350 = 90.86%</b> score is a single-pass OpenAI-compatible run through the published vLLM/LiteLLM serving path. The run used <code>temperature=1.0</code>, <code>top_p=0.95</code>, <code>max_tokens=0</code> (no explicit output cap sent by the runner), <code>request_timeout=3600</code>, and <code>retries=1</code>. The prompt follows the MMLU-Pro 5-shot category-prefix style using validation split <code>cot_content</code> and the system instruction asking for <code>The answer is ...</code>. All 350 requests completed with <code>finish_reason=stop</code>; one response had no parsed answer and is counted as wrong. There were no API errors.
</div>
<div style="overflow-x: auto; width: 100%;">
<table style="display: table; width: 100%; min-width: 100%; table-layout: fixed; border-collapse: collapse; font-size: 13px;">
<thead>
<tr style="background: rgba(124, 58, 237, 0.05);">
<th style="width: 42%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: left; color: #7c3aed; font-weight: bold;">Model / run</th>
<th style="width: 18%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Correct / Total</th>
<th style="width: 16%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Accuracy</th>
<th style="width: 24%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Notes</th>
</tr>
</thead>
<tbody>
<tr style="background: rgba(16, 185, 129, 0.06);">
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 700; color: #047857;">XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 700; color: #047857;">318 / 350</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 800; color: #10b981; font-size: 14px;">90.86%</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 700; color: #047857;">single-pass unrestricted</td>
</tr>
<tr>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1 selected-subset run</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">315 / 350</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: bold;">90.00%</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; color: #10b981; font-weight: 700;">Ornith +0.86 pp</td>
</tr>
<tr>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Qwopus3.6-27B-v2 reference CSV</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">306 / 350</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: bold;">87.43%</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; color: #10b981; font-weight: 700;">Ornith +3.43 pp</td>
</tr>
<tr>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Qwen3.6-27B-v2 reference CSV</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">297 / 350</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: bold;">84.86%</td>
<td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; color: #10b981; font-weight: 700;">Ornith +6.00 pp</td>
</tr>
</tbody>
</table>
</div>
<div style="overflow-x: auto; width: 100%;">
<table style="display: table; width: 100%; min-width: 100%; table-layout: fixed; border-collapse: collapse; font-size: 12px;">
<thead>
<tr style="background: rgba(124, 58, 237, 0.03);">
<th style="width: 34%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: left;">Category</th>
<th style="width: 22%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #047857;">XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256</th>
<th style="width: 22%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #7c3aed;">XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1</th>
<th style="width: 22%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">Delta</th>
</tr>
</thead>
<tbody>
<tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Biology</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #047857;">48 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">48 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">0</td></tr>
<tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Business</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #10b981;">47 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">45 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #10b981; font-weight: 700;">+2</td></tr>
<tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Chemistry</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #dc2626;">44 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">45 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #dc2626; font-weight: 700;">-1</td></tr>
<tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Computer Science</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #047857;">40 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">40 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">0</td></tr>
<tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Health</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #047857;">42 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">42 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">0</td></tr>
<tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Mathematics</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #10b981;">49 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">48 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #10b981; font-weight: 700;">+1</td></tr>
<tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Physics</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #10b981;">48 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">47 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #10b981; font-weight: 700;">+1</td></tr>
</tbody>
</table>
</div>
<div style="overflow-x: auto; width: 100%;">
<table style="width: 100%; border-collapse: collapse; font-size: 13px; min-width: 500px;">
<thead>
<tr style="background: rgba(124, 58, 237, 0.05);">
<th style="padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: left; color: #7c3aed; font-weight: bold;">Runtime metric</th>
<th style="padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Observed value</th>
</tr>
</thead>
<tbody>
<tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Prompt tokens</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">509,541</td></tr>
<tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Completion tokens</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">860,087</td></tr>
<tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Request elapsed sum</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">5,708.0s</td></tr>
<tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Completion throughput</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 700; color: #7c3aed;">150.68 tok/s</td></tr>
</tbody>
</table>
</div>
<p style="margin: 0; font-size: 13px; color: #475569; line-height: 1.6;">
<b>Summary:</b> On the selected 350-question MMLU-Pro evaluation set, `XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256` reached <b>90.86%</b> accuracy in a single unrestricted pass. On the same 350-question selected subset, the prior `XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1` unrestricted reference run reached <b>90.00%</b>. Treat the difference as a small-sample local validation signal rather than a leaderboard claim.
</p>
<div style="display: inline-flex; align-items: center; width: fit-content; background: #ffffff; border: 1px solid #e2e8f0; border-radius: 999px; padding: 6px 10px; font-size: 11px; color: #64748b; font-weight: 600; box-shadow: 0 1px 2px rgba(15,23,42,0.04);">
Scope note: this is not a full MMLU-Pro evaluation or official leaderboard submission. It is a selected-subset regression/quality check over the public 350-question selected subset used on the Qwopus card.
</div>
</div>
</div>
</div>
</div>
Terminal-Bench 2.0 Smoke24
Terminal-Bench 2.0 Smoke24 is a fixed 24-task coding-agent comparison corpus.
It is useful for fast regression and local serving comparison, but it is not a
full Terminal-Bench leaderboard submission.
XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256 used the long-context card-validation shape:
max_model_len=262144, max_input_tokens=220000, 30 minute task timeout,
32 CPU / 48 GiB sandbox, thinking_token_budget=32768, max_output_tokens=40000,
temperature 1.0, top-p 0.95, top-k 20, and preserve_thinking=true.
| Run | Score | Success rate | Wall-time | Output tokens | Observed decode | LLM API time |
|---|
ornith-1.0-35b-gptq-pro-foem-4bit-g128-ns256-ctx262k | 17/24 | 70.8% | 186.4m | 498.2k | 137.6 tok/s | 60.3m |
Smoke24 task list and harness summary:
Validation Status
Completed:
- Source model loaded for quantization.
- Code-oriented calibration mix generated.
- GPTQ-Pro FOEM quantization completed.
- Final checkpoint saved as five safetensors shards.
- Hugging Face repo file list verified after upload.
- vLLM serving metrics collected through Prometheus.
- Terminal-Bench 2.0 Smoke24 validation run completed.
Not yet validated:
- Full Terminal-Bench leaderboard submission.
- Vision or multimodal serving.
- MTP/speculative decoding.
- Broad multi-GPU serving matrix.
Limitations
- Experimental quantization.
- Smoke24 is a small 24-task slice; read one-task differences with caution.
- The calibration mix is code-heavy and was chosen for coding-agent use, not as
a general-purpose calibration corpus.
- Treat this release as non-MTP for speculative decoding.
- Vision support is not validated.
- Loader behavior may vary across vLLM, Transformers, GPTQModel, and
GPTQ-Marlin versions.
Files
Key files:
model.safetensors.index.json
model-00001-of-00005.safetensors through model-00005-of-00005.safetensors
config.json
generation_config.json
processor_config.json
quantize_config.json
tokenizer.json
tokenizer_config.json
chat_template.jinja
References
Individual Project Notice
This repository is an individual research project. It is not affiliated with,
sponsored by, or endorsed by any employer or organization.