pyrrho-v2-nano-g1

Pyrrho v2 nano g1

Pyrrho is a CPU-runnable co-processor for retrieval-augmented generation. It reads a question before retrieval to suggest what evidence to seek, then reads the question with retrieved passages to assess whether those passages support an answer. A surrounding RAG runtime decides whether to answer, retrieve again, or surface a conflict. The broader project is described in the Pyrrho repository.

What this checkpoint does

The evidence verdict is SUFFICIENT, DISPUTED, or INSUFFICIENT. These are predictions about the supplied passages. The model does not retrieve sources, generate answers or citations, check external facts, or prove that a corpus has been searched completely.

Input format

The two passes use the same encoder with different input prefixes:

[PYRRHO_PRE]
Question: <user question>

[PYRRHO_POST]
Question: <user question>

Sources:
[1] <retrieved passage>
[2] <retrieved passage>

Inputs are truncated at 2,048 tokens by the published inference setup. For the PRE pass, use only the retrieval-intent and evidence-kind heads. For the POST pass, use all four heads.

Output contract

The checkpoint produces 18 logits in four heads, not one 18-class distribution. Labels below are in logit order.

A generic Hugging Face text-classification pipeline does not decode these heads correctly. A downstream runtime must decode each slice and apply its own thresholds and decision policy. Softmax and sigmoid scores are not calibrated assurances that an answer is safe.

Running locally

The repository contains a Transformers checkpoint (model.safetensors), an FP32 ONNX export (model.onnx), and a dynamic INT8 ONNX export (model_quantized.onnx). All can run on CPU; a GPU or hosted LLM is not required for inference. The ONNX exports use the same 18-logit contract. The INT8 export has not been qualified as an interchangeable evidence gate.

An integration should choose its own thresholds and test the resulting decisions on the evidence it expects to encounter.

Evaluation

The tables below reproduce the reported measurements for this checkpoint. They describe the original development run and Fitz-Sage integration checks; their limitations are explained after the tables. The development run was modernbert_base_v2_dual_from_g1_41358_active_20260704_seed42.

Post-retrieval development split

Pre-retrieval development split

Fitz-Sage integration checks

How to read these results

The 41,358-row training snapshot includes a deterministic 5,000-row tranche with only 32 unique post-retrieval model inputs. The original split used row IDs, so identical inputs appeared in both training and evaluation. The fixed-evidence suite also shared wording with that tranche. The development and fixed-suite scores therefore reproduce the original experiment, but are not independent estimates of generalization. The live benchmark is an integration check, not a substitute for a separate test set.

Later calibration diagnostic

A separate 2,500-example synthetic calibration comparison used a 0.34 runtime threshold for this published checkpoint:

This calibration set is also synthetic and does not establish performance on real retrieval traffic.

Training and limitations

The model judges only the evidence it receives. Synthetic examples and benchmark-like inputs do not establish behavior on live retrieval traffic, other languages, unseen document formats, or high-stakes decisions. Use independent evaluation before relying on its verdict to stop retrieval.

License

CC BY-NC 4.0.