ConvMemory-v3-Validity-Context

ConvMemory v3 Validity Context Model Card

This document separates the three source-of-truth layers for ConvMemory v3:

1. method-level evaluation;
2. the exported representative checkpoint;
3. package-level API measurement with that checkpoint.

These are intentionally different provenance layers. Method-level numbers estimate the v3 approach across seeds. The checkpoint is a representative implementation of that approach. The package-level benchmark is the number a user should expect when loading that checkpoint through the public API on the fixed dense Memora retrieval benchmark.

Scope

ConvMemory v3 adds validity evidence to the existing v1/v2 retrieval path. The default v3 use is validity_mode="context": it attaches a structured validity field to returned memories and preserves the candidate set and ranking order.

validity_mode="demote" is explicit opt-in. It is intended for dense current-state/update workloads where a top-1 source evidence policy is available. It preserves the candidate set and may reorder by applying a validity penalty.

ConvMemory v3 does not make full automatic dependency-graph propagation the default retrieval path. Multi-hop graph propagation is used as an evidence-path and analysis capability unless the caller supplies a workload where graph construction has been validated.

Checkpoint

The representative v3 validity checkpoint is exported by the v557 recipe:

The checkpoint implements the v511 query-conditioned validity method. It should not be used as a replacement for the v511 multi-seed method-level estimate when reporting method quality.

Input Format

The validity scorer uses the v506/v511 query/source/target format:

USER_QUERY:
...

SOURCE_EVIDENCE:
...

TASK: Decide whether the target memory should be demoted for this user query.

paired with:

TARGET_MEMORY:
...

The package exposes two scoring paths:

• ValidityEvidenceModule.apply(...): annotate or demote RerankResult objects while preserving the mode contracts.
• ValidityEvidenceModule.score_evidence_pairs(...): batch explicit query/source/target pairs after source evidence has already been selected.

The second path is the preferred dense-workload path because it avoids per-pair CrossEncoder calls.

Package Integration

Use convmemory>=0.6.2 for the bounded integrated source-selection path. Applications with an update index can pass one selected source per target with validity_source_map; otherwise the package uses a bounded top-1 lexical fallback over the protected result prefix. The package does not run an unbounded all-pairs CrossEncoder search.

Method-Level Evaluation

The v511 5-seed Memora-retrieval benchmark is the method-level estimate. It scores 69,200 source-query rows across seeds [7, 11, 23, 31, 47].

Top-1 retrieved source, max aggregation:

This table is the right citation for method-level claims.

Package-Level Check

The v558 public API benchmark loads the exported v557 checkpoint through ValidityEvidenceModule.from_pretrained(...) and scores the same top-1 source policy through the package API.

Top-1 retrieved source, max aggregation:

The v558 number is the package-level reproducibility check for this checkpoint. It is a single-checkpoint measurement, not a replacement for the v511 multi-seed method-level estimate.

Safety Contracts

The package-level safety checks from v558 all pass:

The test suite also covers off-mode byte identity, context-mode rank preservation, demote candidate-set preservation, explicit opt-in semantics, forbidden-field rejection, safe evidence output, checkpoint round-trip, and batched CrossEncoder scoring.

Operating Policy

Top-3/top-5 source aggregation is not the default policy because earlier v499, v502, and v503 runs showed that adding more sources can introduce false positive demotions. Full top-500 graph construction is also not the default path because learned graph errors can be amplified by propagation.

Source-Of-Truth Ledger

The results/... packets are source-of-truth evaluation artifacts kept with the author workspace unless explicitly packaged with a release. The package code, tests, and documentation are the public reproducibility surface once tag v0.6.0 is cut.

Known Boundaries

• The v3 checkpoint is trained for query-conditioned validity decisions with source evidence. It is not a generic factuality judge.
• Automatic demotion is intended for dense current-state/update workloads. General sparse retrieval should use context annotation by default.
• Broad learned source retrieval and automatic strict dependency graph construction are not part of the default v3 retrieval contract.
• The v511 method-level estimate and v558 package-level benchmark use different but connected provenance layers; report them with their layer names.