news-impact-v1

TBV NewsImpact v1 — Research Diagnostic Checkpoint

⚠️ NOT FOR PRODUCTION USE — QUARANTINED RESEARCH ARTIFACT

This checkpoint failed its own internal audit (2026-05-13). It is published as a diagnostic artifact so the catchem governance pipeline and downstream research can reproduce the audit findings against the actual weights. The companion catchem project loads it only through newsimpact_guarded_adapter which refuses to enable it outside research_diagnostic mode and only when governance_index.release_gate_passed == False.

If you are looking for a working news-impact model, this is not it. Read AUDIT_REPORT.md before any other section.

---

TL;DR

---

Why this is published despite failing audit

catchem is an open trading-news pipeline that consumes this checkpoint only in research-diagnostic mode. The audit findings are not hidden — they are the entire point of the quarantine. Publishing the weights lets independent researchers:

1. Reproduce the cross-attention collapse proof (cross_attention_collapse_proof.md)
2. Verify that swapping text inputs produces bit-identical outputs (the canonical text-is-unused test, see audit CRITICAL-1)
3. Test the same architecture with the collapse fixed against this baseline
4. Confirm the daily/4h chart resolution mixup at label time (CRITICAL-2)

---

⚠️ Known critical defects (must-read)

These are summarised here so a casual reader understands the risk before downloading. Full evidence and line numbers in AUDIT_REPORT.md.

CRITICAL-1 — Cross-attention fusion is a mathematical no-op

The fusion layer is nn.MultiheadAttention(1024, 8, batch_first=True) but the forward pass passes single-token tensors (B, 1, D) as Q/K/V. With T=1 the softmax over a one-element score vector is identically 1.0, so attention weights cannot modulate the value. The text encoder's output never reaches the prediction surface. The 336M-parameter RoBERTa-large is decorative.

You can verify this yourself by loading the checkpoint and comparing outputs for two different news strings paired with the same chart — they will be bit-identical. See section 7 of AUDIT_REPORT.md for the assertion script.

CRITICAL-2 — charts_4h/ contains daily bars; equity labels silently rejected

fetch_yfinance_daily writes daily bars to a directory named charts_4h/. compute_impact_label computes pre['close'].pct_change().std() over a 24h pre-window — which for daily bars contains 0–1 rows → std() = NaN → label rejected at pipeline.py:1051. The effective training corpus is filtered to crypto symbols only, silently. Equities, FX, indices, and commodities contribute zero labels despite being ingested.

CRITICAL-3 — Reported mean_val_loss is pre-merge per-shard, not the merged model

merge_shards_and_test writes the soft-averaged state dict and reports the mean of per-shard pre-merge validation losses. No re-assessment of the merged checkpoint occurs. The actual performance of final_best.pt is unknown in the original report. Weight averaging of independently-trained shards typically degrades performance; the merged model may be materially worse than 2.4951.

---

Architecture summary (from model_analysis.json)

final_best.pt
├── text_model        391 tensors  ->  355.36 M params  (1.36 GB)  -- roberta-large (frozen at train time)
├── chart_enc          53 tensors  ->    0.99 M params  (~4 MB)    -- 4-layer Transformer over (B, 128, 5) OHLCV
├── fuse                                                           -- nn.MultiheadAttention(1024, 8)  <- collapses (see CRITICAL-1)
├── head_sig                                                       -- Linear(1024, 2)   significance
├── head_dir                                                       -- Linear(1024, 3)   direction
├── head_mag                                                       -- Linear(1024, 1)   magnitude
└── head_reason                                                    -- Linear(1024, 1024) reason embedding (NEVER receives loss signal)

---

Files in this repository

ccf6c8b8f80d2f1264dedae93398a7c70dbd100a95701341f8447c1b1c585141  final_best.pt

Verify locally:

shasum -a 256 -c SHA256SUMS
# final_best.pt: OK

---

Intended use

This artifact exists for research diagnostics only:

• ✅ Reproducing the cross-attention collapse audit
• ✅ Benchmark baseline for "chart-only" performance
• ✅ Ablation studies (replace forward to enable fusion, then compare)
• ✅ Recurrence-test target for the catchem governance pipeline
• ✅ Teaching example of how a 360M-parameter model can be mathematically equivalent to a 6M-parameter chart classifier wearing a costume

It is not intended for:

• ❌ Live trading signals
• ❌ News-impact production scoring
• ❌ Down-stream fine-tuning without fixing CRITICAL-1 first
• ❌ Comparison against properly trained baselines without first reading the audit (the reported metric is materially misleading)

---

How catchem uses it

The catchem project at https://github.com/nazmiefearmutcu/catchem contains the NewsImpactGuardedAdapter that is the only sanctioned integration. The adapter loads governance metadata read-only and refuses to instantiate unless three independent guards hold:

1. catchem is configured for research_diagnostic mode (production_safe is refused outright),
2. the guards.newsimpact_diagnostic_enabled flag is True,
3. governance_index.json shows release_gate_passed = False (the expected quarantined state).

The adapter does not load final_best.pt; it only reads governance_index.json and emits a clearly-labeled newsimpact_diagnostic_v0 payload alongside each catchem record.

---

Suggested next steps before any production use

In priority order (from AUDIT_REPORT.md § 10):

1. Fix the cross-attention collapse. Either concat-then-project, or have ChartEncoder.forward return a (B, T, D) sequence (drop the .mean(dim=1) pool) and use cross-attention with chart as key/value sequence vs text-pool as a single query token. Add the test_text_input_affects_output pytest.
2. Fix compute_impact_label to pick one bar resolution per symbol and fail loudly on mismatch. Fix the vol-window ↔ return-window dimensional mismatch (use pre_vol * √12 for 48h horizon over 4h bars).
3. Add a held-out global test set and have merge_shards_and_test assess the merged checkpoint on it. Report both pre-merge mean and merged-on-holdout.
4. Set seeds at the top of every runner (torch.manual_seed, random.seed, np.random.seed). Run each config 3× with seeds {42, 1337, 7} and report mean ± std.
5. Either remove head_reason or train it against a meaningful target (sentence-encoder embedding of a per-pair rationale).

---

Citation

@misc{tbv_newsimpact_v1_2026,
  author       = {{Nazmi Efe Armutçu}},
  title        = {{TBV NewsImpact v1 (research-diagnostic, quarantined)}},
  year         = {2026},
  publisher    = {{Hugging Face}},
  howpublished = {\url{https://huggingface.co/AylinMaylinn/news-impact-v1}},
  note         = {Failed internal audit 2026-05-13; published as quarantined
                  research artifact. See AUDIT\_REPORT.md.}
}

---

License

Custom research-diagnostic-only license. Summary: free to download and study; not licensed for any production deployment, financial decision making, or redistribution as a "validated" news-impact model. If you build a corrected derivative, please cite the audit findings.

---

Original training: 2026-05-12 (Google Drive). Audit: 2026-05-13. Published to Hugging Face under catchem's governance pipeline: 2026-05-27.