ngnn-mteb-local-preview

NGNN MTEB Local Publish Preview

This repository is a public research preview for review and discussion. It is not official MTEB leaderboard data.

It contains 9 rows across 3 tasks for these public result labels: Full embedding baseline, 256-dim baseline, Sparse NGNN.

Model Summary

This preview reports exploratory MTEB-style results for an NGNN-based embedding compression path. The experiments compare full-size text embeddings, a 256-dimensional baseline, and a sparse NGNN compressed representation across three benchmark tasks.

The base embedding source is OpenAI text-embedding-3-large. The NGNN path is evaluated as an experimental compression method for reducing embedding representation size while tracking task quality.

This is not a released standalone encoder and not an official MTEB leaderboard submission.

Public Comparison Rows

The preview intentionally exposes only three public comparison rows:

• Full embedding baseline: the uncompressed embedding reference.
• 256-dim baseline: a 256-dimensional PCA/SVD baseline.
• Sparse NGNN: the restricted public NGNN sparse compressed representation.

The preview intentionally omits additional non-public comparison rows and keeps only the restricted public comparison set.

Intended Use

This preview is intended for research review of embedding compression behavior. It can be used to inspect how a sparse NGNN compressed representation compares with full embeddings and a 256-dimensional baseline on the included tasks.

It should not be used as evidence of official MTEB leaderboard performance, and it should not be treated as a production-ready public encoder.

Evaluation

The included results cover three tasks:

• SciFact
• STSBenchmark
• Banking77Classification.v2

These rows are exploratory task-local evaluations and are not official MTEB leaderboard submissions. No mteb.evaluate(...) run or ResultCache export was used to create this preview. All rows set eligible_for_mteb_leaderboard=false.

Restricted Result Table

The same restricted result table is also available as public_results_table.md, with machine-readable copies in public_results.csv and public_results.json.

Limitations

These results are exploratory and task-local. The NGNN path is not yet a frozen public encoder, and the current preview does not establish leaderboard eligibility or broad generalization beyond the included tasks.