Downloads · 30 days
0
crab-1-benchmark
gaidar12345/crab-1-benchmark
crab-1-benchmark is a machine learning model from gaidar12345. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc0-1.0.
The evaluation set used for the numbers in the Crab-1 model card: 30 French companies with hand-verified ground truth (official website, department, region, SIREN, registry data).
Access
Public
Updated Sep 1, 2026
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.json16 KB · 85%
At a glance
- License
- cc0-1.0
- Access
- Public
- Created
- Sep 1, 2026
- Updated
- Sep 1, 2026
- SHA
- e02f7326
Model card
From the Hugging Face model README
Crab-1 benchmark — 30 French companies, ground truth
The evaluation set used for the numbers in the Crab-1 model card: 30 French companies with hand-verified ground truth (official website, department, region, SIREN, registry data).
File
eval_ground_truth_v3_enriched.json— array of 30 records, each with:name— company name as given to the modelexpected— ground truth: website, city, department (+ code), region, postal code, SIREN, headcount, sectordifficulty— easy / medium / hardregistry— SIREN/SIRET, NAF code, headcount code from the official French registry
How it's used
The eval harness (eval/run_eval.py in
github.com/gaidar0yegor/crab-1)
runs the model against these 30 companies with live web tools and scores each
profile with harness/reward.py (website at registrable-domain level,
location at department level, sector when ground truth exists).
⚠️ The benchmark is perishable — the tools hit the live web, so scores drift over time. Only compare models measured the same day.
License
CC0 — the data is public facts (French company registry + public websites).
- License
- cc0-1.0
- Created
- Sep 1, 2026
- Updated
- Sep 1, 2026