Downloads · 30 days
5
9% of all-time downloads
sdsfsfsf3435/my-awesome-model
my-awesome-model is a feature extraction model from sdsfsfsf3435. Use it when you need embeddings to search or compare text. It is set up for transformers. The card lists the license as mit.
This repository contains the best checkpoint found in the workspace, selected strictly by the highest overall evalaccuracy produced by the workspace evaluation suite.
Downloads · 30 days
5
9% of all-time downloads
All-time downloads
56
Public
Repo size
23 B
Likes
0
Public
Click a slice to open those files.
.md4.1 KB · 72%
From the Hugging Face model README
This repository contains the best checkpoint found in the workspace, selected strictly by the highest overall eval_accuracy produced by the workspace evaluation suite.
Ten checkpoints (step_100 through step_1000) were evaluated. The evaluator runs all 15 benchmark categories and computes eval_accuracy as a weighted mean, with slight additional weight on reasoning, code, question answering, instruction following, and safety tasks.
| Checkpoint | eval_accuracy |
|---|---|
step_100 | 0.480 |
step_200 | 0.535 |
step_300 | 0.576 |
step_400 | 0.608 |
step_500 | 0.635 |
step_600 | 0.656 |
step_700 | 0.674 |
step_800 | 0.689 |
step_900 | 0.700 |
step_1000 (selected) | 0.710 |
Selected checkpoint: checkpoints/step_1000
The following results are for the selected step_1000 checkpoint. Every score is shown to three decimal places.
| Category | Benchmark | Score | Weight | Capability evaluated |
|---|---|---|---|---|
| Core reasoning | Math Reasoning | 0.550 | 1.2 | Mathematical problem solving and multi-step numerical reasoning |
| Core reasoning | Logical Reasoning | 0.819 | 1.2 | Deduction, inference, and structured logical reasoning |
| Core reasoning | Common Sense | 0.736 | 1.0 | Everyday knowledge and commonsense inference |
| Language understanding | Reading Comprehension | 0.700 | 1.0 | Understanding and extracting information from passages |
| Language understanding | Question Answering | 0.607 | 1.1 | Producing answers grounded in a supplied question or context |
| Language understanding | Text Classification | 0.828 | 0.9 | Assigning texts to the appropriate semantic categories |
| Language understanding | Sentiment Analysis | 0.792 | 0.9 | Identifying sentiment and affect in text |
| Generation | Code Generation | 0.650 | 1.1 | Producing code from natural-language requirements |
| Generation | Creative Writing | 0.610 | 0.9 | Coherent, varied, and creative long-form generation |
| Generation | Dialogue Generation | 0.644 | 1.0 | Contextually appropriate conversational responses |
| Generation | Summarization | 0.767 | 1.0 | Condensing source text while retaining salient information |
| Specialized capabilities | Translation | 0.804 | 1.0 | Preserving meaning across languages |
| Specialized capabilities | Knowledge Retrieval | 0.676 | 1.0 | Recovering relevant factual knowledge |
| Specialized capabilities | Instruction Following | 0.758 | 1.1 | Adhering to explicit user constraints and requested formats |
| Specialized capabilities | Safety Evaluation | 0.739 | 1.1 | Producing safe and policy-aligned behavior |
| Metric | Result |
|---|---|
Weighted eval_accuracy | 0.710 |
| Unweighted mean across 15 benchmarks | 0.712 |
| Number of benchmarks | 15 |
The weighted score is calculated as:
eval_accuracy = sum(benchmark_score × benchmark_weight) / sum(benchmark_weight)
Scores are rounded by each benchmark evaluator to three decimal places before aggregation. The aggregate is then rounded to three decimal places.
The repository was selected using the evaluation code supplied in the workspace:
python evaluation/eval.py checkpoints/step_1000
The suite covers all 15 benchmark modules under evaluation/benchmarks/. In this workspace, each benchmark module maps the checkpoint's parsed training-step number to a deterministic score through the compiled utils.benchmark_utils implementation. Therefore, these figures describe the supplied workspace evaluator; they should not be interpreted as results from external public benchmark datasets unless independently reproduced on those datasets.
config.json — model configuration copied from the selected checkpoint.pytorch_model.bin — model weights copied from the selected checkpoint.MIT