Downloads · 30 days
17
1% of all-time downloads
open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-1T-4096-rope_theta-100k
open-sci/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-1T-4096-rope_theta-100k
open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-1T-4096-rope_theta-100k is a text generation model from open-sci. Use it when you need the model to write or continue text. It is set up for transformers.
1.7B open-sci-ref model trained on FineWeb-Edu for 1T tokens (sequence length 4096, RoPE theta = 100000).
All-time downloads
2.3K
Public
Parameters
1.7B
315 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- opensci
- Access
- Public
- Created
- May 30, 2026
- Updated
- May 30, 2026
- SHA
- 4f69454f
Try a prompt
Model card
From the Hugging Face model README
open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-1T-4096-rope_theta-100k
1.7B open-sci-ref model trained on FineWeb-Edu for 1T tokens (sequence length 4096, RoPE theta = 100000).
The main branch holds the final checkpoint (iter 238419). Intermediate checkpoints (iters 58000-238000, every 2000) are available as branches named iter_XXXXXXX.
Evaluation
Final checkpoint on the open-sci-0.01 suite (lm-eval-harness). Metrics collected with oellm collect-results.
| Task | n-shot | Metric | Score |
|---|---|---|---|
| arc_challenge | 10 | acc_norm | 0.4983 |
| arc_easy | 10 | acc_norm | 0.7984 |
| boolq | 10 | acc | 0.7465 |
| commonsense_qa | 10 | acc | 0.2285 |
| copa | 0 | acc | 0.8400 |
| hellaswag | 10 | acc_norm | 0.7008 |
| lambada_openai | 0 | acc | 0.5849 |
| mmlu | 5 | acc | 0.3187 |
| openbookqa | 0 | acc_norm | 0.4360 |
| piqa | 10 | acc_norm | 0.7873 |
| social_iqa | 0 | acc | 0.4365 |
| winogrande | 0 | acc | 0.6425 |
| average | 0.5849 |
- Task
- Text Generation
- Library
- transformers
- Type
- opensci
- Created
- May 30, 2026
- Updated
- May 30, 2026