Downloads · 30 days
31
2% of all-time downloads
open-sci-ref-v0.02-1.7b-dclm-1T-4096-rope_theta-100k
open-sci/open-sci-ref-v0.02-1.7b-dclm-1T-4096-rope_theta-100k
open-sci-ref-v0.02-1.7b-dclm-1T-4096-rope_theta-100k is a text generation model from open-sci. Use it when you need the model to write or continue text. It is set up for transformers.
1.7B open-sci-ref model trained on DCLM for 1T tokens (sequence length 4096, RoPE theta = 100000).
All-time downloads
2K
Public
Parameters
1.7B
315 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- opensci
- Access
- Public
- Created
- May 30, 2026
- Updated
- May 30, 2026
- SHA
- 7e213fbb
Try a prompt
Model card
From the Hugging Face model README
open-sci-ref-v0.02-1.7b-dclm-1T-4096-rope_theta-100k
1.7B open-sci-ref model trained on DCLM for 1T tokens (sequence length 4096, RoPE theta = 100000).
The main branch holds the final checkpoint (iter 238419). Intermediate checkpoints (iters 58000-238000, every 2000) are available as branches named iter_XXXXXXX.
Evaluation
Final checkpoint on the open-sci-0.01 suite (lm-eval-harness). Metrics collected with oellm collect-results.
| Task | n-shot | Metric | Score |
|---|---|---|---|
| arc_challenge | 10 | acc_norm | 0.4761 |
| arc_easy | 10 | acc_norm | 0.7689 |
| boolq | 10 | acc | 0.7664 |
| commonsense_qa | 10 | acc | 0.4676 |
| copa | 0 | acc | 0.8200 |
| hellaswag | 10 | acc_norm | 0.7367 |
| lambada_openai | 0 | acc | 0.7019 |
| mmlu | 5 | acc | 0.4002 |
| openbookqa | 0 | acc_norm | 0.4080 |
| piqa | 10 | acc_norm | 0.7845 |
| social_iqa | 0 | acc | 0.4463 |
| winogrande | 0 | acc | 0.6606 |
| average | 0.6198 |
- Task
- Text Generation
- Library
- transformers
- Type
- opensci
- Created
- May 30, 2026
- Updated
- May 30, 2026