Downloads · 30 days
16
17% of all-time downloads
overthelex/qwen2.5-3b-edrsr-legal-uk
qwen2.5-3b-edrsr-legal-uk is a text generation model from overthelex. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Ukrainian legal domain model obtained by continued pretraining (CPT) of Qwen2.5-3B on the EDRSR corpus of Ukrainian court decisions.
Downloads · 30 days
16
17% of all-time downloads
All-time downloads
94
Public
Parameters
3.1B
6.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors6.2 GB · 100%
From the Hugging Face model README
Ukrainian legal domain model obtained by continued pretraining (CPT) of Qwen2.5-3B on the EDRSR corpus of Ukrainian court decisions.
Part of a scaling experiment (0.5B / 1.5B / 3B / 14B) for the PhD dissertation at Glushkov Institute of Cybernetics, NAS of Ukraine.
| Metric | Value |
|---|---|
| Initial loss (step 10) | 1.08 |
| Final loss (step 9,536) | 0.231 |
| Loss reduction | -79% |
| Base perplexity | 3.83 |
| CPT perplexity | 1.30 |
| Perplexity reduction | -66.1% |
All four models in the series converge to similar perplexity after CPT:
This is a base model (not instruction-tuned). It is intended for: