Downloads · 30 days
26
12% of all-time downloads
overthelex/qwen2.5-1.5b-edrsr-legal-uk
qwen2.5-1.5b-edrsr-legal-uk is a text generation model from overthelex. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Ukrainian legal domain model obtained by continued pretraining (CPT) of Qwen/Qwen2.5-1.5B on the EDRSR corpus of Ukrainian court decisions.
Downloads · 30 days
26
12% of all-time downloads
All-time downloads
221
Public
Parameters
1.5B
3.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.1 GB · 100%
From the Hugging Face model README
Ukrainian legal domain model obtained by continued pretraining (CPT) of Qwen/Qwen2.5-1.5B on the EDRSR corpus of Ukrainian court decisions.
Part of a scaling experiment (0.5B / 1.5B / 3B / 14B) for the PhD dissertation at Glushkov Institute of Cybernetics, NAS of Ukraine.
| Metric | Value |
|---|---|
| Initial loss (step 10) | 1.2216 |
| Final loss (step 9,536) | 0.2391 |
| Loss reduction | -80% |
| Base perplexity | 4.61 |
| CPT perplexity | 1.31 |
| Perplexity reduction | -71.5% |
All four models in the series converge to similar perplexity after CPT:
This is a base model (not instruction-tuned). It is intended for: