Downloads · 30 days
76
14% of all-time downloads
Qwen2.5-7B-olm-v1.0
jeffmeloy/Qwen2.5-7B-olm-v1.0
Qwen2.5-7B-olm-v1.0 is a text generation model from jeffmeloy. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Optimized Layer Merging (OLM) Is a transformer optimization framework implementing automated layer recombination.
All-time downloads
543
Public
Parameters
7.6B
33.2 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors15.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- qwen2
- Access
- Public
- Created
- Dec 12, 2024
- Updated
- Dec 29, 2024
- SHA
- d113a301
Try a prompt
Base models
Model card
From the Hugging Face model README
Model Description
Optimized Layer Merging (OLM) Is a transformer optimization framework implementing automated layer recombination.
Olm create Frankenstein's monster out of language models by cherry-picking the best performing layers across different models to create a superior hybrid. The core mechanism:
- Takes multiple language models as input
- Uses a base model as the foundation
- Iteratively replaces individual layers, evaluating performance on specified datasets
- Keeps the best performing layer at each position based on metrics like perplexity, exact match, and a custom "quality" score
- Builds a fusion model layer-by-layer while maintaining or improving performance
- Task
- Text Generation
- Library
- transformers
- Type
- qwen2
- License
- apache-2.0
- Languages
- en
- Created
- Dec 12, 2024
- Updated
- Dec 29, 2024