Downloads · 30 days
356
11% of all-time downloads
olmo-gguf
calcuis/olmo-gguf
olmo-gguf is a text generation model from calcuis. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
All-time downloads
3.1K
Public
Repo size
57.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf57.3 GB · 100%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Access
- Public
- Created
- Oct 23, 2024
- Updated
- Oct 23, 2024
- SHA
- 20c0a40f
Try a prompt
Base models
Model card
From the Hugging Face model README
GGUF quantized version of OLMo-7B-0724-Instruct Model
project original source: base model
Q_2_K (not nice)
Q_3_K_S (acceptable)
Q_3_K_M is acceptable (good for running with CPU)
Q_3_K_L (acceptable)
Q_4_K_S (okay)
Q_4_K_M is recommanded (balance)
Q_5_K_S (good)
Q_5_K_M (good in general)
Q_6_K is good also; if you want a better result; take this one instead of Q_5_K_M
Q_8_0 which is very good; need a reasonable size of RAM otherwise you might expect a long wait
f16 is similar to the original hf model; opt this one or hf also fine; make sure you have a good machine
how to run it
use any connector for interacting with gguf; i.e., gguf-connector
<img src="https://allenai.org/olmo/olmo-7b-animation.gif" alt="OLMo Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/> this picture is from the base model- Task
- Text Generation
- License
- apache-2.0
- Languages
- en
- Created
- Oct 23, 2024
- Updated
- Oct 23, 2024