Downloads · 30 days
13
16% of all-time downloads
demonlxrd/olmoe-openhermes-ultrafeedback-dora-dpo-merged
olmoe-openhermes-ultrafeedback-dora-dpo-merged is a text generation model from demonlxrd. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This is the merged version of demonlxrd/olmoe-openhermes-ultrafeedback-dora-dpo - a preference-aligned OLMoE model trained with DoRA and DPO.
Downloads · 30 days
13
16% of all-time downloads
All-time downloads
83
Public
Parameters
6.9B
13.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors13.8 GB · 100%
From the Hugging Face model README
This is the merged version of demonlxrd/olmoe-openhermes-ultrafeedback-dora-dpo - a preference-aligned OLMoE model trained with DoRA and DPO.
A fully merged model ready for production deployment. The DoRA adapter has been merged into the base OLMoE-1B-7B weights for:
Training pipeline:
# Serve
vllm serve demonlxrd/olmoe-openhermes-ultrafeedback-dora-dpo-merged \
--max-model-len 4096 \
--dtype bfloat16
# Inference
curl -s http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "demonlxrd/olmoe-openhermes-ultrafeedback-dora-dpo-merged",
"messages": [
{"role": "user", "content": "Explain machine learning in simple terms."}
],
"max_tokens": 200,
"temperature": 0.7
}' | jq -r '.choices[0].message.content'
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("demonlxrd/olmoe-openhermes-ultrafeedback-dora-dpo-merged")
model = AutoModelForCausalLM.from_pretrained(
"demonlxrd/olmoe-openhermes-ultrafeedback-dora-dpo-merged",
device_map="auto",
torch_dtype=torch.bfloat16
)
messages = [{"role": "user", "content": "What is quantum computing?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.inference_mode():
outputs = model.generate(**inputs, max_tokens=200, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8000/v1",
api_key="dummy"
)
response = client.chat.completions.create(
model="demonlxrd/olmoe-openhermes-ultrafeedback-dora-dpo-merged",
messages=[
{"role": "user", "content": "Write a Python function to calculate fibonacci numbers."}
],
max_tokens=300,
temperature=0.7
)
print(response.choices[0].message.content)
| Parameter | Value |
|---|---|
| Architecture | OLMoE (Mixture of Experts) |
| Parameters | ~1B active, 7B total |
| Precision | bfloat16 |
| Context Length | 4096 tokens |
| Training | SFT + DPO with DoRA adapters |
| Base Model | 1024m/OLMoE-1B-7B-0924-Base |
User:
<message>
Assistant:
<response>
Roles supported: system, user, assistant
Looking for the lightweight adapter weights? Check out demonlxrd/olmoe-openhermes-ultrafeedback-dora-dpo (~8.3MB)
Apache 2.0. Please also check the license of the base model.
If you use this model, please cite: