qwen2.5-7b-openhermes-v1

Qwen2.5-7B OpenHermes V1

A fine-tuned version of Qwen/Qwen2.5-7B on the OpenHermes-2.5 dataset for instruction following.

Model Details

Training Configuration

LoRA Target Modules

• q_proj, k_proj, v_proj, o_proj
• gate_proj, up_proj, down_proj

Training Results

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("samarthraina/qwen2.5-7b-openhermes-v1")
tokenizer = AutoTokenizer.from_pretrained("samarthraina/qwen2.5-7b-openhermes-v1")

messages = [
    {"role": "user", "content": "Explain quantum computing in simple terms."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Hardware

• GPU: NVIDIA RTX 4090 (48GB)
• Training Time: ~21 hours

Limitations

• Trained on 250k samples (subset of full OpenHermes-2.5)
• Single epoch training
• Conservative hyperparameters (lr=2e-5, r=32)
• May not perform optimally on tasks outside the training distribution

License

This model is released under the Apache 2.0 license, following the base model's license.

Citation

If you use this model, please cite:

@misc{qwen2.5-7b-openhermes-v1,
  author = {Samarth Raina},
  title = {Qwen2.5-7B OpenHermes V1},
  year = {2026},
  publisher = {HuggingFace},
  url = {https://huggingface.co/samarthraina/qwen2.5-7b-openhermes-v1}
}