Hermes-2.5-Yi-1.5-9B-Chat-GGUF

Hermes-2.5-Yi-1.5-9B-Chat-GGUF

This model is a fine-tuned version of 01-ai/Yi-1.5-9B-Chat on the teknium/OpenHermes-2.5 dataset. I'm very happy with the results. The model now seems a lot smarter and "aware" in certain situations. It got quite an big edge on the AGIEval Benchmark for models in it's class.

I plan to extend its context length to 32k with POSE. This is the GGUF repo. You can find the main repo here Hermes-2.5-Yi-1.5-9B-Chat.

Model Details

• Base Model: 01-ai/Yi-1.5-9B-Chat
• chat-template: chatml
• Dataset: teknium/OpenHermes-2.5
• Sequence Length: 8192 tokens
• Training:
• Epochs: 1
• Hardware: 4 Nodes x 4 NVIDIA A100 40GB GPUs
• Duration: 48:32:13
• Cluster: KIT SCC Cluster

Benchmark n_shots=0

For detailed benchmark results, including sub-categories and various metrics, please refer to the full benchmark table at the end of this README.

GGUF and Quantizations

• llama.cpp b3166
• juvi21/Hermes-2.5-Yi-1.5-9B-Chat-GGUF is availabe in:
• F16 Q8_0 Q6_KQ5_K_M Q4_K_M Q3_K_M Q2_K

chatml

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
Knock Knock, who is there?<|im_end|>
<|im_start|>assistant
Hi there! <|im_end|>

License

This model is released under the Apache 2.0 license.

Acknowledgements

Special thanks to:

• Teknium for the great OpenHermes-2.5 dataset
• 01-ai for their great model
• KIT SCC for FLOPS

Citation

If you use this model in your research, consider citing. Although definetly cite NousResearch and 01-ai:

@misc{
  author = {juvi21},
  title = Hermes-2.5-Yi-1.5-9B-Chat},
  year = {2024},
}

full-benchmark-results