Downloads · 30 days
561
8% of all-time downloads
RichardErkhov/tenyx_-_TenyxChat-7B-v1-gguf
tenyx_-_TenyxChat-7B-v1-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
561
8% of all-time downloads
All-time downloads
7K
Public
Repo size
82.2 GB
Likes
0
Public
Click a slice to open those files.
.gguf95.2 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
TenyxChat-7B-v1 - GGUF
| Name | Quant method | Size |
|---|---|---|
| TenyxChat-7B-v1.Q2_K.gguf | Q2_K | 2.53GB |
| TenyxChat-7B-v1.IQ3_XS.gguf | IQ3_XS | 2.81GB |
| TenyxChat-7B-v1.IQ3_S.gguf | IQ3_S | 2.96GB |
| TenyxChat-7B-v1.Q3_K_S.gguf | Q3_K_S | 2.95GB |
| TenyxChat-7B-v1.IQ3_M.gguf | IQ3_M | 3.06GB |
| TenyxChat-7B-v1.Q3_K.gguf | Q3_K | 3.28GB |
| TenyxChat-7B-v1.Q3_K_M.gguf | Q3_K_M | 3.28GB |
| TenyxChat-7B-v1.Q3_K_L.gguf | Q3_K_L | 3.56GB |
| TenyxChat-7B-v1.IQ4_XS.gguf | IQ4_XS | 3.67GB |
| TenyxChat-7B-v1.Q4_0.gguf | Q4_0 | 3.83GB |
| TenyxChat-7B-v1.IQ4_NL.gguf | IQ4_NL | 3.87GB |
| TenyxChat-7B-v1.Q4_K_S.gguf | Q4_K_S | 3.86GB |
| TenyxChat-7B-v1.Q4_K.gguf | Q4_K | 4.07GB |
| TenyxChat-7B-v1.Q4_K_M.gguf | Q4_K_M | 4.07GB |
| TenyxChat-7B-v1.Q4_1.gguf | Q4_1 | 4.24GB |
| TenyxChat-7B-v1.Q5_0.gguf | Q5_0 | 4.65GB |
| TenyxChat-7B-v1.Q5_K_S.gguf | Q5_K_S | 4.65GB |
| TenyxChat-7B-v1.Q5_K.gguf | Q5_K | 4.78GB |
| TenyxChat-7B-v1.Q5_K_M.gguf | Q5_K_M | 4.78GB |
| TenyxChat-7B-v1.Q5_1.gguf | Q5_1 | 5.07GB |
| TenyxChat-7B-v1.Q6_K.gguf | Q6_K | 5.53GB |
| TenyxChat-7B-v1.Q8_0.gguf | Q8_0 | 7.17GB |
license: apache-2.0 language:
Introducing TenyxChat, a series of ChatGPT-like models trained to function as useful assistants through preference tuning, using Tenyx's recently released advanced fine-tuning technology (VentureBeat article). Our first chat model in the series, TenyxChat-7B-v1, is trained using the Direct Preference Optimization (DPO) framework on the open-source AI feedback dataset UltraFeedback.
We fine-tune Openchat-3.5 with our proprietary approach (blog, service), which shows an increase in MT-Bench, without a drop in performance of the model on other benchmarks. Our approach aims to mitigate forgetting in LLMs in a computationally efficient manner, thereby enabling continual fine-tuning capabilities without altering the pre-trained output distribution. TenyxChat-7B-v1 was trained using eight A100s (80GB) for two hours, with a training setup obtained from HuggingFaceH4 (GitHub).
Our model uses a simple chat template based on OpenChat 3.5. The chat template usage with a Hugging face generation example is shown below.
{{ bos_token }}
{% for message in messages %}
{% if message['role'] == 'user' %}
{{ 'User:' + message['content'] + eos_token }}
{% elif message['role'] == 'system' %}
{{ 'System:' + message['content'] + eos_token }}
{% elif message['role'] == 'assistant' %}
{{ 'Assistant:' + message['content'] + eos_token }}
{% endif %}
{% if loop.last and add_generation_prompt %}\n{{ 'Assistant:' }}{% endif %}\n{% endfor %}
import torch
from transformers import pipeline
pipe = pipeline("text-generation", model="tenyx/TenyxChat-7B-v1", torch_dtype=torch.bfloat16, device_map="auto")
messages = [
{"role": "system", "content": "You are a friendly chatbot who always responds in the style of a pirate."},
{"role": "user", "content": "Hi. I would like to make a hotel booking."},
]
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipe(prompt, max_new_tokens=512, do_sample=False)
<s> System:You are a friendly chatbot who always responds in the style of a pirate.<|end_of_turn|>
User:Hi. I would like to make a hotel booking.<|end_of_turn|>
Assistant: Ahoy there me hearty! Arr, ye be lookin' fer a place to rest yer weary bones, eh?
Well then, let's set sail on this grand adventure and find ye a swell place to stay!
To begin, tell me the location ye be seekin' and the dates ye be lookin' to set sail.
And don't ye worry, me matey, I'll be sure to find ye a place that'll make ye feel like a king or queen on land!
At the time of release (Jan 2024), TenyxChat-7B-v1 is the highest-ranked 7B chat model on the MT-Bench evaluation available for download and commercial use. We list here the benchmark results on several standard setups while comparing popular 7B models as baselines.
MT-Bench is a benchmark made up of 80 high-quality multi-turn questions. These questions fall into eight categories: Writing, Roleplay, Reasoning, Math, Coding, Extraction, STEM, and Humanities. The chat models are rated using GPT-4 on a scale of 1 to 10, with higher values corresponding to better responses.
| Model | First Turn | Second Turn | Average |
|---|---|---|---|
| GPT-4* | 8.95625 | 9.02500 | 8.990625 |
| TenyxChat-7B-v1 | 8.45000 | 7.75625 | 8.103125 |
| Starling-lm-7B-alpha | 8.42500 | 7.68750 | 8.056250 |
| OpenChat-3.5 | 8.18125 | 7.41250 | 7.796875 |
| GPT-3.5-turbo* | 8.07500 | 7.81250 | 7.943750 |
| OpenLLM Leader-7B** | 8.05000 | 7.61250 | 7.831250 |
*values reported on lmsys ChatBot Arena
**The OpenLLM Leader as of Jan 5, 2024 is the merge model available as samir-fama/SamirGPT-v1

| Model | First Turn | Second Turn | Average |
|---|---|---|---|
| TenyxChat-7B-v1 | 8.45000 | 7.756250 | 8.103125 |
| SamirGPT-v1 | 8.05000 | 7.612500 | 7.831250 |
| FernandoGPT-v1 | 8.08125 | 7.256250 | 7.668750 |
| Go-Bruins-v2 | 8.13750 | 7.150000 | 7.643750 |
| mistral_tv-neural-marconroni | 7.76875 | 6.987500 | 7.378125 |
| neuronovo-7B-v0.2 | 7.73750 | 6.662500 | 7.200000 |
| neural-chat-7b-v3-3 | 7.39375 | 5.881250 | 6.637500 |
We assess models on 7 benchmarks using the Eleuther AI Language Model Evaluation Harness. This setup is based of that used for Open LLM Leaderboard.
These benchmarks test reasoning and knowledge in various tasks in few-shot settings (higher scores are better).
| Model | MMLU | Winogrande | GSM8k | ARC | HellaSwag | TruthfulQA | Average |
|---|---|---|---|---|---|---|---|
| TenyxChat-7B-v1 | 63.6 | 72.3 | 69.0 | 62.7 | 66.6 | 46.7 | 63.48 |
| Starling-7B-alpha | 63.5 | 72.1 | 67.9 | 61.1 | 66.1 | 42.1 | 62.13 |
| OpenChat-3.5 | 63.6 | 72.1 | 68.2 | 61.3 | 65.2 | 41.8 | 62.03 |
| Mistral-7B | 62.4 | 74.0 | 38.1 | 57.2 | 62.8 | 37.8 | 55.38 |
| OpenLLM Leader-7B | 64.3 | 78.7 | 73.3 | 66.6 | 68.4 | 58.5 | 68.3 |
Note: While the Open LLM Leaderboard indicates that these chat models perform less effectively compared to the leading 7B model, it's important to note that the leading model struggles in the multi-turn chat setting of MT-Bench (as demonstrated in our evaluation above). In contrast, TenyxChat-7B-v1 demonstrates robustness against common fine-tuning challenges, such as catastrophic forgetting. This unique feature enables TenyxChat-7B-v1 to excel not only in chat benchmarks like MT-Bench, but also in a wider range of general reasoning benchmarks on the Open LLM Leaderboard.
TenyxChat-7B-v1, like other small-sized language models, has its own set of limitations. We haven’t fine-tuned the model explicitly to align with human safety preferences. Therefore, it is capable of producing undesirable outputs, particularly when adversarially prompted. From our observation, the model still tends to struggle with tasks that involve reasoning and math questions. In some instances, it might generate verbose or extraneous content.
TenyxChat-7B-v1, similar to OpenChat 3.5, is distributed under the Apache License 2.0.
If you use TenyxChat-7B for your research, cite us as
@misc{tenyxchat2024,
title={TenyxChat: Language Model Alignment using Tenyx Fine-tuning},
author={Tenyx},
year={2024},
}