Downloads · 30 days
17
2% of all-time downloads
jondurbin/bagel-dpo-1.1b-v0.3
bagel-dpo-1.1b-v0.3 is a text generation model from jondurbin. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Downloads · 30 days
17
2% of all-time downloads
All-time downloads
825
Public
Parameters
1.1B
2.2 GB on disk
Likes
2
Public
Click a slice to open those files.
.safetensors2.2 GB · 100%
From the Hugging Face model README

This model is basically unusable; sadly tinyllama is not a useful base model
An experimental fine-tune of tinyllama using bagel
Yes, you will see benchmark names in the list, but this only uses the train splits, and a decontamination by cosine similarity is performed at the end as a sanity check
Only the train splits were used (if a split was provided), and an additional pass of decontamination is performed using approximate nearest neighbor search (via faiss).
In sticking with the theme of the bagel, I didn't want to use a single prompt format, so I used 4 - vicuna, llama-2, alpaca, and chat-ml (sorta). I also didn't want to randomly select a single prompt format for each item (hoping each instruction would generalize more when used in a variety of prompt formats), so each instruction is actually converted into every prompt format.
This means each epoch of our fine-tune is really basically 4 epochs. So, for the fine-tunes, I would recommend only doing 1 epoch (or 0.75 epochs). I am testing with a single epoch using a relatively low learning rate.
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{system prompt, if provided}
{instruction}
### Response:
The main difference here is that because of the dataset formatting and variety of data sources, it would have been much to tedious to add an ### Input: block, so the inputs are just in the instruction section.
{system prompt, if provided, randomly defaulting to "A chat between a user and an unbiased, uncensored assistant."}
USER: {instruction}
ASSISTANT:
I don't really understand the point of having special tokens for <|im_start|> and <|im_end|>, because in practice they just act as BOS and EOS tokens (but, please correct me if I'm wrong).
So, instead of:
{bos}<|im_start|>{role}
{text}
<|im_end|>{eos}
I just changed it to:
{bos}{role}
{text}
{eos}
If you really want to use <|im_start|> and <|im_end|>, just update your tokenizer_config.json to use <|im_start|> instead of <s> and <|im_end|> instead of </s> and when tokenizing. And if you still don't like what I've done to this chat-ml-ish format, feel free to cry into your pillow or fork the code and do a new fine-tune.
[INST] <<SYS>>
{system}
<</SYS>>
{instruction} [/INST]
The model's tokenizer_config.json includes the default chat template (llama-2), so you can simply use the apply_chat_template method to build the full prompt.
import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained('jondurbin/bagel-8x7b-v0.2')
chat = [
{"role": "system", "content": "You are Bob, a friendly AI assistant."},
{"role": "user", "content": "Hello, how are you?"},
{"role": "assistant", "content": "I'm doing great. How can I help you today?"},
{"role": "user", "content": "I'd like to show off how chat templating works!"},
]
print(tokenizer.apply_chat_template(chat, tokenize=False))
If you're interested in new functionality/datasets, take a look at bagel repo and either make a PR or open an issue with details.
To help me with the fine-tuning costs (which are extremely expensive for these large combined datasets):
The base model is tinyllama, which is licensed as apache-2.0 - no issues there.
The fine-tuning data, however, includes several datasets that have data generated at least in part by OpenAI's gpt-4.
I am not a lawyer, so I can't help determine if this is actually commercially viable, but some questions that often come up are:
Use your best judgement and seek legal advice if you are concerned about the terms. In any case, by using this model, you agree to completely indemnify me.