Downloads · 30 days
563
22% of all-time downloads
Deeokay/mistral-7b-v0.3-custom-tokens-GGUF
mistral-7b-v0.3-custom-tokens-GGUF is a machine learning model from Deeokay. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
- Developed by: Deeokay - License: apache-2.0 - Finetuned from model : unsloth/mistral-7b-v0.3-bnb-4bit
Downloads · 30 days
563
22% of all-time downloads
All-time downloads
2.5K
Public
Repo size
75.3 GB
Likes
0
Public
Click a slice to open those files.
.gguf25.1 GB · 100%
From the Hugging Face model README
This mistral model was trained 2x faster with Unsloth and Huggingface's TRL library.
This is a test model on a the following
The whole point of conversion for me was I wanted to be able to to use it through Ollama or (other local options) For Ollama, it required to be a GGUF file. Once you have this it is pretty straight forward
If you want to try it first, the Q2_K version of this is available in Ollama => deeokay/minimistral
ollama pull deeokay/minimistral
FROM ./mistrial_unsloth.Q2_K.gguf
PARAMETER stop <|STOP|>
PARAMETER stop "<|STOP|>"
PARAMETER stop <|END_RESPONSE|>
PARAMETER stop "<|END_RESPONSE|>"
PARAMETER temperature 0.4
TEMPLATE """<|BEGIN_QUERY|>
{{.Prompt}}
<|END_QUERY|>
<|BEGIN_RESPONSE|>
"""
SYSTEM """You are an AI assistant. Respond to the user's query between the BEGIN_QUERY and END_QUERY tokens. Use the appropriate BEGIN_ and END_ tokens for different types of content in your response.""""""
ollama create mycustomai # "mycustomai" <- you can name it anything u want
After than you should be able to use this model to chat! This GGUF is based on unsloth/mistral-7b-instruct-v0.3-bnb-4bit by Unslot,
Please note this is not for the purpose of production, but result of Fine Tuning through self learning This is my Fine Tuning pass through with personalized customized dataset. Please feel free to customize the Modelfile, and if you do get a better response than mine, please share!!
If would like to know how I started creating my dataset, you can check this link Crafting GPT2 for Personalized AI-Preparing Data the Long Way (Part1)
special_tokens_dict = {
'eos_token': '<|STOP|>',
'bos_token': '<|STOP|>',
'pad_token': '<|PAD|>',
'additional_special_tokens': ['<|BEGIN_QUERY|>', '<|BEGIN_QUERY|>',
'<|BEGIN_ANALYSIS|>', '<|END_ANALYSIS|>',
'<|BEGIN_RESPONSE|>', '<|END_RESPONSE|>',
'<|BEGIN_SENTIMENT|>', '<|END_SENTIMENT|>',
'<|BEGIN_CLASSIFICATION|>', '<|END_CLASSIFICATION|>',]
}
tokenizer.add_special_tokens(special_tokens_dict)
model.resize_token_embeddings(len(tokenizer))
tokenizer.eos_token_id = tokenizer.convert_tokens_to_ids('<|STOP|>')
tokenizer.bos_token_id = tokenizer.convert_tokens_to_ids('<|STOP|>')
tokenizer.pad_token_id = tokenizer.convert_tokens_to_ids('<|PAD|>')
def combine_text(user_prompt, analysis, sentiment, new_response, classification):
user_q = f"<|STOP|><|BEGIN_QUERY|>{user_prompt}<|END_QUERY|>"
analysis = f"<|BEGIN_ANALYSIS|>{analysis}<|END_ANALYSIS|>"
new_response = f"<|BEGIN_RESPONSE|>{new_response}<|END_RESPONSE|>"
classification = f"<|BEGIN_CLASSIFICATION|>{classification}<|END_CLASSIFICATION|>"
sentiment = f"<|BEGIN_SENTIMENT|>Sentiment: {sentiment}<|END_SENTIMENT|><|STOP|>"
return user_q + analysis + new_response + classification + sentiment