Downloads · 30 days
3.3K
23% of all-time downloads
Deeokay/mistral-7b-v0.3-llama_tokens-GGUF
mistral-7b-v0.3-llama_tokens-GGUF is a machine learning model from Deeokay. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
- Developed by: Deeokay - License: apache-2.0 - Finetuned from model : unsloth/mistral-7b-instruct-v0.3-bnb-4bit
Downloads · 30 days
3.3K
23% of all-time downloads
All-time downloads
14.2K
Public
Repo size
124 GB
Likes
1
Public
Click a slice to open those files.
.gguf32.8 GB · 100%
From the Hugging Face model README
This mistral model was trained 2x faster with Unsloth and Huggingface's TRL library.
This is a test model on a the following
The whole point of conversion for me was I wanted to be able to to use it through Ollama or (other local options) For Ollama, it required to be a GGUF file. Once you have this it is pretty straight forward (if it is in llama3 which this model is)
Quick Start:
FROM ./unsloth.Q4_K_M.gguf
PARAMETER temperature 0.6
PARAMETER repeat_penalty 1.3
PARAMETER top_p 0.6
PARAMETER top_k 30
PARAMETER stop <|start_header_id|>
PARAMETER stop <|end_header_id|>
PARAMETER stop <|eot_id|>
TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
{{ .Response }}<|eot_id|>"
ollama create mycustomai # "mycustomai" <- you can name it anything u want
This GGUF is based on mistral-7b-v0.3
Please note this is not for the purpose of production, but result of Fine Tuning through self learning
The llama3 Special Tokens where used to convert the tokenizer.
I wanted to test if the model would understand additional headers that I created such as what my datasets has
Multiple pass through my personalized customized dataset, future updates will be made to this repo.
If would like to know how I started creating my dataset, you can check this link Crafting GPT2 for Personalized AI-Preparing Data the Long Way (Part1)
the training data has the following Template:
<|begin_of_text|> <|start_header_id|>user<|end_header_id|>
{{.Prompt}}<|eot_id|><|start_header_id|>analysis<|end_header_id|>
{{.Analysis}}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{{.Response}}<|eot_id|><|start_header_id|>classification<|end_header_id|>
{{.Classification}}<|eot_id|><|start_header_id|>sentiment<|end_header_id|>
{{.Sentiment}}<|eot_id|><|end_of_text|>