Downloads · 30 days
456
2% of all-time downloads
Vikhr-7B-instruct_0.4-GGUF
Vikhrmodels/Vikhr-7B-instruct_0.4-GGUF
Vikhr-7B-instruct_0.4-GGUF is a machine learning model from Vikhrmodels. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for llama-cpp.
- Quantized from the original diffusers BF16 version: Vikhrmodels/Vikhr-7B-instruct0.4
All-time downloads
22K
Public
Repo size
122 GB
Likes
9
Public
Repo makeup
Click a slice to open those files.
.gguf136 GB · 100%
At a glance
- Library
- llama-cpp
- Access
- Public
- Created
- May 1, 2024
- Updated
- May 5, 2024
- SHA
- 1e972d6e
Model card
From the Hugging Face model README
- Quantized from the original diffusers BF16 version: Vikhrmodels/Vikhr-7B-instruct_0.4
Ollama
To get Q4_1 version model, one can simply
ollama pull wavecut/vikhr
or create the model using other bpw versions using Ollama Modelfile
FROM ./vikhr-7b-instruct_0.4.INSERT_YOUR_QUANT_HERE.gguf
PARAMETER temperature 0.25
PARAMETER top_k 50
PARAMETER top_p 0.98
PARAMETER num_ctx 1512
PARAMETER stop <|im_end|>
PARAMETER stop <|im_start|>
SYSTEM """"""
TEMPLATE """<s>{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
"""
ollama create vikhr -f Modelfile
ollama run vikhr
- Library
- llama-cpp
- Languages
- ru, en
- Created
- May 1, 2024
- Updated
- May 5, 2024