Downloads · 30 days
286
6% of all-time downloads
BuRPInfinity_9B-GGUF-IQ-Imatrix
Lewdiculous/BuRPInfinity_9B-GGUF-IQ-Imatrix
BuRPInfinity_9B-GGUF-IQ-Imatrix is a machine learning model from Lewdiculous. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as other.
This repository hosts GGUF-IQ-Imatrix quants for jeiku/BuRPInfinity9B.
All-time downloads
4.6K
Public
Repo size
74.4 GB
Likes
5
Public
Repo makeup
Click a slice to open those files.
.gguf74.3 GB · 100%
At a glance
- Library
- transformers
- License
- other
- Access
- Public
- Created
- Mar 16, 2024
- Updated
- Mar 16, 2024
- SHA
- 4a38bded
Base models
Model card
From the Hugging Face model README
This repository hosts GGUF-IQ-Imatrix quants for jeiku/BuRPInfinity_9B.
Thanks @jeiku for merging this!
This is an experimental model. Feedback is appreciated as always.
Steps:
Base⇢ GGUF(F16)⇢ Imatrix-Data(F16)⇢ GGUF(Imatrix-Quants)
Using the latest llama.cpp at the time.
quantization_options = [
"Q4_K_M", "Q4_K_S", "IQ4_XS", "Q5_K_M", "Q5_K_S",
"Q6_K", "Q8_0", "IQ3_M", "IQ3_S", "IQ3_XXS"
]
BuRPInfinity_9B
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the passthrough merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: Endevor/InfinityRP-v1-7B
layer_range: [0, 20]
- sources:
- model: ChaoticNeutrals/BuRP_7B
layer_range: [12, 32]
merge_method: passthrough
dtype: float16
- Library
- transformers
- License
- other
- Languages
- en
- Created
- Mar 16, 2024
- Updated
- Mar 16, 2024