Downloads · 30 days
31
2% of all-time downloads
RichardErkhov/Xclbr7_-_caliburn-12b-gguf
Xclbr7_-_caliburn-12b-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
31
2% of all-time downloads
All-time downloads
1.4K
Public
Repo size
141 GB
Likes
0
Public
Click a slice to open those files.
.gguf163 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
caliburn-12b - GGUF
| Name | Quant method | Size |
|---|---|---|
| caliburn-12b.Q2_K.gguf | Q2_K | 4.46GB |
| caliburn-12b.IQ3_XS.gguf | IQ3_XS | 4.94GB |
| caliburn-12b.IQ3_S.gguf | IQ3_S | 5.18GB |
| caliburn-12b.Q3_K_S.gguf | Q3_K_S | 5.15GB |
| caliburn-12b.IQ3_M.gguf | IQ3_M | 5.33GB |
| caliburn-12b.Q3_K.gguf | Q3_K | 5.67GB |
| caliburn-12b.Q3_K_M.gguf | Q3_K_M | 5.67GB |
| caliburn-12b.Q3_K_L.gguf | Q3_K_L | 6.11GB |
| caliburn-12b.IQ4_XS.gguf | IQ4_XS | 6.33GB |
| caliburn-12b.Q4_0.gguf | Q4_0 | 6.59GB |
| caliburn-12b.IQ4_NL.gguf | IQ4_NL | 6.65GB |
| caliburn-12b.Q4_K_S.gguf | Q4_K_S | 6.63GB |
| caliburn-12b.Q4_K.gguf | Q4_K | 6.96GB |
| caliburn-12b.Q4_K_M.gguf | Q4_K_M | 6.96GB |
| caliburn-12b.Q4_1.gguf | Q4_1 | 7.26GB |
| caliburn-12b.Q5_0.gguf | Q5_0 | 7.93GB |
| caliburn-12b.Q5_K_S.gguf | Q5_K_S | 7.93GB |
| caliburn-12b.Q5_K.gguf | Q5_K | 8.13GB |
| caliburn-12b.Q5_K_M.gguf | Q5_K_M | 8.13GB |
| caliburn-12b.Q5_1.gguf | Q5_1 | 8.61GB |
| caliburn-12b.Q6_K.gguf | Q6_K | 9.37GB |
| caliburn-12b.Q8_0.gguf | Q8_0 | 12.13GB |
license: mit library_name: transformers model-index:
This model is a 12 billion parameter language model created by merging multiple existing models using the MergeKit library. It is designed for general text generation tasks.
This is a large language model with 12 billion parameters, created by merging multiple pre-existing models using the MergeKit library. The model is based on the transformer architecture and is fine-tuned for general text generation tasks.
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 22.68 |
| IFEval (0-Shot) | 35.76 |
| BBH (3-Shot) | 35.64 |
| MATH Lvl 5 (4-Shot) | 9.67 |
| GPQA (0-shot) | 11.52 |
| MuSR (0-shot) | 13.78 |
| MMLU-PRO (5-shot) | 29.72 |
This model can be used for various natural language processing tasks, including:
The model can be fine-tuned for specific tasks or domains to improve performance on targeted applications.
This model should not be used for generating harmful, biased, or unethical content. It should not be relied upon for critical decision-making without human oversight.
Users should be aware of the model's limitations and potential biases. It's recommended to use the model with appropriate content filtering and human oversight, especially for public-facing applications.
Use the following code to get started with the model:
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tokenizer = AutoTokenizer.from_pretrained("./models/12b-merged")
model = AutoModelForCausalLM.from_pretrained("./models/12b-merged", torch_dtype=torch.float16).to("cuda")
prompt = "Your prompt here"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs.to("cuda"), max_new_tokens=100)
result = tokenizer.batch_decode(outputs, skip_special_tokens=True)
print(result)