Downloads · 30 days
22
14% of all-time downloads
Rocinante-X-12B-v1-GPTQ
tacodevs/Rocinante-X-12B-v1-GPTQ
Rocinante-X-12B-v1-GPTQ is a machine learning model from tacodevs. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for auto-gptq. The card lists the license as apache-2.0.
GPTQ quantized version of TheDrummer/Rocinante-X-12B-v1.
All-time downloads
159
Public
Repo size
8.4 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors8.4 GB · 100%
At a glance
- Library
- auto-gptq
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Apr 3, 2026
- Updated
- Apr 3, 2026
- SHA
- ae0854be
Base models
Model card
From the Hugging Face model README
Rocinante-X-12B-v1-GPTQ
GPTQ quantized version of TheDrummer/Rocinante-X-12B-v1.
Quantization Details
| Parameter | Value |
|---|---|
| Bits | 4 |
| Group Size | 128 |
| Desc Act | False |
| Format | Safetensors |
| Calibration | WikiText-2 (128 samples) |
| Original Size | ~24 GB (BF16) |
| Quantized Size | ~5.6 GB |
About the Original Model
Rocinante-X-12B-v1 is a 12B parameter model by TheDrummer, fine-tuned from Mistral-Nemo-Instruct-2407 for creative writing, roleplay, and entertainment. It prioritizes creativity and usability over pure alignment.
Key strengths:
- Pleasant, compelling prose and storytelling
- Strong instruction adherence
- Handles complex themes maturely
- Supports
<thinking></thinking>tags - Chat template: Mistral v3 Tekken (recommended)
Usage
With vLLM
python -m vllm.entrypoints.openai.api_server \
--model Irvollo/Rocinante-X-12B-v1-GPTQ \
--quantization gptq \
--dtype float16
With transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"Irvollo/Rocinante-X-12B-v1-GPTQ",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Irvollo/Rocinante-X-12B-v1-GPTQ")
Credits
- Original model by TheDrummer
- Quantized by Irvollo
- Library
- auto-gptq
- Type
- mistral
- License
- apache-2.0
- Created
- Apr 3, 2026
- Updated
- Apr 3, 2026