Downloads · 30 days
155
7% of all-time downloads
mindrage/Manticore-13B-Chat-Pyg-Guanaco-GPTQ-4bit-128g.no-act-order.safetensors
Manticore-13B-Chat-Pyg-Guanaco-GPTQ-4bit-128g.no-act-order.safetensors is a text generation model from mindrage. Use it when you need the model to write or continue text. It is set up for transformers.
--- Manticore-13b-Chat-Pyg by openaccess-ai-collective with the Guanaco 13b qLoRa by TimDettmers applied through Monero, quantized by mindrage, uncensored
Downloads · 30 days
155
7% of all-time downloads
All-time downloads
2.2K
Public
Repo size
7.3 GB
Likes
14
Public
Click a slice to open those files.
.safetensors7.3 GB · 100%
From the Hugging Face model README
Manticore-13b-Chat-Pyg by openaccess-ai-collective with the Guanaco 13b qLoRa by TimDettmers applied through Monero, quantized by mindrage, uncensored
Quantized to 4bit GPTQ, groupsize 128, no-act-order.
Command used to quantize: python3 llama.py Manticore-13B-Chat-Pyg-Guanaco-GPTQ-4bit-128g.no-act-order.safetensors c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors
The model seems to have noticeably benefited from further augmentation with the Guanaco qLora. Its capabilities seem broad, even compared with other Wizard or Manticore models, with expected weaknesses at coding. It is very good at in-context-learning and (in its class) reasoning. It both follows instructions well, and can be used as a chatbot. Refreshingly, it does not seem to insist on aggressively sticking to narratives to justify formerly hallucinated output as much as similar models. It's output seems... eerily smart at times. I believe the model is fully unrestricted/uncensored and will generally not berate.
Presumably due to the very diverse training-data the model accepts a variety of prompting styles with relatively few issues, including the ###-Variant, but seems to work best using:
user: "USER:" - bot: "ASSISTANT:" - context: "This is a conversation between an advanced AI and a human user."
Turn Template: <|user|> <|user-message|>\n<|bot|><|bot-message|>\n
Settings that work well without (subjectively) being too deterministic: